Comments on: Audio Track Accessibility for HTML5

By: silvia

silvia — Wed, 01 Dec 2010 03:51:11 +0000

@megas you get the video’s current playback position through video.currentTime and you set the audio’s playback position to the video’s position with audio.currentTime=video.currentTime . As simple as this.

By: megas

megas — Tue, 30 Nov 2010 20:37:43 +0000

I don’t understand how to manage audio track. I guessed that by video timeout I extract the time witch video is now playing, but how to extract audio timeout and how to correct audio playback?

By: silvia

silvia — Sat, 17 Apr 2010 08:43:14 +0000

@jb As far as I know on the iPad only MPEG-4 video in Safari will work – Flash is definitely not supported. But I don’t own an iPad, so cannot really help.

By: jb

jb — Fri, 16 Apr 2010 20:47:46 +0000

Thanks very much for this I am trying to figure out how to do an animated film that will run native on the ipad (the client says that flash won’t work). Your lag catchup suggestion was superb. I have to do an arrow effect.

By: silvia

silvia — Mon, 15 Feb 2010 11:24:37 +0000

@Jack No need to be defensive about SMIL. I think SMIL did a great job in specifying synchronisation issues between all kinds of different input media for a multimedia presentation that can take on all sorts of compositions. It’s however not what the media elements in HTML5 need, since there is only one timeline and everything runs along that timeline.

I would, btw, consider introducing a element into HTML5 to group such independent media resources along a single combined timeline, if that was something that I thought HTML5 was ready for yet. I don’t think so, but I wanted to experiment with HTML5 and JavaScript to see if it was at least possible to imitate that functionality with existing HTML5 elements and JavaScript APIs. It is, alas not perfectly.

Just as SMIL has to rely on the underlying media framework to get this right, an introduction of something like a element would need to rely on the media framework to do this, too. It is particularly hard to do with media coming from different servers, since you also have to deal with network effects. So, I can’t see this happening in HTML5 in the near future.

By: Jack Jansen

Jack Jansen — Mon, 15 Feb 2010 09:57:27 +0000

Silvia,
I know I’m starting to sound like a broken record:-) but it’s precisely to solve issues like this that SMIL got underway, 13 years ago. The syncBehavior/syncMaster/syncTolerance allow the document author fine-grained control over whether media items should be synchronised or not, and how tight that synchronisation should be.

Of course, you still rely on the underlying engine to actually implement the syncing correctly, but at least the web page author doesn’t have to add tricky javascript to resync things.

By: Weekend Gazette – Link Collection for February 14 at STC AccessAbility SIG

Weekend Gazette – Link Collection for February 14 at STC AccessAbility SIG — Sun, 14 Feb 2010 17:04:35 +0000

[...] the same vein, we have the article about audio track accessibility for HTML5 by Silvia Pfeiffer. Across the internets comes the message that technical communication is moving [...]

By: silvia

silvia — Sat, 13 Feb 2010 04:24:27 +0000

The JavaScript resync makes sure that the media framework below is told to align the two based on playback time. This is at a higher level than the media framework and will be slightly less accurate, but will not be basically different to having them played back in sync by the media framework itself.

If you run pulseaudio or alsa underneath, these tell the media framework (and thus the JavaScript) what time the device has managed to playback. So, you really want to use the audio as the synchronising timeline and align the video to it (which then would also lead to a small pause in the video). With multiple audio tracks, that becomes almost impossible indeed.

It would be nice to build synchronisation ability into the browser so the browser can have the media framework do the synchronisation without the browser having to step in. But from talks with several browser vendors, this is a features that is not going to be implemented for a while. So, I have tried to find out how limited the use of JavaScript would be for this purpose and for some use cases it will work fine, while for others it will indeed be difficult to do well.

By: Jeremy

Jeremy — Sat, 13 Feb 2010 04:02:02 +0000

Yes, it’s a massive delay. My Bluetooth headset also comes with an analogue-to-Bluetooth dongle which lets me use it with a normal audio jack, e.g. MP3 players, etc. If I use that with my computer’s audio output, then lip sync is way off.

However, when pairing the headset with my computer directly (no analogue

By: silvia

silvia — Sat, 13 Feb 2010 03:56:52 +0000

@Jeremy that’s quite a massive delay for a headset. How do you get lipsync with a normal video? In any case: the javascript is actually quite flexible and you can hack it together yourself in the way you need it.