Jumping to time offsets in HTML5 video
For many years now I have been progressing a deeper view of video on the Web than just as a binary blob. We need direct access to time offsets and sections of videos.
Such direct access can be achieved either by providing a javascript interface through which a video’s playback position can be controlled, or by using URLs that directly communicate with the Web server about controlling the playback position. I will explain the approaches that can be applied on the HTML5 <video> tag for such deep video interaction.
Controlling a video’s playback with javascript
currentTime
Right now, you can use the video element’s “currentTime” property to read and set the current playback position of a video resource. This is very useful to directly jump between different sections in the video, such as exemplified in the BBC’s recent R&D TV demo. To jump to a time offset in a video, all you have to do in javascript is:
var video = document.getElementsByTagName("video")[0];
video.currentTime = starttimeoffset;
timeupdate
Further, if you want to stop playback at a certain time point, you can use another functionality of the HTML5 <video> tag: the “timeupdate” event:
video.addEventListener("timeupdate", function() {
if (video.currentTime >= endtimeoffset) {
video.pause();
}}, false);
When the “timeupdate” event fires, which is supposed to happen at a min resolution of 250ms, you can catch the end of your desired interval fairly accurately.
setTimeout / setInterval
Alternatively to using the “timeupdate” event that is provided by the <video> tag, there is always the possibility of using the javascript “setTimeout” or “setInterval” functions:
setTimeout(video.pause(), (endtimeoffset - starttimeoffset)*1000);
The “setTimeout” function is used to call a function or evaluate an expression after a specified number of milliseconds. So, you’d have to call this straight after starting the playback at the given starttimeoffset.
If instead you wanted something to happen at a frequent rate in parallel to the video playback (such as check if you need to display a new ad or a new subtitle), you could use the javascript setInterval function:
setInterval( function() {displaySubtitle(video.currentTime);}, 100);
The “setInterval” function is used to call a function or evaluate an expression at the specified intervall. So, in the given example, every 100ms it is tested whether a new subtitle needs to be displayed for the video current playback time.
Note that for subtitles it makes a lot more sense to use the existing “timeupdate” event of the video rather than creating a frequenty setInterval interrupt, since this will continue calling the function until clearInterval() is called or the window is closed. Also, the BBC found in experiments with Firefox that “timeupdate” is more accurate than polling the “currentTime” regularly.
Controlling a video’s playback through a URL
There are some existing example implementations that control a video’s playback time through a URL.
In 2001, in the Annodex project we proposed temporal URIs and implemented the spec for Ogg content. This is now successfully in use at Metavid.org, where it is very useful since Metavid handles very long videos where direct access to subsections is critical. A URL such as http://metavid.org/wiki/Stream:Senate_proceeding_02-13-09/0:05:40/0:47:29 work well to directly view that segment.
More recently, YouTube rolled out a URI scheme to directly jump to an offset in a YouTube video, e.g. http://www.youtube.com/watch?v=PjDw3azfZWI#t=31m09s. While most YouTube content is short form, and such direct access may not make much sense for a video of less than 2 min duration, some YouTube content is long enough to make this a very useful feature.
You may have noticed that the YouTube use of URIs for jumping to offsets is slightly different to the one used by Metavid. The YouTube video will be displayed as always, but the playback position in the video player changes based on the time offset. The Metavid video in contrast will not display a transport bar for the full video, but instead only present the requested part of the video with an appropriate localised keyframe.
Having realised the need for such URLs, the W3C created a Media Fragments working group.
Proposed Time schemes
For temporal addressing, it currently proposes the following schemes:
t=10,20
t=npt:10,20
.
t=120s,121.5s
t=npt:120,0:02:01.5
.
t=smpte-30:0:02:00,0:02:01:15
t=smpte-25:0:02:00:00,0:02:01:12.1
.
t=clock:20090726T111901Z,20090726T121901Z
If there is no time scheme given, it defaults to “npt”, which stands for “normal playback time”. It is basically a time offset given in seconds, but can be provided in a few different formats.
If a “smpte” scheme is given, the time code is provided in the way in which DVRs display time codes, namely according to the SMPTE timecode standard.
Finally, a “clock” time scheme can be given. This is relevant in particular to live streaming applications, which would like to provide a URL under which a live video is provided, but also allow the user to jump back in time to previously streamed data.
Fragments and Queries
Further, the W3C Media Fragment Working Group is discussing the use of both URI addressing schemes for time offsets: fragments (“#”) and queries (“?”).
The important difference is that queries produce a new resource, while fragments provide a sub-resource.
This means that if you load a URI such as http://www.example.org/video.ogv?t=60,100 , the resulting resource is a video of duration 40s. Since relates to the full resource, it is possible to expect from the user agent (i.e. web browser) to display a timeline of 60-100 rather than 0-40 – after all, the browser could just get this out of the URL. However, it is essentially a new resource and could therefore just be regarded as a different video.
If instead you load a URI such as http://www.example.org/video.ogv#t=60,100, the user agent recognizes http://www.example.org/video.ogv as the resource and knows that it is supposed to display the 40s extract of that resource. Using no special server support, the browser could just implement this using the currentTime and timeUpdate javascript functionality.
An optimisation should, however, be made on this latter fragment delivery such that a user does not have to wait until the full beginning of the resource is downloaded before playback starts: Web servers should be expected to implement a server extension that can deal with such offsets and then deliver from the time offset rather than the beginning of the file.
How this is communicated to the server – what extra headers or http communication mechanisms should be used – is currently under discussion at the W3C Media Fragments working group.
on August 20th, 2009 at 10:20 am
[...] http://blog.gingertech.net/2009/08/19/jumping-to-time-offsets-in-videos/ [...]
on August 20th, 2009 at 12:35 pm
Ginger,
Thanks for the post! I’ve been working with HTML5 for a few months and my application ties in closely with the timeupdate function. For an example, see:
http://www.ressq.com/videoTag/video.html
In the future, my application will support creating playlists of stats from multiple games. I’ll be interested to see where the fragments and queries spec goes in the future. Do you sit on the W3C Media Fragments working group?
Mike Ressler
on August 29th, 2009 at 6:12 am
@Micheal: yes, I do.
on January 20th, 2010 at 12:10 am
Hi !
I would like to be able to seek at one frame at a time.
This means that I need to have a timestamp in fraction of seconds.
Is this possible ?
Thanks in advance !
JM
on January 21st, 2010 at 9:20 pm
What do you need that accuracy for? In theory you can address such an accuracy, but in practice, your seeking will not be able to get you that much accuracy in general. So, I wonder what the use case is.
on January 21st, 2010 at 9:42 pm
Well, I would like to be able to do something like virtualDub or something like that. You say it is possible in theroy : how ?
And why couldn’t it be possible in practice ?
Thx !
on January 21st, 2010 at 10:03 pm
jmh: you can address e.g. t=5.678901234 which has a very high resolution. But any encoded file, such as a mp4 or ogg theora file will not be able to deliver that kind of resolution in the compressed domain. Even when decoded it, your accuracy will be limited by the sampling frequency of the video (and to a lesser extent of the audio) channel.
on January 21st, 2010 at 10:34 pm
The accuracy I am looking for is the framerate of the video. So something like 0.03s.
on January 21st, 2010 at 11:37 pm
My use case for something like this would be to step through sports videos for player review. I wouldn’t need audio at all. I’d just like to be able to allow my users to step frame by frame through the video.
I’m looking forward to the variable playback rate for this very reason. Frame by frame stepping would be useful as well.
on January 22nd, 2010 at 5:32 am
Stepping through a video is a feature of a player, so it’s up to the software to provide this. The URI addressing scheme can accommodate it. So, you’re free to implement a video player that provides that accuracy. Current browser implementations will do the best they can and with the “currenttime” attribute in HTML5 media you will be able to control it. You can write your own javascript to test this, btw. You can get inspiration from http://annodex.net/~silvia/itext/mediafrag.html . (Though: I believe there is a bug in there that ignores subseconds).
on January 25th, 2010 at 12:58 am
I just updated Firefox to 3.6 and now the javascript video.currentTime no longer works. It worked perfectly in 3.5.
I used the following code to start the video at a specific time that I calculated using javascript beforehand:
fallback
var video = document.getElementById(“myVideo”);
As I said, this worked fine in 3.5 but when I open the same html-file in 3.6, the video always starts playing at beginning and not at the time offset given in “TimeToJump”.
I also tried putting video.currentTime=TimeToJump into the on the bottom right under var=[...] but still, the video played from the beginning instead of the time offset.
The example given at http://annodex.net/~silvia/itext/mediafrag.html works in 3.6 so the browser seems to be capable of playing videos at a time offset, just not with the simple video.currentTime function in javascript. The code that is used for this example is way more complicated than what I used and what used to work in 3.5!
Here is the complete html-code http://pastebin.com/f40d55f69
Any help on how to get this to work as simple as possible with Firefox 3.6 (and all other browsers) would be awesome!
Thx a lot for the great information so far,
Thomas
on January 25th, 2010 at 11:25 am
@Thomas maybe the order in which the onload event is executed and the video load is executed has changed. I don’t actually know and it doesn’t actually matter. According to the HTML5 specification, you need to wait until the video is loaded, i.e. in loadedmetadata status before you can execute the currentTime attribute reliably. Otherwise it’s random what happens.
on January 25th, 2010 at 11:38 am
@Thomas, exactly to Silvia’s point, I use the loadedmetadata callback like so:
video.addEventListener(“loadedmetadata”, function () {
video.currentTime = newTime;
}, false);
And it works perfectly. I was having similar problems with Firefox 3.5 and found this solution with Silvia’s help. Good luck!
on January 25th, 2010 at 11:46 am
Thomas,
As of Firefox 3.6, we no longer fire the ‘load’ event for videos (bug 521359 at bugzilla.mozilla.org explains why), so the code you have in will never be executed. I suspect the other method you tried (setting currentTime after calling getElementById) tried to seek before the video had initialized (open the script error console via Tools->Error Console and look for an error to confirm this)–you need to wait until it’s safe to seek, e.g. by waiting for the loadedmetadata event.
on January 25th, 2010 at 11:58 am
@Thomas, you could try simply replacing the “onload” attribute with a “onloadedmetadata” – that might do the trick.
on January 25th, 2010 at 4:55 pm
Awesome! Thanks for the great help, onloadedmetadata instead of just onload worked fine! Great and thx again!
on January 25th, 2010 at 6:25 pm
The open media community rocks indeed
there’s help everywhere!
on February 5th, 2010 at 6:08 pm
[...] then perform actions (e.g. scripts) when you hit those points?” with taking a quick look at this link I don’t see why not? You certainly seem to be able to access the current time using [...]