Anyway, I’m not a media specialist, but wanted to point out to your readers (it’s probably obvious to you already) that this notion of using URI fragments and/or HTTP Range headers to address portions of a resource will have implications outside the media realm.

If ’smart’ browsers start converting well-defined URI fragments into alternate-units Range headers (and ’smart’ proxies start caching accordingly) then the gate is opened for this same technique to be used for addressing sub-portions of JSON or XML resources in web services.

Here’s one proposal that uses URI fragments and Range/Content-Range/etc. (albeit separately) in JSON resources:
http://groups.google.com/group/restful-json/browse_thread/thread/6beef1547c11426a

Best regards to you and the WG.

Ok good, that’s what I was thinking of too.
I don’t think of chain segments as separate files. Rather there are two clocks, one for the complete stream, and one within each segment. I would suggest the units in the range request measure time relative to the start of the resource and contain everything available in that resource, regardless of how it is constructed. And that the server was doing shifting regardless, since at least for ogg files, the internal timestamps don’t necessarily start anywhere near zero? Conceptually the same addressing should apply equally well to a playlist, e.g. http://example.com/latest.xspf#t=1h23m16.12s Of course, I don’t know how you’d implement that today with a cross-faded mp3 playback.

Re: sample-accurate returns, this is my point. If you return byte ranges which can be concatenated to duplicate the origin’s resource, the time ranges are not contiguous. If the time ranges are contiguous, the byte ranges overlap. I don’t see a way to resolve this, though I would be happy if it is possible.

Re: file chopping
The file chopping is of course only virtual – there aren’t different pieces of content on the server. But otherwise, yes, that’s how it works.

Re: headers
That is a very good question and hasn’t been resolved yet. There may be a special additional HTTP request for just the headers that relate to the requested time range for file formats that require such. As for chained Ogg files – these are really multiple files and I am not sure how to handle them properly, because in theory time also resets at the segment boundaries. What do you suggest?

Re: sample-accurate
The idea is to return as much data as is necessary to decode the requested time range properly. If that requires more packets from before and after the current time range, then that’s the way it goes. It should definitely be sample-accurate.

Re: mux granularity
If the files were not created for streaming in the first place, then you cannot return byte ranges but have to return the full file. That’s just how it goes.

Re: decodable stream
That is also a very good question that we have not fully explored yet. The answer depends on the use cases that we want to support. If it’s just about buffering strategies inside media players, then it doesn’t need to be a fully decodable stream. If it’s about sharing and reusing the media fragment, then it should be a fully decodable stream.

In Annodex, we assumed the need for fully decodable stream, which is why oggzchop and mod_annodex do it in this way. But the current Firefox plugin deals with byte range requests without bothering to get a corrected header – the data is for internal buffering purposes only. If as mentioned above the header data is requested in a separate roundtrip, then it could be left up to the user agent to request it if needed, or not to. The reconstructed stream should in any case be data-equivalent, but not necessarily be header-equivalent to the original resource.

Re: byte range reconstruction
Byte range reconstruction was only discussed as part of the requirements for Web proxies. It is indeed planned not to allow specification of more than one time range in a media fragment URI (any subsequent ones will be ignored). If the user agent requires several temporal fragments, it should indeed use playlists.

So the idea is to chop the file up into byte ranges, but report the corresponding time range with the returned piece? In this scheme the boundaries would start with a video keyframe and, presumedly, go to the next?

A number of points remain unclear to me. How do I get the headers so I can decode the stream? Do I first do a magic Range: time -0m0s request to get those? What about Apple .caf where the metadata can be at the end? What if it’s a chained Ogg stream, how do I get the headers for the current segment if they’re not at the start of the timeline?

Also, this cannot be sample-accurate: the preroll required by the vorbis codec, or the overlap of open-GOP structures in Dirac and mpeg video mean I won’t be able to start or finish decoding exactly at the time-segment boundaries, although complete playback is possible after adjacent segments are concatenated. Is that acceptable?

What about files with a large mux granularity, like MXF or (badly constructed) avi? What do I do if the smallest byte range I can return which contains the requested temporal range is the whole file?

For these reasons, I thought it made more sense to accept the overhead of always sending a decodable stream. This is just what ogg-chop currently does, by pre/post-pending the required extra headers and data. By marking the target temporal segment in the skeleton header it’s possible to concatenate these things and have playback be identical to the original timeline. But the resource so constructed is not the same stream as the original resource.

For formats without Ogg’s legal concatenation, I’d have to know how to merge adjacent requests. More likely, I’d just make a playlist with all the pieces. That’s fairly simple to implement at least; I’d like to compare that to any proposals that would let me reconstruct a standalone file by byte range concatenation.

I share your concerns about a time range request and the possibilities of getting it accurate. We’ve had a massive amount of discussion in the working group on exactly this topic. There are currently two approaches under discussion.

The first one is similar to the temporal URI specification that we in the Annodex community proposed a long time ago: http://annodex.net/TR/draft-pfeiffer-temporal-fragments-03.html#anchor8. It goes along the lines of having two roundtrips, where the first one tells the server which time range the client is looking for and it returns the right byte range for that, so in the second round trip, the client just does a byte range request for the correct data.

In the working group we are however trying to avoid more than one roundtrip to get the first data. Thus, an alternative proposal is to create time range requests, where the client asks the server for a time range, and the server replies with the data and the actual time range it is providing. Then the client can decode the data and play back only the actual time range that was requested.

The concatenation problem that you are referring to in your comment is then solved through the adapted time ranges that the server replies with. It will only have a finite set of actual time points for which it can return data (start/end). This is because compressed codecs devide time up into packets. The expectation is that because of this “discretisation” of time, concatenation will work.

Using your example: the first request for 5 seconds at 24m16s would return 23m24s-24m21, the second request for five more at 24m21s would give you 24m21s-24m26s, which you can safely concatenate.

Do you think that can work?

Also, re accuracy of time resolution: thanks for the input – point taken.

The examples above all just reference fragments to the nearest seconds. That’s insufficient accuracy. If I want to pick out a particular line in a song, or the start of a scene in a movie, one second resolution means there would be a few extra words, or a few frames of the previous scene at the beginning, quite distracting.

For video, something like smpte timecodes, which can refer to frames or milliseconds would be good enough. I don’t know if milliseconds would be good enough for audio people. Certainly for other time-based data I can imagine wanting microseconds for something like network event logging, which means nanoseconds wouldn’t be a bad option.

I really like the idea of the client turning the fragment request into something more efficient. This leaves user agents the freedom to just download the whole file and then present a portion, the way fragment references to html anchors work. More sophisticated agents can use whatever tricks they know for the protocol in question to negotiate transfer of just the region of interest.

What surprised me is that you’re considering alternate units for HTTP Range requests as a mechanism for this. I looked at this a few years ago, trying to solve the seek-latency problem but abandoned it because the semantics just aren’t clear.

There are a couple of issues here. The simplest is that byte range requests commute and time range requests do not. What I mean is, if you request bytes 0-999 of a file, then bytes 1000-2000, and concatenate the two, you have exactly what you’d get if you requested bytes 0-2000. Multimedia files can’t be done this way. Digital media is discrete, and of course the playback engine can round the requested point on the timeline to the nearest sample and start playback there, but it is not stored in sample units. Generally samples are grouped into blocks and the blocks are compressed with prediction from previous blocks. So to begin playback at 24m16s, the server might need to start returning data from 23m24s onward, and the user agent would have to feed all that data to the decoder, discarding the output until it got to 24m16s.

But that means requesting 5 seconds at 24m16s and five more at 24m21s gives you two files, one of which covers 23m24s-24m21 and one that covers 23m24s-24m26s. You can’t meaningfully concatenate them, you can only merge them, which requires detailed knowledge of the media format.

A cache can’t do as well here unless it understands enough of the file format to re-merge the temporal ranges and serve new temporal Range requests out of that.

If the server is just returning subsets of a static file though, what about answering a Range request in time units with a Content-Range in byte units? Then caches could still do something useful. Unfortunately I don’t see how this can work either. The problem is again that the server has to “round” the Range request to includes a safe restart point for the decoder. If the client gets back a byte range set instead then it knows how to reassemble the file, but it no longer knows what temporal range it’s received. In some cases it might be able to work out where the requested range lies based on internal timestamps, but that isn’t possible in general. There may be no absolute timestamps, or the requested interval may need to be encoded in the stream itself, in which case the returned result isn’t a simple byte Range request result anyway.

I’d love to hear if there’s a way around these issues. As far as I got it seemed like this is a format-specific extension, which didn’t justify a new Range unit. I’m having trouble seeing how this is better than just using a query uri like in the annodex proposal.

1) The browser will simply add the time range request to *every* URI that it is requesting over HTTP. The server will either ignore that parameter or reply to it appropriately if it is indeed a media resource and the server is able to provide the time range. Then it adds some other HTTP header to the reply and the user agent will know from this whether it is a media resource and it has received the subpart of a media file and has to start display of a audio or video file – or whether alternatively it is a totally different page and it needs to e.g. jump to an anchor of that name.

2) Alternatively we discussed whether we should include into the URI scheme a means to give a hint to the user agent that this is expected to be a video and therefore the time range request has to be added. This can be achieved, e.g. by adding an identifying character directly behind the “#” character. In particular if we choose something that is not allowed as a character in an element id, we will never clash with anchor offsets of HTML pages. Example characters are: “;”, “:”, “@”, “!”, “$” and many more (you can find out by comparing the allowed characters in a URI fragment at http://www.ietf.org/rfc/rfc3986.txt with the ID and NAME token restriction at http://www.w3.org/TR/html4/types.html.

Either way should really work because the HTTP protocol prescribes that parameters that cannot be understood should be ignored.

http://example.com/foooooo#t=24m16s-30m12s

means jumping to anchor “t=24m16s-30m12s” on a page “foooooo” or downloading a segment of a file “foooooo”. Also, how can users predict this? I might be missing bits, can you shed some light or point me to somewhere with answers?