Thanks for your thoughts!

In the picture, I would think that everything is “continuous” media, since everything has a relationship to the main video timeline. I agree that DFXP as aligned to a video is also “continuous” media, even if it might have gaps. It belongs into the “time-aligned text” category. “Discrete” media would only be media that has no relationship to time and just exist. BTW: video and in particular audio can also have non-constant frame rate, so it’s not that different after all.

I do believe that DAISY – just like SMIL – consists of presentations with multiple timelines and it is what DAISY needs. It would be interesting to see if the DAISY specification – just like the SMIL specification – could be implemented using HTML5 and JavaScript. With all the modern extensions of HTML5, it may well be possible to do so. I am curious as to what the limitations would be. Hierarchical navigation will certainly be implementable with Media Fragment URIs, HTML and JavaScript. Just look at the linking that is done in http://oggify.com/videos/7 – an experimental site of ours.

We’re not fully there with the standard and not fully there with implementations, but we are certainly getting closer and I hope it will all be resolvable within the next 6 months.

I always find it interesting to compare terminologies. In the Media Fragment diagram, there is a type of track composed of “discrete” media, as per the SMIL definition. By contrast, some tracks are based on “continuous” Media.

http://www.w3.org/TR/SMIL3/smil-timing.html#Timing-DiscreteContinuousMedia

http://www.w3.org/TR/SMIL3/smil-extended-media-object.html#smilMediaNS-Definitions

Time-aligned text, in particular W3C-TimedText-AFXP-DFXP-TTML (or whatever it ends-up being known as) is an interesting case. The simple internal timing model means that such media track can be seen as a linear (i.e. from a defined “begin” to a known “end” without intrinsic discontinuity) time-based resource (i.e. not frame-based) that returns a visual depending on essentially 2 input parameters: a time offset and the coordinates+dimensions of a host rendering area. Both parameters get resolved against particular reference points, such as the wallclock for time or a video rectangle for the space domain. Generally-speaking, this kind of “text media” can actually be thought of as a motion-pictures track, with similar content addressing behaviors (i.e. “give me a time offset and I will return a frame to render, in a deterministic/reproducible manner”). Contiguous “frames” of static text do not necessarily have identical durations (i.e. non-constant frame rate), so the analogy with video ends here (in SMIL, the “textstream” element is recommended instead of “text”). However, this model is also compatible with transitions (visual animation from one internal frame to the next, with no impact on the execution timeline), although DFXP only offers roll-up captioning, whereas SMIL offers much more with its library of transitions.

In my line of work (DAISY Digital Talking Books), text is primarily a static resource that is hosted within a SMIL presentation so that an audio narration can be finely-synchronized with its corresponding textual source (paragraph, sentence, word, etc.). This model also works with the reverse scenario where the text is a transcript of an original audio track, so this may also be suitable for the video+captions use-case. DAISY offers an additional structure on top of the timing and synchronization layer: hierarchical navigation, slightly richer to that of an ePUB book (table of content, page numbers, footnotes, sidebars, etc.). In this context, cross-publication navigation (i.e. outwards hyper-linking) is not a top-priority, however bookmarks and annotations are must-haves (inwards references to points or ranges within a given distribution unit).

Web-browsers offer off-the-shelf implementations of many crucial components (text rendering, navigation, etc.), so we’re keeping a close eye on the developments of HTML5 and Media Fragments. Let’s hope it doesn’t take another decade for these to be ratified by standardization bodies

Cheers, Daniel

Yes, DFXP is one such format that supports outgoing hyperlinks. Also, DFXP has been mapped into FLV and MPEG4 (and with it QuickTime and 3GPP), so it does in fact have quite some support for the binary formats, too. It is definitely one format that turns media resources into hypermedia. SRT doesn’t provide for such. I would, however, not yet go as far as calling DFXP the one format in use by everyone – the spread just isn’t there yet and the open source tools support.

When I talk about “separating the media resource concept from actual files” I mean what we are already talking about with captions and the source element: the tracks of a media resource may actually be in several files. The actual markup inside HTML already has an indirection, if you so want: it points to the actual files that contribute to the resource.

If you really want to compare it to SMIL, then yes, we already have similarities in HTML5 to SMIL, where we reference the contributing resources. But other than that, we need to be careful about comparing to SMIL, since SMIL allows for so many complexities that go far beyond a single linear timeline and my post is trying to clarify that this is not what we want or can work with in HTML5.

Naive question: You wrote:
“It seems to be clear that hyperlinks out of media files will come from some type of textual track. But a standard format for such time-aligned text tracks doesn’t yet exist. This is a challenge to be addressed in the near future.”

Does DFXP not allow already to have outbound links in the caption? My understanding was that DFXP provides the ability to have rich (stylish) text instead of plain text a la SRT. Is it restricted to style information (color, font) or can you also include hyperlinks in DFXP?

What does it mean, concretely, to “separate the media resource concept from actual files”? Adding a level of indirection? SMIL?