There are use cases for both, text inside a video/audio file, and text outside, but related. For most Web developers, outside is in fact a lot more sensible since it’s easier to update such files.

I am neither trying to avoid a patent reality nor trying to hack around issues. Even if there existed only a single format in which we encoded and encapsulated audio and video, I would still propose to use both: in-stream and external (out-of-band) time-aligned text.

For Web pages we already have the reality that it consists of many files that together create a consistent presentation. It’s not a problem – we have zip/tar files and many other means to solve this issues.

In fact, I think it will be even less of a problem for video, since a server can provide the additional service of embedding a text files (or, in fact, text from a database) inside a video file upon download, should that be desirable. Also, it is possible to download the video and the associated text files as package. If the text “mysteriously disappears”, it’s a feature/bug of the Website rather than a fundamental design issue.

Your idea is a hack around the stupid patent driven reality.

Your idea is well designed but I am afraid it also reenforces the reality. Why should people care to use a open standard for video with text embeding abilities if they can just use your hack?

People saving the video will end up with a useless file not containing the text they might need. And no indication of that before they save it. The text will mysteriously disappear.

That's right. I was also trying to say (probably not very clearly!) that it would be good to be able to listen for enter and leave events as well as being able to to attach event handler callbacks to onenter and onleave — if only to encourage coders to move JavaScript out of HTML.

Thanks for the extensive feedback – I’ve added it to https://wiki.mozilla.org/Accessibility/Experiment2_feedback .

Re SMIL: yes, it is oriented towards multi-media presentations where the timeline is in control – that is not how Web pages work, which are essentially static text content enriched with interactive and media elements. Thus the poor fit.

Re live timed events: I think it is possible to point the video src url to a live broadcast, which then gets updated continuously. It might make sense to turn off the controls for such an element. I also don’t see a problem in attaching a subtitle file that is continuously updated in an itext element to such a live video source. The text could continue to be pushed/pulled. With javascript, it would also be possible to continue pulling other content, such as images or other text.

Re timed events: I assume you are saying that you like the callback methods that were introduced on the timed text segments, since they allow you to do such timed updates? I can see how that would be possible – nice ideas!

{
“start”: 62.13,
“end”: 65.29,
“sender”: “video#myVideo”
“event”: “timeupdate”
“receiver”: “div#subtitle”
“type”: “CSS”,
“value”: {“font-weight”: “bold”}
}

As for linking to subtitles in a HTML head element: that won’t work when you have multiple video elements on the web page. You really do need a solution that clearly associates with a particular video element.

“category” is very different to “role” – it is the category of time-aligned text we are talking about and there is a limited list part of the spec.

“name” could be replaced by “title” or something else – I am not particularly fussed about this though I needed it as an attribute rather than as content model, which would have been more obvious.

I am also consciously refraining from re-implementing SMIL. I do not want the full complexity of the “seq” and “par” elements. Also, the “text” or “ref” elements do not compare to “itext” which references a particular type of interactive text files similar to how “img” references particular types of image files.

Further, HTML doesn’t do namespaces, so every adoption from another standard would need to be replicated into HTML anyway. And since there is not an exact match between the needs that itext and itextlist express and those provided by other specs, I’d rather avoid that complexity.

The important thing here is though that we have looked at existing syntaxes and have learnt from them, so even through there is no direct re-use, there is indeed conceptual re-use and learnings.