Comments on: Manifests for exposing the structure of a Composite Media Resource

By: silvia

silvia — Mon, 11 Jan 2010 19:59:46 +0000

@giander That’s helpful, thanks!

By: giander

giander — Mon, 11 Jan 2010 18:14:29 +0000

@Silvia

“alternate group” is used in order to switch between tracks, such as when you have a podcast with multiple subtitles or audio tracks. You can modify its settings if you go to QuickTime Pro > Show Movie Properties. Dumpster for Mac is a very good program to analyze the structure of mp4 files.

By: silvia

silvia — Mon, 28 Dec 2009 10:48:14 +0000

@Davy

Now, that’s very interesting indeed. I only found reference in some papers on the “alternate group”, but it seems to be possible to use it to allow the user to switch between alternative tracks. I wonder if it is in practical use! But it certainly is a good idea and should be capable of encoding most of the ROE style specification. Very interesting idea!

By: Davy Van Deursen

Davy Van Deursen — Mon, 28 Dec 2009 10:16:26 +0000

Within the ISO Base Media File Format (which forms the basis for MP4, 3GPP, …), there exists a concept called ‘alternate group’. More specifically, for each track, you can specify the group it belongs to: (copy-paste from the spec)

“alternate_group”: is an integer that specifies a group or collection of tracks. If this field is 0 there is no information on possible relations to other tracks. If this field is not 0, it should be the same for tracks that contain alternate data for one another and different for tracks belonging to different such groups. Only one track within an alternate group should be played or streamed at any one time, and must be distinguishable from other tracks in the group via attributes such as bitrate, codec, language, packet size etc. A group may have only one member.

By: silvia

silvia — Mon, 28 Dec 2009 09:53:51 +0000

@Reimar

It seems the program specific information in MPEG allows you to list the different programs available (in parallel or sequential) WITH all their respective tracks. That last part is indeed interesting. However, I haven’t found a way to describe conditions between tracks – which tracks basically belong together from a language or accessibility POV.

I guess, maybe my model is too broad and unrealistic. In reality, a video is only made for one language. Then the tracks inside it must contribute on that basis, i.e. contribute subtitles in other languages, contribute captions in that language, contribute sign language tracks that are typical for the area, contribute an audio description in only the main language.

Then, when a dubbed version is created, a new resource is created that has again all the respective tracks as part of it.

This will simplify the problem. I’ll have to think some more about it. Thanks.

By: Reimar

Reimar — Mon, 28 Dec 2009 08:53:53 +0000

Hmm, seems I might have had my facts wrong, I haven’t found anything for Matroska so far.
However for MPEG there is this: http://en.wikipedia.org/wiki/MPEG_transport_stream#PAT
It is mostly used for broadcast because there are multiple TV programs on one frequency, and this allows separating them out.
It allows grouping stream IDs along with a bit of metadata.

By: silvia

silvia — Mon, 28 Dec 2009 01:13:11 +0000

@Reimar

Thanks for your comment.

Do you have a link to the MPEG or Matroska programs? I wasn’t able to find that. I know there are program streams in MPEG, but that’s just an encapsulation format for MPEG streams. Matroska describes tracks very well, but I couldn’t find a description for how to choose which tracks to display together.

By: Reimar

Reimar — Sun, 27 Dec 2009 12:07:05 +0000

To “which tracks should be displayed together with another track choice may be a good idea”, MPEG and Matroska etc. use so-called “programs” for that, maybe that is worth imitating?
As for sign language, I think there really should be a way to specify a video track with the sign language display already integrated and one with only the sign language contents.
Maybe it is even possible that some time in the future useful sign language can be given via a kind of subtitle track that only describes the gestures (not that likely I admit).

By: silvia

silvia — Wed, 25 Nov 2009 13:36:52 +0000

@Davy

Yes, indeed, Microsoft’s client-side manifesto is well explained here: http://msdn.microsoft.com/en-us/library/ee673438.aspx . It actually also contains all the data from the server manifest, but additionally all the chunking (or “fragment”) information. Thus, it actually exposes binary blocks that can be addressed. Very interesting!

By: Davy Van Deursen

Davy Van Deursen — Wed, 25 Nov 2009 12:56:23 +0000

Hi Silvia, interesting post, thanks. Just a small comment regarding the server manifest of Microsoft’s Smooth Streaming technology. Microsoft only uses this manifest on server-side, to interpret incoming requests. The client (i.e., the Silverlight player) uses a so-called client manifest, which is based on a proprietary XML format, and is thus not aware of the SMIL-based server manifest. But as you said, the information contained in the server manifest is very similar to what we need here and could also be useful on client-side.