HTML5 audio and video accessibility, internationalisation and usability

Silvia Pfeiffer

Xiph.Org, Vquence Pty Ltd, Mozilla Contractor

Overview

• Media accessibility requirements
• Current HTML5 media accessibility activities
• What is your itch?

HTML5 media elements

  <video src="elephant.ogv" poster="elephant.png" controls>Fallback</video>
  
  <audio src="chocolate_rain.ogg" controls>Fallback</audio>

Further attributes: autoplay, preload, loop

Markup: multiple alternative sources

  <video controls>
    <source src="video.webm" type="video/webm">
    <source src="video.ogv" type="video/ogg">
    <source src="video.mp4" type="video/mp4">
    Fallback
  </video>

Further attributes on <source>: media

Future state with video on the Web

Demos

Video & CSS

• http://www.zachstronaut.com/lab/isocube.html
• http://people.mozilla.com/~prouget/demos/mashup/video.xhtml

Video & SVG

• http://double.co.nz/video_test/video.svg
• http://people.mozilla.com/~prouget/demos/round/index.xhtml

Video & Canvas

• http://people.mozilla.com/~prouget/demos/green/green.xhtml
• http://people.mozilla.com/~prouget/demos/DynamicContentInjection/play.xhtml

More Demos

Video & Web Worker Threads & Canvas

http://hacks.mozilla.org/2009/06/connecting-html5-video/

Video & Synchronous Data Display in Canvas

http://people.mozilla.org/~blizzard/launch/

Accessibility Requirements

Vision-Impaired

• Audio Descriptions; Extended Audio Descriptions
• Texted Audio Descriptions (screenreader / braille)
• Clear Audio
• Structured Content Navigation
• Keyboard Access
• Time Scale Modification

More Requirements

Hard-of-Hearing

• Captions
• Extended Captions
• Sign-Language

Deaf-blind

• Transcripts

Foreigners / Adverse Environments

• Subtitles

Current Hacks

Subtitles

• http://people.mozilla.com/~prouget/demos/srt/index2.xhtml
• http://v2v.cc/~j/jquery.srt/

Chapters & Captions

• http://open.bbc.co.uk/rad/demos/html5/rdtv/episode2/index.html

Animated Lyrics with SVG

• http://svg-wow.org/audio/animated-lyrics.svg

Transcript

• Interactive Transcript

More Current Hacks

Integrated Proposal: has textual audio descriptions, captions, subtitles, chapters, lyrics

• http://annodex.net/~silvia/itext/elephant_no_skin_v2.html
• http://annodex.net/~silvia/itext/chocolate_rain.html

Current Technology Activities

1. Multitrack Video Support
2. External Text Tracks Markup in HTML5
3. External Text Track File Format
4. Direct Access to Media Fragments

1. Multitrack Media JavaScript API

  interface TimedTrack {
    readonly attribute DOMString kind;
    readonly attribute DOMString label;
    readonly attribute DOMString language;
    readonly attribute DOMString type;

    const unsigned short NONE = 0;
    const unsigned short LOADING = 1;
    const unsigned short LOADED = 2;
    const unsigned short ERROR = 3;
    readonly attribute unsigned short readyState;
    readonly attribute Function onload;
    readonly attribute Function onerror;

1. Multitrack Media JavaScript API (continued)

    const unsigned short OFF = 0;
    const unsigned short HIDDEN = 1;
    const unsigned short SHOWING = 2;
             attribute unsigned short mode;

    readonly attribute TimedTrackCueList cues;
    readonly attribute TimedTrackCueList activeCues;

    readonly attribute Function oncuechange;
  };

Proposed Track Kinds (roles)

Text tracks:

"captions", "subtitles", "descriptions", "metadata",
"chapters", "lyrics", "karaoke"

Video tracks:

"main", "alternate" (e.g. different camera angle),
"sign" (for sign language) 

Audio tracks:

"main", "alternate" (probably linked to an alternate video track),
"dub", "description", "music", "sfx" (sound effects) 

2. External text tracks Markup

  interface HTMLTrackElement : HTMLElement {
             attribute DOMString kind;
             attribute DOMString label;
             attribute DOMString src;
             attribute DOMString srclang;
             attribute DOMString type;
             attribute DOMString media;

    readonly attribute TimedTrack track;
  };

2. Example Markup

<video src="video.ogv">
  <track src="video_tad.srt" type="text/srt; charset='UTF-8'"
      srclang="en" kind="descriptions"></track>
  <track src="video_sub_en.srt" type="text/srt; charset='Windows-1252'"
      srclang="en" kind="subtitles"></track>
  <track src="video_sub_de.srt" type="text/srt; charset='ISO-8859-1'"
      srclang="de" kind="subtitles"></track>
  <track src="video_sub_ja.srt" type="text/srt; charset='EUC-JP'"
      srclang="ja" kind="subtitles"></track>
  </track>
</video>

3. File Format for External Text Track

• WebSRT
• TTML
• WMML

Comparison of Format Examples

Text Track Baseline Format Discussions will have to happen

4. Media Fragment Addressing

• http://www.w3.org/TR/media-frags/
• Demos for HTML5 media elements
• Firefox Plugin for demonstration

Next Steps

• continue development of standards
• discussion of baseline format
• implementation of best practices Web pages
• implementation of specs in Firefox
• address all requirements

Questions?