HTML5
Audio and Video

Using these exciting new elements in practice

Use your cursor keys to move forward and backward through the slides.

Best viewed in Firefox 4beta

1
Cross browser <video> element
              <video controls poster="HelloWorld.png" width="500">
                  <source src="HelloWorld.mp4" type="video/mp4">
                  <source src="HelloWorld.webm" type="video/webm">
                  <source src="HelloWorld.ogv" type="video/ogg">
              </video>
            
2
Cross browser <audio> element
              <audio controls>
                  <source src="revolve.mp3" type="audio/mp4">
                  <source src="revolve.ogg" type="audio/ogg">
              </audio>
            
3
Encoding

MPEG-4 H.264/AAC

              ffmpeg -i HelloWorld.dv \
                     -vcodec libx264 -vpre normal -vb 3000k \
                     -acodec libfaac -ab 192k \
                     -threads 0 HelloWorld.mp4
            

Ogg Theora/Vorbis

              ffmpeg2theora -o HelloWorld.ogv \
                            --videobitrate 3000 \
                            --audiobitrate 192 \
                            HelloWorld.dv
            

WebM VP8/Vorbis

              ffmpeg -i HelloWorld.dv \
                     -vb 3000k \
                     -ab 192k \
                     HelloWorld.webm
            
4
Fallback Considerations
  • legacy browsers:
    placed inside audio/video element
  • lack of codec support:
    use canPlayType() to check support
    Flash for Opera/Firefox
    Cortado for Safari/IE
  • iPhone / iPad: m3u8 for HTTP live streaming
5
CSS and <video> - samples
              border-radius: 15px;
              box-shadow: 4px 4px 4px gray;
              transition-property: all;
              transition-duration: 0.5s;
              transition-timing-function: linear;
              transform: rotate(43deg);
            
6
CSS and <video>
7
<video> and the JavaScript API
8
<video> and the JavaScript API - samples
              video = document.getElementsByTagName("video")[0];
              
              // functions
              video.load();
              video.play();
              video.pause();
              
              // properties
              video.currentSrc
              video.currentTime
              video.duration
              
              // events
              loadedmetadata
              timeupdate
              pause
              play
              ended
            
9
<video> and SVG
      <style>
        .target {
          mask: url("mask.svg#c1");
          -webkit-mask: url("mask.svg");
        }
      </style>
      <video class="target" height="270px" width="480px">
          <source src="HelloWorld.mp4"  type="video/mp4">
          <source src="HelloWorld.webm" type="video/webm">
          <source src="HelloWorld.ogv"  type="video/ogg">
      </video>
      // In mask.svg
      <mask id="c1" maskUnits="userSpaceOnUse" maskContentUnits="userSpaceOnUse">
        <circle id="circle" cx="240" cy="135" r="135" fill="white"/>
      </mask>
            
10
<video> and Canvas - Image regions
        sctxt.drawImage(video, 0, 0, 320, 160);
        
        for (x = 0; x < tiles; x ++) {
          for (y = 0; y < tiles; y ++) {
            context.drawImage(scratch, x*tw, y*th, tw, th,
                                       x*(tw+gap), y*(th+gap), tw, th);
          }
        }
            
11
<video> and Canvas - average color
        for (var i = 0; i < frame.data.length; i += 4) {
          r += frame.data[i];
          g += frame.data[i + 1];
          b += frame.data[i + 2];
        }
        ambience.style.backgroundColor = 'rgb('+r+','+g+','+b+')';
            
12
<video> and Canvas - reflection
              rctxt.translate(0,160);
              rctxt.scale(1,-1);
              gradient = rctxt.createLinearGradient(0, 105, 0, 160);
              gradient.addColorStop(1, "rgba(255, 255, 255, 0.3)");
              gradient.addColorStop(0, "rgba(255, 255, 255, 1.0)");
              rctxt.fillStyle = gradient;
              rctxt.drawImage(video, 0, 0, 320, 160);
            
13
<video> and Web Workers - face detection
    function isSkin(rn, gn, bn, base) {
      if (rn > 0.35 && rn < 0.5 && gn > 0.2 && gn < 0.5 && bn > 0.2 && bn < 0.35 
                    && base > 250) {
        return true;
      } else {
        return false;
      }
    }
            
14
<video> and accessibility - <track> (draft)
Source: Monty Montgomery
  <video src="video6.ogv" controls>
     <track label="English" kind="subtitles" src="video6_en.wsrt" srclang="en">
     <track label="French" kind="subtitles" src="video6_fr.wsrt" srclang="fr">
     <track label="Russian" kind="subtitles" src="video6_ru.wsrt" srclang="ru">
     <track label="English" kind="chapters" src="video6_chapters.wsrt" srclang="en">
  </video>
            
15
audio descriptions

Demo: http://www.annodex.net/~silvia/itext/elephant_no_skin.html

sign warning drivers of blind people
Image source: lupinehorror

<video src="video6.ogv" controls>
   <track label="English" kind="descriptions" src="video6_audesc.wsrt" srclang="en">
</video>
            
16
the WebSRT file format
  1-Intro
  00:00:00,000 --> 00:00:05,000
  The orange open movie project presents

  2-Intro
  00:00:05,010 --> 00:00:12,000
  Introductory titles are showing on the background of a water pool with fishes
  swimming and mechanical objects lying on a stone floor.

  3-Title
  00:00:12,010 --> 00:00:14,800
  elephants dream

  4
  00:00:26,100 --> 00:00:28,206
  Two people stand on a small bridge.
            
17
Bonus: audio API plans
Get a Firefox 4 beta
and enjoy
http://videos.mozilla.org/serv/blizzard/audio-slideshow/
18
Thank you
Questions?

              Silvia Pfeiffer
              blog.gingertech.net
              silviapfeiffer1@gmail.com
              @silviapfeiffer
              book: The Definitive Guide to HTML5 Video
            
Thanks go to the Blender organisation and to Creative Commons for their videos.
19