Instead of using text (optionally using HTML for styling, multi lines, etc.), it is also possible to use JSON objects as cue values that can be manipulated from JavaScript. JSON means “JavaScript Object Notation”. It’s an open standard for describing JavaScript objects as plain text.

Here is an example cue from a WebVTT file encoded as JSON instead of plain text. JSON is useful for describing “structured data”‘, and processing such data from JavaScript is easier than parsing plain text.

1. WEBVTT
3. Wikipedia
4. 00:01:15.200 –> 00:02:18.800
5. {
6.    “title”: “State of Wikipedia”,
7.    “description”: “Jimmy Wales talking …”,
8.    “src”: “http://upload.wikimedia.org/…../120px-Wikipedia-logo-v2.svg.png”,
9.    “href”: “http://en.wikipedia.org/wiki/Wikipedia”
10. }

This JSON object (in bold green) is a JavaScript object encoded as a text string. If we listen for cue events or if we read a WebVTT file as done in previous examples, we can extract this text content using the  cue.text property. For example:

1. var videoElement = document.querySelector(“#myvideo”);
2. var textTracks = videoElement.textTracks; // one for each track element
3. var textTrack = textTracks[0]; // corresponds to the first track element
5. var cues = textTrack.cues;
6. var cue = cues[0]; // first cue
8. // cue.text is in JSON format, with JSON.parse we turn it back
9. // to a real JavaScript object
10. var obj = JSON.parse(cue.text); 
12. var title = obj.title; // “State of Wikipedia”
13. var description = obj.description; // Jimmy Wales talking…
14. etc…

This is a powerful way of embedding metadata, especially when used in conjunction with listening for cue and track events.

improved approach: make a nicer chapter menu by embedding a richer description of chapter markers

Earlier we saw an example that could display chapter markers as clickable text on the right of a video.

This example used only standard plain text content for the cues:

1. WEBVTT
2.  
3. chapter-1
4. 00:00:00.000 –> 00:00:26.000
5. Introduction
6.  
7. chapter-2
8. 00:00:28.206 –> 00:01:02.000
9. Watch out!
10.  …

We used this example to manually capture the images from the video that correspond to each of the seven chapters:

• We clicked on each chapter link on the right, then paused the video,
• then we used a screen capture tool to grab each image that corresponds to the beginning of chapter,
• Finally, we resized the images with Photoshop to approximately 200×400 pixels.

(For advanced users: it’s possible to semi-automatize this process using the ffmepg command line tool, see for example this and that).

Here are the images which correspond to the seven chapters of the video from the previous example:

To associate these images with its chapter description, we will use JSON objects as cue contents:

elephants-dream-chapters-en-JSON.vtt:

1. WEBVTT
2.  
3. chapter-1
4. 00:00:00.000 –> 00:00:26.000
5. {
6. “description”: “Introduction”,
7. “image”: “introduction.jpg”
8. }
9.  
10.  
11. chapter-2
12. 00:00:28.206 –> 00:01:02.000
13. {
14. “description”: “Watch out!”,
15. “image”: “watchOut.jpg”
16. }
17. …

Before explaining the code, we propose that you try this example at JSBin that uses this new .vtt file:

HTML code:

1. …
2. <video id=”myVideo” preload=”metadata” controls crossOrigin=”anonymous”>
3. <source src=”https://&#8230;../elephants-dream-medium.mp4″
4.          type=”video/mp4″>
5. <source src=”https://&#8230;../elephants-dream-medium.webm”
6.          type=”video/webm”>
7. <track label=”English subtitles”
8.         kind=”subtitles”
9.         srclang=”en” src=”https://&#8230;../elephants-dream-subtitles-en.vtt” >
10. <track label=”Deutsch subtitles”
11.         kind=”subtitles”
12.         srclang=”de” src=”https://&#8230;../elephants-dream-subtitles-de.vtt” default>
13. <track label=”English chapters”
14.          kind=”chapters”
15.         srclang=”en” src=”https://&#8230;../elephants-dream-chapters-en-JSON.vtt”>
16. </video>
17. <h2>Chapter menu</h2>
19.  …

It’s the same code we had in the first example, except that this time we use a new WebVTT file that uses JSON cues to describe each chapter. For the sake of simplicity, we also removed the buttons and all the code for displaying a clickable transcript of the subtitles/captions on the right of the video.

JavaScript code:

1. var video, chapterMenuDiv;
2. var tracks, trackElems, tracksURLs = [];
3.  
4. window.onload = function() {
5.    console.log(“init”);
6.    // When the page is loaded
7.    video = document.querySelector(“#myVideo”);
8.    chapterMenuDiv = document.querySelector(“#chapterMenu”);
10.    // Get the tracks as HTML elements
11.    trackElems = document.querySelectorAll(“track”);
12.    for(var i = 0; i < trackElems.length; i++) {
13.       var currentTrackElem = trackElems[i];
14.       tracksURLs[i] = currentTrackElem.src;
15.    }
17.    // Get the tracks as JS TextTrack objects
18.    tracks = video.textTracks;
20.    // Build the chapter navigation menu for the given lang and kind
21.    buildChapterMenu(‘en’, ‘chapters’);
22. };
23.  
24. function buildChapterMenu(lang, kind) {
25.   // Locate the track with language = lang and kind=”chapters”
26.   for(var i = 0; i < tracks.length; i++) {
27.      // current track
28.      var track = tracks[i];
29.      var trackAsHtmlElem = trackElems[i];
31.      if((track.language === lang) && (track.kind === kind)) {
32.         // the track must be active, otherwise it will not load
33.         track.mode=”showing”; // “hidden” would work too
34.  
35.         if(trackAsHtmlElem.readyState === 2) {
36.            // the track has already been loaded
37.            displayChapterMarkers(track);
38.         } else {
39.            displayChapterMarkersAfterTrackLoaded(trackAsHtmlElem, track);
40.         }
41.      }
42.    }
43. }
44.  
45. function displayChapterMarkers(track) {
46.    var cues = track.cues;
48.    // We must not see the cues on the video
49.    track.mode = “hidden”;
51.    // Iterate on cues
52.    for(var i=0, len = cues.length; i < len; i++) {
53.       var cue = cues[i];
54.      var cueObject = JSON.parse(cue.text);
55.       var description = cueObject.description;
56.       var imageFileName = cueObject.image;
57.       var imageURL = “https://mainline.i3s.unice.fr/mooc/” + imageFileName;
59.       // Build the marker. It’s a figure with an img and a figcaption inside.
60.       // The img has an onclick listener that will make the video jump
61.       // to the start time of the current cue/chapter
62.       var figure = document.createElement(‘figure’);
63.       figure.classList.add(“img”);
65.       figure.innerHTML = “<img onclick=’jumpTo(“
66.                          + cue.startTime + “);’ class=’thumb’ src='”
67.                          + imageURL + “‘><figcaption class=’desc’>”
68.                          + description + “</figcaption></figure>”;
69.       // Add the figure to the chapterMenuDiv
70.       chapterMenuDiv.insertBefore(figure, null);
71.     }
72. }
73.  
74. function displayChapterMarkersAfterTrackLoaded(trackElem, track) {
75.   // Create a listener that will only be called when the track has
76.   // been loaded
77.   trackElem.addEventListener(‘load’, function(e) {
78.      console.log(“chapter track loaded”);
79.      displayChapterMarkers(track);
80.   });
81. }
82.  
83. function jumpTo(time) {
84.   video.currentTime = time;
85.   video.play();
86. }
87.  

Explanations:

• Lines  4-18: when the page is loaded, we assemble all of the track HTML elements and their corresponding TextTrack objects.
• Line 19: using that we can build the chapter navigation menu. All is done in the window.onload callback, so nothing happens until the DOM is ready.
• Lines 24-43: the buildChapterMenu function first locates the chapter track for the given language, then checks if this track has been loaded by the browser. Once it has been confirmed that the track is loaded, the function displayChapters is called. 
• Lines 45-65: the displayChapters(track) function will iterate over all of the cues within the chapter track passed as its parameter. For each cue, the JSON content is re-formatted back into a JavaScript object (line 55) and the image filename and description of the chapter/cue are extracted (lines 56-57). Then an HTML description for the chapter is built and added to the div element with id=chapterMenu. Here is the HTML code for one menu marker:
  
  
  1. <figure class=”img”>
  2.    <img onclick=”jumpTo(0);” class=”thumb” src=”https://&#8230;../introduction.jpg”>
  3.    <figcaption class=”desc”>
  4.       Introduction
  5.    </figcaption>
  6. </figure>

Notice that we add a click listener to each thumbnail image. Clicking a chapter thumbnail will cause the video to jump to the chapter time location (the example above is for the first chapter with start time = 0).

We also added CSS classes “img”, “thumb” and “desc”, which make it easy to style and position the thumbnails using CSS.

CSS source code extract:

1. #chapterMenuSection {
2.    background-color: lightgrey;
3.    border-radius:10px;
4.    padding: 20px;
5.    border:1px solid;
6.    display:inline-block;
7.    margin:0px 30px 30px 30px;
8.    width:90%;
9. }
10.  
11. figure.img {
12.   margin: 2px;
13.   float: left;
14. }
15.  
16. figcaption.desc {
17.   text-align: center;
18.   font-weight: normal;
19.   margin: 2px;
20. }
21.  
22. .thumb {
23.   height: 75px;
24.   border: 1px solid #000;
25.   margin: 10px 5px 0 0;
26.   box-shadow: 5px 5px 5px grey;
27.   transition: all 0.5s;
28. }
29.  
30. .thumb:hover {
31.   box-shadow: 5px 5px 5px black;
32. }

A sample menu marker is shown below (it’s also animated – hover your mouse over the thumbnail to see its animated shadow):

combining techniques: A clickable transcript and A chapter menu

This example is the same as the previous one except that we have kept the features that we saw previously: the buttons for displaying a clickable transcript. The code is longer, but it’s just a combination of the “clickable transcript” example from the previous lesson, and the code from earlier in this lesson.

Try it at JSBin