Animated HTML that syncs with video

A while back many of us saw this supercrazy Japanese music video, which shows how awesome it can be for HTML animations to play back in sync with an embedded video. But hand-coding this kind of content wouldn’t be much fun, so I thought I’d share how to visually author similar effects with Edge Animate. For the code and source see below, but first here’s the video tutorial:

Note! I made a few code updates since finishing the video - details at the end of the post.

If you want to try it out, below are some resources to get you started:

And finally, the code that makes it all work. This code should be placed into a creationComplete event handler, and it assumes that there is a symbol called video_holder somewhere in your Edge Animate project.

/*
* SETTINGS
*/

// video code of video to embed
var video_code = 'k6wf9QjWo8w';

// width and height of video to embed
var vid_w = '560';
var vid_h = '315';

// choose between embedded flash or iFrame youtube API
var useIFrame = true;

// site URL - required when embedding an iFrame youtube video
var site_url = 'https://fenomas.com';

// when testing iframe versions directly from Edge Animate use this:
var site_url = '127.0.0.1:54321';

/*
* REST OF SCRIPT BELOW
*/

// style the base page
$('body').css('backgroundColor', '#000')

// youtube video reference to be used throughout
var player = undefined;

// event handler for when the youtube video plays or pauses
window.onVideoState = function(status) {
// status will be 1 when playing, 2 when paused
var time = player.getCurrentTime(); // in seconds
if (status==1) {
// video is playing
sym.play(time*1000);
} else if (status==2) {
// video paused
sym.stop(time*1000);
}
}

// the youtube API sends no events at all on seek,
// so unfortunately we have to poll the video if
// we want to react to when the user seeks manually. :(
function updatePlayback() {
if (player && player.getCurrentTime) {
var t = player.getCurrentTime() * 1000;
var state = player.getPlayerState();
var dt = sym.getPosition() - t;
// match up times if discrepancy exceeds some tolerance
var tolerance = 500; //ms
if (Math.abs(dt) > tolerance) {
if (state==1) { sym.play(t); }
else { sym.stop(t); }
}
}
requestAnimationFrame(updatePlayback);
}
requestAnimationFrame(updatePlayback);

/*
EMBED VERSION
This chunk of code creates an embedded (Flash) youtube player
and sets up the events for the handler above.
It's simpler and easier than using an iframe but won't work on mobile.
*/

if ( ! useIFrame) {
// catch youtube API ready event - this gets fired by the embed
window.onYouTubePlayerReady = function() {
// keep a reference to the video player object
player = document.getElementById('player');
// direct state change events to the handler above
player.addEventListener("onStateChange", "onVideoState");
}

// create a youtube embed tag with the proper params
var params = '?version=3&enablejsapi=1';
var embedScript = '<object width="'+ vid_w +'" height="'+ vid_h +'"><param name="movie" value="https://www.youtube.com/v/'+video_code+params+'"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed id="player" src="https://www.youtube.com/v/'+video_code+params+'" type="application/x-shockwave-flash" width="'+ vid_w +'" height="'+ vid_h +'" allowscriptaccess="always" allowfullscreen="true"></embed></object>';

// get the HTML element for the 'video_holder' symbol,
// and replace its content with the <embed> tag
sym.$('video_holder').html( embedScript );

} // end embed insertion

/*
IFRAME VERSION
This chunk of code creates a youtube player in an iFrame
(so whether it plays in flash or html5 is up to youtube).
Catching the right events is more complicated due to
cross-site restrictions but the youtube API does most of the work.
*/

if (useIFrame) {
// catch the ready event, which will be fired by the youtube API
window.onYouTubeIframeAPIReady = function() {
// create the reference to the player
player = new YT.Player('player', {
events: {
// on state change, catch the event and send its
// 'data' property to the event handler defined above
'onStateChange': function(e) {
window.onVideoState(e.data);
}
}
});
}

// insert youtube iframe script into page. This fires the events
// and handles messaging back and forth with the iframe content
var tag = document.createElement('script');
tag.src = "https://www.youtube.com/iframe_api";
var firstScriptTag = document.getElementsByTagName('script')[0];
firstScriptTag.parentNode.insertBefore(tag, firstScriptTag);

// define params. Note that you must declare your
// site's URL! This is so youtube can allow the cross-site scripting.
var params = '?version=3&enablejsapi=1&origin=' + site_url;

// create the iframe tag with the right params
var embedScript = '<iframe id="player" width="'+ vid_w +'" height="'+ vid_h +'" src="https://www.youtube.com/embed/' + video_code + params + '" frameborder="0"></iframe>';

// get the HTML element for the 'video_holder' symbol,
// and replace its content with the <embed> tag
sym.$('video_holder').html( embedScript );

} // end iFrame

That’s it! If you make something cool be sure and let me know.

Updates!
(2014/7/9) - I made two changes to the JS code. I didn’t update the video, but if you grab the updated source code things should just work. Details:

  1. Local preview changed from “localhost:54321” to “127.0.0.1:54321”, to match Edge Animate’s behavior as of the CC 2014 release.
  2. Added a requestAnimationFrame loop that polls the youtube player to keep the page animation and video in sync. This is necessary because the Youtube HTML player doesn’t emit any events at all when the user seeks the video - apparently this is by design, though it makes no sense. (The youtube Flash player sends status events on seek, so the polling loop should rarely/never fire when not using the iframe embed.)