Most web browsers only support a few ancient image formats (mainly PNG, JPEG and GIF), but video formats have improved significantly since those formats were defined. Google is attempting to fix this with their WebP image format, based on VP8. Unfortunately, this only works with Google Chrome and Opera. Since what we want is to encode an image using the advancements in VP8 or h.264, I thought it would be interesting to try encoding single-frame videos and using them as images.

Update: Since writing this, several browsers including Safari and anything based on WebKitGTK now natively support using videos in the <img> tag, so on supported browsers, you can also use <img src="path/to/video.mp4">.

To make the fallback easy, I want to have a video available for any browser which supports the <video> tag, so I'll provide h.264, VP8 and JPEG. For additional bandwidth savings, I'm also providing VP9, which some browsers support. I would do h.265 also, but I can't find any browsers that support it at the moment.

To encode the videos, I decided to use FFMPEG. For some reason, a lot of FFMPEG packages don't support VP8, so I needed to build it myself:

cd workspace
git clone git://source.ffmpeg.org/ffmpeg.git ffmpeg
cd ffmpeg
./configure --enable-libvpx --enable-libx264 --enable-gpl

Then I needed some test images. I decided to use this picture of a bear by a Flickr user named Soldatnytt (CC BY 2.0).

cd ~/Downloads
wget https://farm5.staticflickr.com/4067/4665287289_cc672822df_o_d.jpg -O original.jpg

Since I don't actually want to deal with massive images, I started by resizing the JPG to 500x750. You can do this with ImageMagick, or just open it in your favorite file editor and resize it:

convert original.jpg -resize 500x750 -quality 70 bear.jpg

Now we want to get the same file as a single-frame h.264 MP4 video and VP8 WebM video:

# Note: -crf indicates the quality level. It won't work for me for WebM, so
# I used -b:v, which indicates the bitrate.
# Since quality settings have different meanings for each format, I tried to
# make each version as small as possible before I started noticing artifacts.
~/workspace/ffmpeg/ffmpeg -i original.jpg -c:v libvpx -b:v 200K -vf scale=500:750 bear.vp8.webm
~/workspace/ffmpeg/ffmpeg -i original.jpg -c:v libvpx-vp9 -b:v 1.5M -vf scale=500:750 -strict -2 bear.vp9.webm

# I had to use -pix_fmt yuv420p to make this work on Chrome and Safari. yuv444p would also
# work, but the higher color didn't seem to matter for this image.
~/workspace/ffmpeg/ffmpeg -i original.jpg -c:v libx264 -crf 20 -vf scale=500:750 -pix_fmt yuv420p bear.h264.mp4

Running du -h * shows that these videos are much smaller than the JPEG version:

36K bear.h264.mp4
76K bear.jpg
40K bear.vp8.webm
28K bear.vp9.webm

Now we just need to add them to an HTML file like this:

<script>
    document.addEventListener("load", function() {
        // Detect video support
        // http://diveinto.html5doctor.com/detect.html#video
        if (!!document.createElement('video').canPlayType) {
            // <img> fallback
            var videos = document.querySelectorAll("video.img-fallback");
            for (var i = 0; i < videos.length; ++i) {
                var video = videos[i];
                var img = document.createElement("img");
                img.src = video.getAttribute("data-img-src");
                video.parentNode.replaceChild(img, video);
            }
        }
    });
</script>
<video preload class="img-fallback" data-img-src="bear.jpg">
    <source src="bear.vp9.webm" type="video/webm" type='video/webm; codecs="vp9"'>
    <source src="bear.h264.mp4" type="video/mp4" type='video/mp4'>
    <source src="bear.vp8.webm" type="video/webm" type='video/webm; codecs="vp8"'>
</video>

Note that I arranged the <source> elements in reverse order of size, because that's the order browsers will try them in. The <img> fallback is more complicated than I'd like, since we only want to load the image if the browser doesn't have <video> support. I initially tried something like this:

<video preload>
    <source src="bear.vp9.webm" type="video/webm" type='video/webm; codecs="vp9"'>
    <source src="bear.h264.mp4" type="video/mp4" type='video/mp4'>
    <source src="bear.vp8.webm" type="video/webm" type='video/webm; codecs="vp8"'>
    <img src="bear.jpg">
</video>

But unfortunately, that causes both Chrome and Firefox (and possibly other browsers) to download the image, even though they won't display it.

Anyway, here's what that looks like for real:

Now, given that using the <video> tag is resource intensive, I'd like to immediately replace those videos with an image if I can. Here's some code that finds videos tagged with class "image" and replaces them with a canvas:

<script>
    function replaceVideoWithImage(video) {
        var canvas = document.createElement("canvas");
        canvas.width = video.videoWidth;
        canvas.height = video.videoHeight;
        var context = canvas.getContext("2d");
        context.drawImage(video, 0, 0, video.videoWidth, video.videoHeight);
        try {
            var image = document.createElement("img");
            /* toDataURL() will throw an exception if the video is from
                another domain (cross-origin violation). */
            image.src = canvas.toDataURL();
            video.parentNode.replaceChild(image, video);
        } catch (e) {
            video.parentNode.replaceChild(canvas, video);
        }
    }

    window.addEventListener("load", function() {
        var videos = document.querySelectorAll("video.image");
        for (var i = 0; i < videos.length; ++i) {
            var video = videos[i];
            if (video.readyState >= 2) {
                replaceVideoWithImage(video);
            } else {
                video.addEventListener("loadeddata", function() {
                    replaceVideoWithImage(video);
                });
            }
        }
    });
</script>
<video preload class="image img-fallback" data-img-src="bear.jpg">
    <source src="bear.vp9.webm" type='video/webm; codecs="vp9"'>
    <source src="bear.h264.mp4" type="video/mp4">
    <source src="bear.vp8.webm" type='video/webm; codecs="vp8"'>
</video>

And here it is on this page:

Note that we try to convert the <canvas> to an <img> if possible, but if the video is on another domain, the cross-origin policy won't let us generate a data URL. I'm guessing in most cases, the differences between an <img> and a <canvas> are pretty small anyway. If it does matter, you should be able to fix it by sending the cross-origin access control header with the videos:

Access-Control-Allow-Origin: *

For real use, you'd probably also want to copy at least the id and class attributes from the <video> to whatever you replace it with. I left it out because I didn't want to distract from the example. I also had some trouble getting the final conversion from <canvas> to <img> to work consistently in Google Chrome (sometimes it gives me an empty image), but it seems like you could just leave the image as <video> or <canvas> if that can't be solved.

Anyway, I don't know if this is practical, since it's a lot harder on clients, but it works with today's browsers (and can be created with stock FFMPEG), and for very large images, it could significantly reduce bandwidth.