Consider a package located at URL P that contains 2 resources A and B identified by fragment identifiers #A and #B. If a web page URL uses P#A or P#B, it might be interesting to avoid downloading the entire package when only a small part is needed, or at least to prioritize the download for the parts of the resource being used in the fragment. I’m thinking that such processing could be done in a Service Worker that understands the package format, but unfortunately it cannot at the moment.
Service Workers are meant to behave as a client-side, browser-integrated proxy. As such, they do not receive fragment identifiers in the requested URL of fetch events. However, since a Service Worker is a specific kind of proxy (hey it runs in a browser), I’m wondering if it wouldn’t be interesting to let fragments reach Service Workers. Any thought here ?
If you are interested in my research activities and you want to get an understanding of what I’ve been doing for the past year, I’ve made a diagram illustrating how my different academic publications relate and what topics they cover.
Continue reading Cartography of my publications
As of version 5017 of GPAC, I’ve added a first preliminary support of storage of TTML in MP4. The support is according to the ISOBMFF spec (MPEG-4 Part 30, to appear) and to the DECE CFF format. This post describes why and how to use this implementation. Continue reading First attempt at storing TTML in MP4
Continue reading Using WebVTT to carry media streams
This week I gave a talk during the HTML5 Developer conference in San Francisco. The conference was very interesting and I met many people doing cool things with SVG. You can view the slides of my presentation here. The gist of this presentation is: “If you create timeline-based animations and you structure them properly, using some of my tools (packager and JS player) you can stream SVG content on the Web just like a video, including live or adaptive streaming”. I’m still working on those tools but will be releasing them soon.
Voici les slides de la présentation que j’ai donnée aujourd’hui lors de la réunion du GDR ISIS. Ces slides présentent les concepts du streaming vidéo sur HTTP et en particulier la norme MPEG-DASH.
Responsive image is a term in the Web world corresponding to the techniques involved when an author wants its website to be rendered with the right image for the right client given its viewing condition (screen size, pixel density, network, …). This technique falls, from a research point of view, in the broad category of media adaptation techniques to the user’s context. To me, at first sight, the problem seemed a no-brainer as it has been solved several times, including in the web world for video streaming with the recent approaches of adaptive streaming such as DASH. Naively I thought the same techniques could be used. However, after attending some meetings, including this week’s meetup, it appears that the environment constraints are such that the problem is not so simple to solve. In this post, I want to highlight the differences and give an example of how DASH manifest could be used for responsive images (I’m not really proposing it though). Continue reading Adaptive streaming techniques for responsive images
In a previous post, I described a bit the support in GPAC for the HTML 5 video and audio elements in SVG documents. My initial idea was to be able to play, with the Media Source Extensions API, adaptive streaming content (such as DASH) in SVG. So I started implementing MSE and it is now possible to play DASH content within an SVG presentation using GPAC. This post details how it was implemented. Continue reading HTML 5 Media Source Extensions in SVG
In a previous post, I described how to package and manipulate WebVTT content in MP4 files according to the latest ISO standard using MP4Box. Basic import of WebVTT or SRT file is as follows:
MP4Box -add file1.vtt:lang=en subtitle1.mp4
MP4Box -add file2.srt:FMT=VTT:lang=en subtitle2.mp4
and then, the basic usage to create DASH subtitle segments of 10 seconds:
MP4Box -dash 10000 subtitle.mp4:role=subtitle video.mp4
It is now possible to play these MP4 files with the GPAC players (on all supported platforms: Win, Mac, Linux, Android, iOS). So, try it out and let me know if it has bugs. You can for instance test this file or its DASH version.
This post describes some details on how the rendering was achieved.
Continue reading WebVTT, MP4 files, DASH and GPAC
In my previous post, I described how to use the HTML 5 <video> element to play and control SVG animations, just like a video. HTML 5 and the <video> element also enable displaying text tracks on top of a video, and making sure it stays synchronized with the video thanks to the <track> element. However, this is currently implemented only in some browsers and mostly (only?) for the WebVTT format. Many JS players also provide WebVTT support. In this post, I describe an extension to Video.js to play and control SVG content synchronously on top of a video with the <track> element.
The syntax is pretty simple:
<video controls width='303' height='250'>
<source src='video1.mp4' type='video/mp4' />
<source src='video1.ogv' type='video/ogg' />
<track kind='graphics' src='annotations.svg' srclang='en' label='Squares'/>
<track kind='graphics' src='annotations2.svg' srclang='en' label='Circles'/>
And the result is shown below (example code is here) :
You can play the video, select a graphics track (displaying rectangles or circles whose positions and colors change every 5 sec in a 3-steps cycle) using the “Graphics” menu while the video is playing. You can then seek with the timeline, pause the video, play again and the graphics should stay synchronized. You can also change track while the video is playing.