Timeline hover previews: use with players and Roku devices

What are timeline hover previews?

Timeline hover previews, or trick play, is an option that makes it easier to rewind and find a specific moment in a video. It shows users previews when they point their cursor to any specific point on the video’s timeline. For example:

What do the previews consist of?

In the Gcore Streaming Platform, we use time-based preview. During the transcoding process, the video is divided into equal segments, and screenshots are taken. These screenshots are called tiles.

All tiles are collected in a single array—storyboard. A storyboard is a large JPEG image of tiles chronologically arranged one after another. Here is an example of the storyboard for Blender Studio’s cartoon “Coffee Run”:

Note: We create storyboards for all videos automatically.

Adding timeline hover previews to players

Gcore Player

Timeline hover previews are embedded into Gcore’s Player by default, so previews are displayed automatically. For example:

For external players, previews are available in two ways:

• WebVTT format
• Roku trick play

WebVTT format in API

The Get video API request returns information about the video, including two additional fields:

• sprite: link to the storyboard, which contains the tiles
• sprite_vtt: description of tiles in VTT format

Example:

GET https://api.gcore.com/streaming/videos/{video_id}/

      Copy
      
    

"sprite": "https://demo-public.gvideo.io/static/videoplatform/sprites/2675/2474723_FnlHXwA16ZMxmUr.mp4_sprite.jpg", 

"sprite_vtt": "1 
00:00:00,000 --> 00:00:05,000 
FnlHXwA16ZMxmUr_sprite.jpg#xywh=0,0,100,42

...

16
00:01:15,000 --> 00:01:20,000
FnlHXwA16ZMxmUr_sprite.jpg#xywh=100,84,100,42
17
00:01:20,000 --> 00:01:25,000
FnlHXwA16ZMxmUr_sprite.jpg#xywh=200,84,100,42
18
00:01:25,000 --> 00:01:30,000
FnlHXwA16ZMxmUr_sprite.jpg#xywh=300,84,100,42
...

The VTT field structure is:

• Tile number, e.g., 17
• Timing when a tile is displayed, e.g., 00:01:20,000 --> 00:01:25,000
• The coordinates of the upper left corner and the size of the tile from a storyboard (format is “xywh”): xywh=200,84,100,42

Note: Tiles are read from left to right, top to bottom.

In the picture below, the 17th tile (third one from left, third from top) is highlighted in VTT format:

Roku trick play

According to the Roku specification, we can embed tiles directly in .m3u8 files. Here’s the format for requesting a video master manifest with embedded tiles:

Where:

• {client_id}: your account ID
• {video_id}: identifier of the video or live stream
• [-img]: variable suffix, when specified, information about tiles is added to the manifest

You can use the result by adding the [-img] suffix. It will add a special preview tag to the master manifest #EXT-X-IMAGE-STREAM-INF, which points to an additional tiles manifest containing a link to a storyboard file, tile size, how often tiles should replace each other, and other information.

Example of a master manifest with the suffix for tiles:

      Copy
      
    curl https://demo-public.gvideo.io/videos/2675_FnlHXwA16ZMxmUr/master-img.m3u8

Example of a tiles manifest:

      Copy
      
    curl https://demo-public.gvideo.io/videos/2675_FnlHXwA16ZMxmUr/tiles.m3u8