Reduced Response Time in Vietnam

How we engineered a single pipeline for LL-HLS and LL-DASH

Viewers in sports, gaming, and interactive events expect real-time, low-latency streaming experiences. To deliver this, the industry has rallied around two powerful protocols: Low-Latency HLS (LL-HLS) and Low-Latency DASH (LL-DASH).While they share a goal, their methods are fundamentally different. LL-HLS delivers video in a sequence of tiny, discrete files. LL-DASH delivers it as a continuous, chunked download of a larger file. This isn't just a minor difference in syntax; it implies completely different behaviors for the packager, the CDN, and the player.This duality presents a major architectural challenge: How do you build a single, efficient, and cost-effective pipeline that can serve both protocols simultaneously from one source?At Gcore, we took on this unification problem. The result is a robust, single-source pipeline that delivers streams with a glass-to-glass latency of approximately 2.0 seconds for LL-DASH and 3.0 seconds for LL-HLS. This is the story of how we designed it.Understanding the dualityTo build a unified system, we first had to deeply understand the differences in how each protocol operates at the delivery level.LL-DASH: the continuous feedMPEG-DASH has always been flexible, using a single manifest file to define media segments by their timing. Low-Latency DASH builds on this by using Chunked CMAF segments.Imagine a file that is still being written to on the server. Instead of waiting for the whole file to be finished, the player can request it, and the server can send it piece by piece using Chunked Transfer Encoding. The player receives a continuous stream of bytes and can start playback as soon as it has enough data.Single, long-lived files: A segment might be 2–6 seconds long, but it’s delivered as it’s being generated.Timing-based requests: The player knows when a segment should be available and requests it. The server uses chunked transfer to send what it has so far.Player-driven latency: The manifest contains a targetLatency attribute, giving the player a strong hint about how close to the live edge it should play.LL-HLS: the rapid-fire deliveryLL-HLS takes a different approach. It extends the traditional playlist-based HLS by breaking segments into even smaller chunks called Parts.Think of it like getting breaking news updates. The server pre-announces upcoming Parts in the manifest before they are fully available. The player then requests a Part, but the server holds that request open until the Part is ready to be delivered at full speed. This is called a Blocking Playlist Reload.Many tiny files (Parts): A 2-second segment might be broken into four 0.5-second Parts, each requested individually.Manifest-driven updates: The server constantly updates the manifest with new Parts, and uses special tags like #EXT-X-PART-INF and #EXT-X-SERVER-CONTROL to manage delivery.Server-enforced timing: The server controls when the player receives data by holding onto requests, which helps synchronize all viewers.A simplified diagram visually comparing the LL-HLS delivery of many small parts versus the LL-DASH chunked transfer of a single, larger segment over the same time period.These two philosophies demand different things from a CDN. LL-DASH requires the CDN to intelligently cache and serve partially complete files. LL-HLS requires the CDN to handle a massive volume of short, bursty requests and hold connections open for manifest updates. A traditional CDN is optimized for neither.Forging a unified strategyWith two different delivery models, where do you start? You find the one thing they both depend on: the keyframe.Playback can only start from a keyframe (or I-frame). Therefore, the placement of keyframes, which defines the Group of Pictures (GOP), is the foundational layer that both protocols must respect. By enforcing a consistent keyframe interval on the source stream, we could create a predictable media timeline. This timeline can then be described in two different “languages” in the manifests for LL-HLS and LL-DASH.A single timeline with consistent GOPs being packaged for both protocols.This realization led us to a baseline configuration, but each parameter involved a critical engineering trade-off:GOP: 1 second. We chose a frequent, 1-second GOP. The primary benefit is extremely fast stream acquisition; a player never has to wait more than a second for a keyframe to begin playback. The trade-off is a higher bitrate. A 1-second GOP can increase bitrate by 10–15% compared to a more standard 2-second GOP because you're storing more full-frame data. For real-time, interactive use cases, we prioritized startup speed over bitrate savings.Segment Size: 2 seconds. A 2-second segment duration provides a sweet spot. For LL-DASH and modern HLS players, it's short enough to keep manifest sizes manageable. For older, standard HLS clients, it prevents them from falling too far behind the live edge, keeping latency reduced even on legacy players.Part Size: 0.5 seconds. For LL-HLS, this means we deliver four updates per segment. This frequency is aggressive enough to achieve sub-3-second latency while being coarse enough to avoid overwhelming networks with excessive request overhead, which can happen with part durations in the 100–200ms range.Cascading challenges through the pipeline1. Ingest: predictability is paramountTo produce a clean, synchronized output, you need a clean, predictable input. We found that the encoder settings of the source stream are critical. An unstable source with a variable bitrate or erratic keyframe placement will wreck any attempt at low-latency delivery.For our users, we recommend settings that prioritize speed and predictability over compression efficiency:Rate control: Constant Bitrate (CBR)Keyframe interval: A fixed interval (e.g., every 30 frames for 30 FPS, to match our 1s GOP).Encoder tune: zerolatencyAdvanced options: Disable B-frames (bframes=0) and scene-cut detection (scenecut=0) to ensure keyframes are placed exactly where you command them to be.Here is an example ffmpeg command in Bash that encapsulates these principles:ffmpeg -re -i "source.mp4" -c:a aac -c:v libx264 \ -profile:v baseline -tune zerolatency -preset veryfast \ -x264opts "bframes=0:scenecut=0:keyint=30" \ -f flv "rtmp://your-ingest-url"2. Transcoding and packagingOur transcoding and Just-In-Time Packaging (JITP) layer is where the unification truly happens. This component does more than just convert codecs; it has to operate on a stream that is fundamentally incomplete.The primary challenge is that the packager must generate manifests and parts from media files that are still being written by the transcoder. This requires a tightly-coupled architecture where the packager can safely read from the transcoder's buffer.To handle the unpredictable nature of live sources, especially user-generated content via WebRTC, we use a hybrid workflow:GPU Workers (Nvidia/Intel): These handle the heavy lifting of decoding and encoding. Offloading to GPU hardware is crucial for minimizing processing latency and preserving advanced color formats like HDR+.Software Workers and Filters: These provide flexibility. When a live stream from a mobile device suddenly changes resolution or its framerate drops due to a poor connection, a rigid hardware pipeline would crash. Our software layer can handle these context changes gracefully, for instance, by scaling the erratic source and overlaying it on a stable, black-background canvas, meaning the output stream never stops.This makes our JITP a universal packager, creating three synchronized content types from a single, resilient source:LL-DASH (CMAF)LL-HLS (CMAF)Standard HLS (MPEG-TS) for backward compatibility3. CDN delivery: solving two problems at onceThis was the most intensive part of the engineering effort. Our CDN had to be taught how to excel at two completely different, high-performance tasks simultaneously.For LL-DASH, we developed a custom caching module we call chunked-proxy. When the first request for a new .m4s segment arrives, our edge server requests it from the origin. As bytes flow in from the origin, the chunked-proxy immediately forwards them to the client. When a second client requests the same file, our edge server serves all the bytes it has already cached and then appends the new bytes to both clients' streams simultaneously. It’s a multi-client cache for in-flight data.For LL-HLS, the challenges were different:Handling Blocked Requests: Our edge servers needed to be optimized to hold thousands of manifest requests open for hundreds of milliseconds without consuming excessive resources.Intelligent Caching: We needed to correctly handle cache statuses (MISS, EXPIRED) for manifests to ensure only one request goes to the origin per update, preventing a "thundering herd" problem.High Request Volume: LL-HLS generates a storm of requests for tiny part-files. Our infrastructure was scaled and optimized to serve these small files with minimal overhead.The payoff: ultimate flexibility for developersThis engineering effort wasn't just an academic exercise. It provides tangible benefits to developers building with our platform. The primary benefit is simplicity through unification, but the most powerful benefit is the ability to optimize for every platform.Consider the complex landscape of Apple devices. With our unified pipeline, you can create a player logic that does this:On iOS 17.1+: Use LL-DASH with the new Managed Media Source (MMS) API for ~2.0 second latency.On iOS 14.0 - 17.0: Use native LL-HLS for ~3.0 second latency.On older iOS versions: Automatically fall back to standard HLS with a reduced latency of ~9 seconds.This lets you provide the best possible experience on every device, all from a single backend and a single live source, without any extra configuration.Don't fly blind: observability in a low-latency worldA complex system is useless without visibility, and traditional metrics can be misleading for low-latency streaming. Simply looking at response_time from a CDN log is not enough.We had to rethink what to measure. For example:For an LL-HLS manifest, a high response_time (e.g., 500ms) is expected behavior, as it reflects the server correctly holding the request while waiting for the next part. A low response_time could actually indicate a problem. We monitor “Manifest Hold Time” to ensure this blocking mechanism is working as intended.For LL-DASH, a player requesting a chunk that isn't ready yet might receive a 404 Not Found error. While occasional 404s are normal, a spike can indicate origin-to-edge latency issues. This metric, combined with monitoring player liveCatchup behavior, gives a true picture of stream health.Gcore: one pipeline to serve them allThe paths of LL-HLS and LL-DASH may be different, but their destination is the same: real-time interaction with a global audience. By starting with a common foundation—the keyframe—and custom-engineering every component of our pipeline to handle this duality, we successfully solved the unification problem.The result is a single, robust system that gives developers the power of both protocols without the complexity of running two separate infrastructures. It’s how we deliver ±2.0s latency with LL-DASH and ±3.0s with LL-HLS, and it’s the foundation upon which we’ll build to push the boundaries of real-time streaming even further.

Smart caching and predictive streaming: the next generation of content delivery

As streaming demand surges worldwide, providers face mounting pressure to deliver high-quality video without buffering, lag, or quality dips, no matter where the viewer is or what device they're using. That pressure is only growing as audiences consume content across mobile, desktop, smart TVs, and edge-connected devices.Traditional content delivery networks (CDNs) were built to handle scale, but not prediction. They reacted to demand, but couldn’t anticipate it. That’s changing.Today, predictive streaming and AI-powered smart caching are enabling a proactive, intelligent approach to content delivery. These technologies go beyond delivering content by forecasting what users will need and making sure it's there before it's even requested. For network engineers, platform teams, and content providers, this marks a major evolution in performance, reliability, and cost control.What are predictive streaming and smart caching?Predictive streaming is a technology that uses AI to anticipate what a viewer will watch next, so the content can be ready before it's requested. That might mean preloading the next episode in a series, caching popular highlights from a live event, or delivering region-specific content based on localized viewing trends.Smart caching supports this by storing that predicted content on servers closer to the viewer, reducing delays and buffering. Together, they make streaming faster and smoother by preparing content in advance based on user behavior.Unlike traditional caching, which relies on static popularity metrics or simple geolocation, predictive streaming is dynamic. It adapts in real time to what’s happening on the platform: user actions, traffic spikes, network conditions, and content trends. This results in:Faster playback with minimal bufferingReduced bandwidth and server loadHigher quality of experience (QoE) scores across user segmentsFor example, during the 2024 UEFA European Championship, several broadcasters used predictive caching to preload high-traffic game segments and highlight reels based on past viewer drop-off points. This allowed for instant replay delivery in multiple languages without overloading central servers.Why predictive streaming matters for viewersGlobally, viewers tend to binge-watch new streaming platform releases. For example, sci-fi-action drama Fallout got 25% of its annual US viewing minutes (2.9 billion minutes) in its first few days of release. The South Korean series Queen of Tears became Netflix's most-watched Korean drama of all time in 2024, amassing over 682.6 million hours viewed globally, with more than half of those watch hours occurring during its six-week broadcast run.A predictive caching system can take advantage of this launch-day momentum by pre-positioning likely-to-be-watched episodes, trailers, or bonus content at the edge, customized by region, device, or time of day.The result is a seamless, high-performance experience that anticipates user behavior and scales intelligently to meet it.Benefits for streaming providersTraditional CDNs often waste resources caching content that may never be viewed. Predictive caching focuses only on content that is likely to be accessed, leading to:Lower egress costsReduced server loadMore efficient cache hit ratiosOne of the core benefits of predictive streaming is latency reduction. By caching content at the edge before it’s requested, platforms avoid the delay caused by round-trips to origin servers. This is especially critical for:Live sports and eventsInteractive or real-time formats (e.g., polls, chats, synchronized streams)Edge environments with unreliable last-mile connectivityFor instance, during the 2024 Copa América, mobile viewers in remote areas of Argentina were able to stream matches without delay thanks to proactive edge caching based on geo-temporal viewing predictions.How it worksAt the core of predictive streaming is smart caching: the process of storing data closer to the end user before it’s explicitly requested. Here’s how it works:Data ingestion: The system gathers data on user behavior, device types, content popularity, and location-based trends.Behavior modeling: AI models identify patterns (e.g., binge-watching behaviors, peak-hour traffic, or regional content spikes).Pre-positioning: Based on predictions, the system caches video segments, trailers, or interactive assets to edge servers closest to where demand is expected.Real-time adaptation: As user behavior changes, the system continuously updates its caching strategy.Use cases across streaming ecosystemsSmart caching and predictive delivery benefit nearly every vertical of streaming.Esports and gaming platforms: Live tournaments generate unpredictable traffic surges, especially when underdog teams advance. Predictive caching helps preload high-interest match content, post-game analysis, and multilingual commentary before traffic spikes hit. This helps provide global availability with minimal delay.Corporate webcasts and investor events: Virtual AGMs or earnings calls need to stream seamlessly to thousands of stakeholders, often under compliance pressure. Predictive systems can cache frequently accessed segments, like executive speeches or financial summaries, at regional nodes.Education platforms: In EdTech environments, predictive delivery ensures that recorded lectures, supplemental materials, and quizzes are ready for users based on their course progression. This reduces lag for remote learners on mobile connections.VOD platforms with regional licensing: Content availability differs across geographies. Predictive caching allows platforms to cache licensed material efficiently and avoid serving geo-blocked content by mistake, while also meeting local performance expectations.Government or emergency broadcasts: During public health updates or crisis communications, predictive streaming can support multi-language delivery, instant replay, and mobile-first optimization without overloading networks during peak alerts.Looking forward: Personalization and platform governanceWe predict that the next wave of predictive streaming will likely include innovations that help platforms scale faster while protecting performance and compliance:Viewer-personalized caching, where individual user profiles guide what’s cached locally (e.g., continuing series, genre preferences)Programmatic cache governance, giving DevOps and marketing teams finer control over how and when content is distributedCross-platform intelligence, allowing syndicated content across services to benefit from shared predictions and joint caching strategiesGcore’s role in the predictive futureAt Gcore, we’re building AI-powered delivery infrastructure that makes the future of streaming a practical reality. Our smart caching, real-time analytics, and global edge network work together to help reduce latency and cost, optimize resource usage, and improve user retention and stream stability.If you’re ready to unlock the next level of content delivery, Gcore’s team is here to help you assess your current setup and plan your predictive evolution.Discover how Gcore streaming technologies helped fan.at boost subscription revenue by 133%

Reduced Response Time in Vietnam

Related articles

Gcore partners with AVEQ to elevate streaming performance monitoring

How we engineered a single pipeline for LL-HLS and LL-DASH

Gcore CDN updates: Dedicated IP and BYOIP now available

Smart caching and predictive streaming: the next generation of content delivery

Protecting networks at scale with AI security strategies

Introducing Gcore for Startups: created for builders, by builders

Subscribe to our newsletter