On October 10, 2023, a major security incident on Google customers involving HTTP/2 protocol flaws occurred. This was not the first time this had occurred—the first incidence was on August 25—but the strength of this recent attack was unprecedented. HTTP/2 improves performance by letting multiple requests and responses share one TCP connection, but this feature introduces a vulnerability: it’s susceptible to a specialized distributed denial-of-service (DDoS) attack known as “Rapid Reset.”
How Is the Attack Possible?
Unlike HTTP/1.1, which necessitates a new TCP connection for each request and response, HTTP/2 is designed for efficiency. HTTP/2 achieves this by enabling multiple requests and responses to share a single TCP connection via a mechanism called “streams.” These streams are bidirectional data flows within a single TCP connection, permitting the concurrent exchange of multiple requests and answers. This enhances performance and also enables server push technology, which allows servers to proactively send resources to the client without waiting for a specific request, thereby accelerating page load times.
However, these HTTP/2 advanced features and benefits can be exploited to become a serious security weakness. Bidirectional communication makes it susceptible to a specialized type of denial-of-service (DoS) attack known as the “Rapid Reset” attack.
Rapid Reset Attack Mechanics
In a standard DDoS attack, attackers flood the server with an overwhelming number of requests. The Rapid Reset attack takes a different approach: it exploits the HTTP/2 protocol by repeatedly sending RST_STREAM frames, unilaterally canceling a batch of initiated streams. The attacker exploits this mechanism by repeating this multiple times to overload the server with a flood of RST_STREAM frames, thereby using up considerable system resources.
On the network side, this attack can clog up bandwidth with SYN packets and the RST_STREAM frames. Combined, these factors can stop the server from processing and responding to legitimate requests efficiently. The result is serious server disruption that can render it unresponsive at a low cost.
For those keen on the deep technical details, the vulnerability has been described thoroughly in Google’s blog post and has been identified as CVE-2023-44487.
nginx Recommendations
nginx has issued a series of configuration recommendations to help mitigate such attacks. These are meant to strike a balance between maintaining performance and enhancing security. Below are the key directives to configure in your nginx setup:
- Keepalive requests: Set the keepalive_requests parameter to 1000. This limits the number of requests that can be made over a single keepalive connection, reducing the risk of resource exhaustion.
- Concurrent streams: Configure the http2_max_concurrent_streams to 128. This restricts the maximum number of concurrently open HTTP/2 streams, making it more challenging for attackers to overload the system.
- Rate limiting and connection limiting: Implement the limit_conn and limit_req directives to manage the number of connections and requests from a single client. limit_conn controls the number of connections allowed from a single client, while limit_req regulates the rate at which requests will be processed within a set timeframe from a single client.
Gcore’s Commitment to Countering Rapid Reset Attacks
For those of our current customers who are already familiar with our technology stack, you’ll know that we place a strong emphasis on using nginx in our infrastructure. We take security recommendations seriously, proactively responding to vulnerabilities and associated guidelines, and that goes for Rapid Reset DDoS attacks too.
Our team of infrastructure specialists have conducted rigorous internal tests to assess the potential ramifications of such attacks on our systems. We have also audited our configurations to ensure they align with nginx’s security recommendations. Our analysis affirms that these measures provide a robust layer of protection against HTTP/2-specific threats.
We are actively working on additional safeguards tailored for clients who have specialized configurations. Maintaining the integrity and security of our infrastructure remains a top priority.
Forthcoming nginx Patch and Gcore Response
nginx is in the process of developing a patch specifically designed to bolster safeguards against flood attacks. This forthcoming update will introduce a cap on the number of new HTTP/2 streams that can be created within a single event loop. The limit will be set at twice the value specified in the http2_max_concurrent_streams directive. Additionally, the patch is designed to counteract scenarios where streams are rapidly reset upon initiation—a tactic commonly used in the Rapid Reset attack.
At Gcore, we’re committed to staying aligned with nginx’s evolving best practices and to continuously fortifying our infrastructure against emerging threats. We encourage our clients to stay informed about upcoming updates regarding Rapid Reset attacks.
Conclusion
In the world of cybersecurity, complacency can be expensive, as Rapid Reset attacks illustrate. The recent HTTP/2 vulnerabilities have been a wake-up call for the cybersecurity industry, reiterating the continual need for monitoring and preventive steps. It demonstrates Gcore’s outstanding existing products and best practices, and allows us to find opportunities for further improvement.
Gcore protects against even previously unknown threats, and ensures that your security is the best that money can buy. Our DDoS Protection can save your business time, money, and stress down the line. Be on the cutting edge of defense—request a free consultation with a Gcore security expert!
