GeoIP filtering at 400 Gbps with XDP on an eBPF space

In this article, we’ll share our experience on how to filter packets by location on an eBPF (extended Berkeley Packet Filter) space while maintaining high performance at a low computational cost. We’ll also share benchmarking results that confirm the efficiency of the solution.

Challenge: filtering packets based on the traffic’s source country

In our infrastructure, we use the XDP framework to secure our customers against volumetric DDOS attacks. This architecture allows fast packet processing in the Linux kernel because XDP bypasses most of the network stack in the kernel. The firewall is highly configurable, and our customers can choose to filter packets from specific countries while maintaining high performance.

In this article, we’ll describe how we keep the firewall’s performance high when we need to geo-filter traffic or to:

• process millions of packets per second
• store block or allow lists for countries
• keep computational costs low

You can use our solution if you face similar use cases. Those would be, for example, if your clients need to allow requests from one specific country (allow listing), block a specific location where malicious traffic comes from during a DDoS attack, or limit requests from other geolocations to DNS servers in a certain point of presence.

Solution design: build GeoIP databases in the eBPF by maps

We decided to use GeoIP databases for the geo-filtering solution. Such databases consist of IP address prefixes and country codes in the ISO 3166-1 format to which these addresses belong. For example:

1.2.3.0/24 → NZ1.2.3.4/32 → BE

GeoIP databases are very common. Many companies on the Internet allow you to buy such databases for a small cost and, sometimes, even to use them for free.

To implement the GeoIP databases for eBPF, we used maps, a key-value type of storage that enables data exchange between the user space and the kernel space.

The only difficulty that has to be solved is finding a way to maintain high performance at a low computational cost. To do this, we chose a wide bitmap so that each bit corresponds to one country.

LPM map: add GeoIP data to the eBPF

We need to find the most specific IP block that our source IP belongs to, as well as which country that IP block is from. The LPM (Longest prefix match) algorithm is best for this. In addition, eBPF supports the BPF_MAP_TYPE_LPM_TRIE map type, which uses this algorithm.

We have assigned each country code an ordinal number (ID), starting from zero, and added value pairs of IP addresses with IDs to the eBPF map. For instance, the IP addresses of New Zealand (NZ) are assigned ID 0. Those from Belgium (BE) are assigned 1:

1.2.3.0/24 → 01.2.3.4/32 → 1

etc.

Bitmap: check packets for compliance with user policy

We also added the custom policy for countries as a bit value on the map. The allow policy is 0 bits, and the blocking policy is 1 bit. So, for example, if we decide to block New Zealand (NZ), Dominica (DM), and Lithuania (LT), the bitmap will look as follows:

When using a bitmap, traffic filtering by GeoIP will be performed as follows:

1. Find the corresponding country ID by IP address by using the LPM map.
2. Verify the user policy for the country using the bitmap.
3. Allow or block the packet.

Benchmarks: checking packets for GeoIP match

We ran tests to evaluate how well the performance was maintained. The table below shows the results, including the overhead of our entire firewall pipeline. If you implement a BPF program with just GeoIP blocking, it’ll very likely be way faster.

Equipment used:

• CPU: 2 × Intel® Xeon® Gold 6348 CPU @ 2.60 GHz
• NIC: 2 × Intel ® E810-2cqda2 (100G 2 ports)

As you can see, the smaller the packet size is, the higher the load on the firewall CPU is. At the same time, large packets can be filtered out with a high line rate.