38C3

io_uring, eBPF, XDP and AF_XDP
2024-12-28 , Saal GLITCH
Language: English

Modern high-performance networking APIs on Linux - beyond the classic BSD sockets API.


For many decades, application software programmers have been using the venerable BSD sockets API to make their applications communicate over (at least IP based) networks. Linux has supported TCP/UDP sockets ever since it had a network stack back in the 1990s. While those socket system call APIs are simple and straight-forward,
they were designed at a time when internet access happened over dial-up modems and LANs had no more than 10 MBit/s, if at all.

With today's Multi-Gigabit speeds even in consumer equipment and 40GE/100GE network interface cards for servers being a reality, using those 1980s BSD/POSIX socket interfaces comes with a huge performance penalty.

Some specific use cases like single-flow high-throughput TCP on an end-node have seen optimizations that are transparent to the user (TCP segmentation offloading). But there's only so far you can go with that.

Parts of the industry have proposed user-space network stacks built on DPDK - but then basically you do no longer use the Linux kernel network stack at all, and subsequently have none of its features. Yes, that can
be fast, but Linux becomes nothing but a bootloader, and you have to implement everything from Ethernet to ARP and IP+TCP in your application.

The answer of the Linux kernel community over the last 5+ years has been various new mechanisms and interfaces in the Linux kernel that revolutionize the way how applications can achieve higher network I/O
throughput - whether an end host (server/client) or a packet-forwarding router/bridge/firewall.

This talk provides a brief but deeply technical introduction into the problem space, the new mechanisms and their use cases.

While the talk discusses features of the Linux kernel, we do not discuss their internals; the focus is on how those mechanisms can be used by applications.

Harald Welte is a recovering Linux Kernel hacker with a history of hacking and developing (mostly open source) hardware, firmware and system-level software. Working in network protocols ever since the 1990s, he has dedicated the last 15 years of his life to spearheading Free and Open Source software in the domain of cellular networks.

He has worked extensively on implementations of cellular protocol stacks on virtually any interface at any protocol level from 2G to 4G - most of that in the context of the Osmocom project, which he co-started.

You can find more information on the (sadly rather infrequently updated) blog, and the usually more up-to-date fediverse presence at https://chaos.social/@LaF0rge