Surpassing 10Gb/s over Tailscale · Tailscale
Hello, it’s us once more. You would possibly keep in mind us from when we made important performance-related modifications to wireguard-go, the userspace WireGuard® implementation that Tailscale makes use of. We’re releasing a set of modifications that additional improves shopper throughput on Linux. We intend to upstream these modifications to WireGuard as we did with the earlier set of modifications, which have since landed upstream.
With this new set of modifications, Tailscale joins the 10Gb/s membership on naked metallic Linux, and wireguard-go pushes previous (for now) the in-kernel WireGuard implementation on that {hardware}. How did we do it? By means of UDP segmentation offload and checksum optimizations. You possibly can expertise these enhancements within the present unstable Tailscale shopper launch, and in addition in Tailscale v1.40, out there within the coming days. Proceed studying to study extra, or leap right down to the Results part in case you simply need numbers.
Background
The info airplane in Tailscale is constructed atop wireguard-go, a userspace WireGuard implementation written in Go. wireguard-go acts as a pipeline, receiving packets from the working system by way of a TUN interface. It encrypts them, assuming a sound peer exists for his or her addressed vacation spot, and sends them to a distant peer by way of a UDP socket. The circulate in the wrong way is comparable. Packets from legitimate friends are decrypted after being learn from a UDP socket, then are written again to the kernel’s TUN interface driver.
The modifications we made in v1.36 modified this pipeline, enabling packet vectors to circulate end-to-end, fairly than single packets. The methods utilized on each ends of the pipeline lowered the variety of system calls per packet, and on the TUN aspect they lowered the price of shifting a packet by means of the kernel networking stack.
This greatly improved throughput, and we have now continued to construct upon it with the modifications we describe on this submit.
Baseline
Disclaimer about benchmarks: This submit incorporates benchmarks! These benchmarks are reproducible on the time of writing, and we offer particulars in regards to the environments we ran them in. However benchmark outcomes are likely to fluctuate throughout environments, and so they additionally are likely to go stale as time progresses. Your mileage could fluctuate.
Earlier than entering into the small print of what we modified, we have to report some baselines for later comparability. These benchmarks are performed utilizing iperf3, as single stream TCP checks, with cubic congestion management. All hosts are operating Ubuntu 22.04 with the newest out there Linux kernel for that distribution.
We baselined throughput for wireguard-go@052af4a and in-kernel WireGuard. These checks had been performed between two pairs of hosts:
- 2 x AWS c6i.8xlarge occasion varieties
- 2 x “naked metallic” servers powered by i5-12400 CPUs & Mellanox MCX512A-ACAT NICs
For consistency, the c6i.8xlarge occasion kind is identical we used within the precursory blog post. The cases are in the identical area and availability zone:
ubuntu@c6i-8xlarge-1:~$ ec2metadata | grep -E 'instance-type:|availability-zone:'
availability-zone: us-east-2b
instance-type: c6i.8xlarge
ubuntu@c6i-8xlarge-2:~$ ec2metadata | grep -E 'instance-type:|availability-zone:'
availability-zone: us-east-2b
instance-type: c6i.8xlarge
ubuntu@c6i-8xlarge-1:~$ ping 172.31.23.111 -c 5 -q
PING 172.31.23.111 (172.31.23.111) 56(84) bytes of knowledge.
--- 172.31.23.111 ping statistics ---
5 packets transmitted, 5 acquired, 0% packet loss, time 4094ms
rtt min/avg/max/mdev = 0.109/0.126/0.168/0.022 ms
We’ve added the i5-12400 methods for a naked metallic comparability with interfaces working above 10Gb/s. The i5-12400 CPU is a contemporary (launched Q1 2022) desktop-class chip, out there for $183 USD on the time of writing. The Mellanox NICs are linked at 25Gb/s by way of a direct connect copper (DAC) cable:
jwhited@i5-12400-1:~$ lscpu | grep Mannequin.identify && cpupower frequency-info -d && cpupower frequency-info -p
Mannequin identify: twelfth Gen Intel(R) Core(TM) i5-12400
analyzing CPU 0:
driver: intel_pstate
analyzing CPU 0:
present coverage: frequency needs to be inside 800 MHz and 5.60 GHz.
The governor "efficiency" could determine which velocity to make use of
inside this vary.
jwhited@i5-12400-1:~$ sudo ethtool enp1s0f0np0 | grep Pace && sudo ethtool -i enp1s0f0np0 | egrep 'driver|^model'
Pace: 25000Mb/s
driver: mlx5_core
model: 5.15.0-69-generic
jwhited@i5-12400-2:~$ lscpu | grep Mannequin.identify && cpupower frequency-info -d && cpupower frequency-info -p
Mannequin identify: twelfth Gen Intel(R) Core(TM) i5-12400
analyzing CPU 0:
driver: intel_pstate
analyzing CPU 0:
present coverage: frequency needs to be inside 800 MHz and 5.60 GHz.
The governor "efficiency" could determine which velocity to make use of
inside this vary.
jwhited@i5-12400-2:~$ sudo ethtool enp1s0f0np0 | grep Pace && sudo ethtool -i enp1s0f0np0 | egrep 'driver|^model'
Pace: 25000Mb/s
driver: mlx5_core
model: 5.15.0-69-generic
jwhited@i5-12400-1:~$ ping 10.0.0.20 -c 5 -q
PING 10.0.0.20 (10.0.0.20) 56(84) bytes of knowledge.
--- 10.0.0.20 ping statistics ---
5 packets transmitted, 5 acquired, 0% packet loss, time 4078ms
rtt min/avg/max/mdev = 0.008/0.035/0.142/0.053 ms
Now for the iperf3 baseline checks.
c6i.8xlarge over in-kernel WireGuard:
ubuntu@c6i-8xlarge-1:~$ iperf3 -i 0 -c c6i-8xlarge-2-wg -t 10 -C cubic -V
iperf 3.9
Linux c6i-8xlarge-1 5.19.0-1022-aws #23~22.04.1-Ubuntu SMP Fri Mar 17 15:38:24 UTC 2023 x86_64
Management connection MSS 1368
Time: Wed, 12 Apr 2023 23:56:53 GMT
Connecting to host c6i-8xlarge-2-wg, port 5201
Cookie: 3jzl3sa34hkbpwbmg4dbfh6aovbknnw7x5hn
TCP MSS: 1368 (default)
[ 5] native 10.9.9.1 port 51194 linked to 10.9.9.2 port 5201
Beginning Check: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second take a look at, tos 0
[ ID] Interval Switch Bitrate Retr Cwnd
[ 5] 0.00-10.00 sec 3.11 GBytes 2.67 Gbits/sec 51 1.00 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
Check Full. Abstract Outcomes:
[ ID] Interval Switch Bitrate Retr
[ 5] 0.00-10.00 sec 3.11 GBytes 2.67 Gbits/sec 51 sender
[ 5] 0.00-10.05 sec 3.11 GBytes 2.66 Gbits/sec receiver
CPU Utilization: native/sender 5.1% (0.3percentu/4.8percents), distant/receiver 11.2% (0.2percentu/11.0percents)
snd_tcp_congestion cubic
rcv_tcp_congestion cubic
c6i.8xlarge over wireguard-go@052af4a:
ubuntu@c6i-8xlarge-1:~$ iperf3 -i 0 -c c6i-8xlarge-2-wg -t 10 -C cubic -V
iperf 3.9
Linux c6i-8xlarge-1 5.19.0-1022-aws #23~22.04.1-Ubuntu SMP Fri Mar 17 15:38:24 UTC 2023 x86_64
Management connection MSS 1368
Time: Wed, 12 Apr 2023 23:55:42 GMT
Connecting to host c6i-8xlarge-2-wg, port 5201
Cookie: zlcrq3xqyr6cfmrtysrm42xcg3bbjzir3qob
TCP MSS: 1368 (default)
[ 5] native 10.9.9.1 port 54410 linked to 10.9.9.2 port 5201
Beginning Check: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second take a look at, tos 0
[ ID] Interval Switch Bitrate Retr Cwnd
[ 5] 0.00-10.00 sec 6.21 GBytes 5.34 Gbits/sec 0 3.15 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
Check Full. Abstract Outcomes:
[ ID] Interval Switch Bitrate Retr
[ 5] 0.00-10.00 sec 6.21 GBytes 5.34 Gbits/sec 0 sender
[ 5] 0.00-10.04 sec 6.21 GBytes 5.31 Gbits/sec receiver
CPU Utilization: native/sender 8.6% (0.2percentu/8.4percents), distant/receiver 11.8% (0.6percentu/11.2percents)
snd_tcp_congestion cubic
rcv_tcp_congestion cubic
i5-12400 over in-kernel WireGuard:
jwhited@i5-12400-1:~$ iperf3 -i 0 -c i5-12400-2-wg -t 10 -C cubic -V
iperf 3.9
Linux i5-12400-1 5.15.0-69-generic #76-Ubuntu SMP Fri Mar 17 17:19:29 UTC 2023 x86_64
Management connection MSS 1368
Time: Wed, 12 Apr 2023 23:41:44 GMT
Connecting to host i5-12400-2-wg, port 5201
Cookie: hqkn7s3scipxku5rzpcgqt4rakutkpwybtvx
TCP MSS: 1368 (default)
[ 5] native 10.9.9.1 port 48564 linked to 10.9.9.2 port 5201
Beginning Check: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second take a look at, tos 0
[ ID] Interval Switch Bitrate Retr Cwnd
[ 5] 0.00-10.00 sec 13.7 GBytes 11.8 Gbits/sec 8725 753 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
Check Full. Abstract Outcomes:
[ ID] Interval Switch Bitrate Retr
[ 5] 0.00-10.00 sec 13.7 GBytes 11.8 Gbits/sec 8725 sender
[ 5] 0.00-10.04 sec 13.7 GBytes 11.7 Gbits/sec receiver
CPU Utilization: native/sender 26.3% (0.1percentu/26.2percents), distant/receiver 17.4% (0.5percentu/16.9percents)
snd_tcp_congestion cubic
rcv_tcp_congestion cubic
i5-12400 over wireguard-go@052af4a:
jwhited@i5-12400-1:~$ iperf3 -i 0 -c i5-12400-2-wg -t 10 -C cubic -V
iperf 3.9
Linux i5-12400-1 5.15.0-69-generic #76-Ubuntu SMP Fri Mar 17 17:19:29 UTC 2023 x86_64
Management connection MSS 1368
Time: Wed, 12 Apr 2023 23:39:22 GMT
Connecting to host i5-12400-2-wg, port 5201
Cookie: ohzzlzkcvnk45ya32vm75ezir6njydqwipkl
TCP MSS: 1368 (default)
[ 5] native 10.9.9.1 port 52486 linked to 10.9.9.2 port 5201
Beginning Check: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second take a look at, tos 0
[ ID] Interval Switch Bitrate Retr Cwnd
[ 5] 0.00-10.00 sec 9.74 GBytes 8.36 Gbits/sec 507 3.01 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
Check Full. Abstract Outcomes:
[ ID] Interval Switch Bitrate Retr
[ 5] 0.00-10.00 sec 9.74 GBytes 8.36 Gbits/sec 507 sender
[ 5] 0.00-10.05 sec 9.74 GBytes 8.32 Gbits/sec receiver
CPU Utilization: native/sender 11.7% (0.1percentu/11.6percents), distant/receiver 6.5% (0.2percentu/6.3percents)
snd_tcp_congestion cubic
rcv_tcp_congestion cubic
With the baselines captured, let’s take a look at some profiling knowledge to grasp the place we could also be bottlenecked.
Linux perf and flame graphs
The flame graphs beneath had been rendered from perf data. They symbolize the quantity of CPU time spent for a given operate/stack. The broader the operate, the costlier it (and/or its kids) are. These are interactive; you may click on to zoom and hover to see percentages.
This primary graph is from the iperf3 sender:
Notably, extra time is being spent sending UDP packets than encrypting their payloads. Let’s check out the receiver:
The receiver seems pretty comparable, with UDP reception being practically equal in time spent relative to decryption.
We’re utilizing the {ship,recv}mmsg() (two m’s) system calls, which assist to amortize the price of making a syscall. Nonetheless, on the kernel aspect of the system name, we see {ship,recv}mmsg() calls into {ship,recv}msg() (single m). Which means that we nonetheless pay the price of traversing the kernel networking stack for each single packet, as a result of the kernel aspect merely iterates by means of the batch.
On the TUN aspect of wireguard-go, we make use of TCP segmentation offload (TSO) and generic obtain offload (GRO), which allow a number of TCP segments to move by means of the kernel stack as a single phase:
What we’d like is one thing comparable, however for UDP. Enter UDP generic segmentation offload.
UDP generic segmentation offload (GSO)
UDP GSO permits the kernel to delay segmentation of a batch of UDP datagrams similarly to the TCP variant, decreasing the CPU cycles per byte price of traversing the networking stack. Linux help was authored by Willem de Bruijn and introduced into the kernel in v4.18. UDP GSO was propelled by the adoption of QUIC in the datacenter, however its advantages will not be restricted to QUIC. It’s best described by a part of its abstract commit message:
Segmentation offload reduces cycles/byte for giant packets by
amortizing the price of protocol stack traversal.This patchset implements GSO for UDP. A course of can concatenate and
submit a number of datagrams to the identical vacation spot in a single ship name
by setting socket possibility SOL_UDP/UDP_SEGMENT with the phase dimension,
or passing a similar cmsg at ship time.The stack will ship your entire massive (as much as community layer max dimension)
datagram by means of the protocol layer. On the GSO layer, it’s damaged
up in particular person segments. All obtain the identical community layer header
and UDP src and dst port. All however the final phase have the identical UDP
header, however the final could differ in size and checksum.”
After implementing UDP GSO on the UDP socket aspect of wireguard-go, the transmit path now seems like this:
However what in regards to the obtain path? It will be ultimate to optimize each instructions. Paolo Abeni authored UDP generic obtain offload (GRO) help, and it was introduced into the Linux kernel in v5.0. With UDP GRO the obtain path now seems like this:
Updates to the UDP man page for these new options ultimately arrived, wherein an vital requirement for UDP GSO is described:
Segmentation offload is dependent upon checksum offload, as datagram checksums are computed after segmentation.
Checksum offload is broadly supported throughout ethernet gadgets at this time. It additionally reduces the price of the kernel networking stack, as ethernet gadgets are likely to have specialised {hardware} that may be very environment friendly at computing RFC1071 checksums. It’s typically paired with segmentation offload, which as the person web page describes, could have to be carried out by the layer performing segmentation.
In actual fact, we have already got to dump checksumming within the TCP segmentation offload implementation in wireguard-go. The kernel palms us a “monster phase,” which we’re chargeable for segmenting. This contains calculating checksums for the person segments.
TUN checksum offload
If we glance again on the flame graphs we’ll discover the operate chargeable for computing the web checksum as a part of the present TCP segmentation offloading (tun.checksum(), inlined with tun.checksumNoFold()). It contributes to a modest share of perf samples (6.6% on the sender) earlier than making any modifications. After decreasing the price of the kernel’s UDP stack, the relative price of TUN checksum offload will increase with throughput, and it turns into our subsequent candidate to optimize.
The present tun.checksumNoFold() operate was this:
// TODO: Discover SIMD and/or different meeting optimizations.
func checksumNoFold(b []byte, preliminary uint64) uint64 {
ac := preliminary
i := 0
n := len(b)
for n >= 4 {
ac += uint64(binary.BigEndian.Uint32(b[i : i+4]))
n -= 4
i += 4
}
for n >= 2 {
ac += uint64(binary.BigEndian.Uint16(b[i : i+2]))
n -= 2
i += 2
}
if n == 1 {
ac += uint64(b[i]) << 8
}
return ac
}
It’s chargeable for summing the bytes in b with preliminary, and returning the sum as a uint64. Web checksums are uint16 values, which the return from this operate will get folded into. However why are we returning a uint64 to start with? As a result of there may be already one current optimization current right here. We sum 4 bytes at a time, as an alternative of two. This may lower checksum price in half. RFC 1071 describes the mathematical properties that allow this optimization together with the idea of folding:
On machines which have word-sizes which are multiples of 16 bits,
it’s attainable to develop much more environment friendly implementations.
As a result of addition is associative, we would not have to sum the
integers within the order they seem within the message. As an alternative we
can add them in “parallel” by exploiting the bigger phrase dimension.To compute the checksum in parallel, merely do a 1’s complement
addition of the message utilizing the native phrase dimension of the
machine. For instance, on a 32-bit machine we are able to add 4 bytes at
a time: [A,B,C,D]+’… When the sum has been computed, we “fold”
the lengthy sum into 16 bits by including the 16-bit segments. Every
16-bit addition could produce new end-around carries that have to be
added.
There may be another low-hanging optimization out there to us — unwinding the loops! Checking the size of b after each summation is pricey overhead, particularly for bigger packets. RFC 1071 additionally describes this optimization:
To scale back the loop overhead, it’s typically helpful to “unwind” the
interior sum loop, replicating a sequence of addition instructions inside
one loop traversal. This system typically offers important
financial savings, though it might complicate the logic of this system
significantly.
After making use of some unwinding we find yourself with this operate, which has some repetitive bits omitted for the sake of brevity:
// TODO: Discover SIMD and/or different meeting optimizations.
// TODO: Check native endian hundreds. See RFC 1071 part 2 half B.
func checksumNoFold(b []byte, preliminary uint64) uint64 {
ac := preliminary
for len(b) >= 128 {
ac += uint64(binary.BigEndian.Uint32(b[:4]))
ac += uint64(binary.BigEndian.Uint32(b[4:8]))
// (omitted) continues to 128
b = b[128:]
}
if len(b) >= 64 {
ac += uint64(binary.BigEndian.Uint32(b[:4]))
ac += uint64(binary.BigEndian.Uint32(b[4:8]))
// (omitted) continues to 64
b = b[64:]
}
if len(b) >= 32 {
ac += uint64(binary.BigEndian.Uint32(b[:4]))
ac += uint64(binary.BigEndian.Uint32(b[4:8]))
// (omitted) continues to 32
b = b[32:]
}
if len(b) >= 16 {
ac += uint64(binary.BigEndian.Uint32(b[:4]))
ac += uint64(binary.BigEndian.Uint32(b[4:8]))
ac += uint64(binary.BigEndian.Uint32(b[8:12]))
ac += uint64(binary.BigEndian.Uint32(b[12:16]))
b = b[16:]
}
if len(b) >= 8 {
ac += uint64(binary.BigEndian.Uint32(b[:4]))
ac += uint64(binary.BigEndian.Uint32(b[4:8]))
b = b[8:]
}
if len(b) >= 4 {
ac += uint64(binary.BigEndian.Uint32(b))
b = b[4:]
}
if len(b) >= 2 {
ac += uint64(binary.BigEndian.Uint16(b))
b = b[2:]
}
if len(b) == 1 {
ac += uint64(b[0]) << 8
}
return ac
}
This optimization lowered the run time of the operate by ~57%, as evidenced by the output of benchstat:
$ benchstat previous.txt new.txt
goos: linux
goarch: amd64
pkg: golang.zx2c4.com/wireguard/tun
cpu: twelfth Gen Intel(R) Core(TM) i5-12400
│ previous.txt │ new.txt │
│ sec/op │ sec/op vs base │
Checksum/64-12 10.670n ± 2% 4.769n ± 0% -55.30% (p=0.000 n=10)
Checksum/128-12 19.665n ± 2% 8.032n ± 0% -59.16% (p=0.000 n=10)
Checksum/256-12 37.68n ± 1% 16.06n ± 0% -57.37% (p=0.000 n=10)
Checksum/512-12 76.61n ± 3% 32.13n ± 0% -58.06% (p=0.000 n=10)
Checksum/1024-12 160.55n ± 4% 64.25n ± 0% -59.98% (p=0.000 n=10)
Checksum/1500-12 231.05n ± 7% 94.12n ± 0% -59.26% (p=0.000 n=10)
Checksum/2048-12 309.5n ± 3% 128.5n ± 0% -58.48% (p=0.000 n=10)
Checksum/4096-12 603.8n ± 4% 257.2n ± 0% -57.41% (p=0.000 n=10)
Checksum/8192-12 1185.0n ± 3% 515.5n ± 0% -56.50% (p=0.000 n=10)
Checksum/9000-12 1328.5n ± 5% 564.8n ± 0% -57.49% (p=0.000 n=10)
Checksum/9001-12 1340.5n ± 3% 564.8n ± 0% -57.87% (p=0.000 n=10)
geomean 185.3n 77.99n -57.92%
This optimization additionally translated to a ten% throughput enchancment for a number of the environments we examined. Now, on to the general outcomes.
Outcomes
Making use of UDP segmentation offload, UDP obtain coalescing, and checksum unwinding resulted in important throughput enhancements for wireguard-go, and so additionally within the Tailscale shopper.
wireguard-go (c6i.8xlarge) with UDP GSO, GRO, and checksum unwinding:
ubuntu@c6i-8xlarge-1:~$ iperf3 -i 0 -c c6i-8xlarge-2-wg -t 10 -C cubic -V
iperf 3.9
Linux c6i-8xlarge-1 5.19.0-1022-aws #23~22.04.1-Ubuntu SMP Fri Mar 17 15:38:24 UTC 2023 x86_64
Management connection MSS 1368
Time: Wed, 12 Apr 2023 23:58:19 GMT
Connecting to host c6i-8xlarge-2-wg, port 5201
Cookie: efpxfeszrxxsjdo643josagi2akj3f2lcmdh
TCP MSS: 1368 (default)
[ 5] native 10.9.9.1 port 35218 linked to 10.9.9.2 port 5201
Beginning Check: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second take a look at, tos 0
[ ID] Interval Switch Bitrate Retr Cwnd
[ 5] 0.00-10.00 sec 8.53 GBytes 7.32 Gbits/sec 0 3.14 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
Check Full. Abstract Outcomes:
[ ID] Interval Switch Bitrate Retr
[ 5] 0.00-10.00 sec 8.53 GBytes 7.32 Gbits/sec 0 sender
[ 5] 0.00-10.05 sec 8.53 GBytes 7.29 Gbits/sec receiver
CPU Utilization: native/sender 10.4% (0.2percentu/10.2percents), distant/receiver 20.8% (0.8percentu/20.0percents)
snd_tcp_congestion cubic
rcv_tcp_congestion cubic
wireguard-go (i5-12400) with UDP GSO, GRO, and checksum unwinding:
jwhited@i5-12400-1:~$ iperf3 -i 0 -c i5-12400-2-wg -t 10 -C cubic -V
iperf 3.9
Linux i5-12400-1 5.15.0-69-generic #76-Ubuntu SMP Fri Mar 17 17:19:29 UTC 2023 x86_64
Management connection MSS 1368
Time: Wed, 12 Apr 2023 23:42:52 GMT
Connecting to host i5-12400-2-wg, port 5201
Cookie: q6hm54yvcbxdrsnh2foexkunzdsdudwy5wfj
TCP MSS: 1368 (default)
[ 5] native 10.9.9.1 port 43006 linked to 10.9.9.2 port 5201
Beginning Check: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second take a look at, tos 0
[ ID] Interval Switch Bitrate Retr Cwnd
[ 5] 0.00-10.00 sec 15.2 GBytes 13.0 Gbits/sec 1212 3.01 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
Check Full. Abstract Outcomes:
[ ID] Interval Switch Bitrate Retr
[ 5] 0.00-10.00 sec 15.2 GBytes 13.0 Gbits/sec 1212 sender
[ 5] 0.00-10.04 sec 15.2 GBytes 13.0 Gbits/sec receiver
CPU Utilization: native/sender 18.9% (0.3percentu/18.6percents), distant/receiver 4.0% (0.2percentu/3.8percents)
snd_tcp_congestion cubic
rcv_tcp_congestion cubic
With these efficiency enhancements, Tailscale joins the 10Gb/s membership on naked metallic Linux, and wireguard-go pushes previous (for now) the in-kernel WireGuard implementation on that {hardware}. The AWS c6i.8xlarge cases hit a wall at 7.3Gb/s that seems to be a man-made restrict of the underlay community. We had been unable to exceed the same bitrate for UDP packets with no WireGuard concerned.
Word about UDP GSO in-hardware
Similar to TSO is the “in-hardware” cousin of GSO, UDP GSO has the same variant, listed as tx-udp-segmentation by ethtool:
jwhited@i5-12400-1:~$ ethtool -k enp1s0f0np0 | grep udp-seg
tx-udp-segmentation: on
It extends the delayed segmentation of datagrams by means of to the gadget, and our transmit path circulate now seems like this:
This {hardware} help exists within the 25G NICs we used on the i5-12400 methods. It did enhance throughput barely (5%) for that {hardware}, nevertheless it actually shined for a number of the older era CPUs. For one instance, right here’s an E3-1230-V2 (launched Q2 2012) system with the identical NIC.
E3-1230-V2 over wireguard-go@052af4a:
jwhited@e3-1230-v2:~$ iperf3 -i 0 -c i5-12400-1-wg -t 10 -C cubic -V
iperf 3.9
Linux e3-1230-v2 5.15.0-67-generic #74-Ubuntu SMP Wed Feb 22 14:14:39 UTC 2023 x86_64
Management connection MSS 1368
Time: Thu, 13 Apr 2023 02:27:23 GMT
Connecting to host i5-12400-1-wg, port 5201
Cookie: pcfb7wqlh653l3r6r4oxxjenfxh4hdlqowho
TCP MSS: 1368 (default)
[ 5] native 10.9.9.3 port 35310 linked to 10.9.9.1 port 5201
Beginning Check: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second take a look at, tos 0
[ ID] Interval Switch Bitrate Retr Cwnd
[ 5] 0.00-10.00 sec 3.91 GBytes 3.36 Gbits/sec 0 3.09 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
Check Full. Abstract Outcomes:
[ ID] Interval Switch Bitrate Retr
[ 5] 0.00-10.00 sec 3.91 GBytes 3.36 Gbits/sec 0 sender
[ 5] 0.00-10.05 sec 3.91 GBytes 3.34 Gbits/sec receiver
CPU Utilization: native/sender 10.6% (0.4percentu/10.2percents), distant/receiver 2.0% (0.0percentu/2.0percents)
snd_tcp_congestion cubic
rcv_tcp_congestion cubic
E3-1230-V2 over wireguard-go with UDP GSO, GRO, checksum unwinding, and tx-udp-segmentation off:
jwhited@e3-1230-v2:~$ sudo ethtool -Okay enp1s0f0np0 tx-udp-segmentation off
jwhited@e3-1230-v2:~$ iperf3 -i 0 -c i5-12400-1-wg -t 10 -C cubic -V
iperf 3.9
Linux e3-1230-v2 5.15.0-67-generic #74-Ubuntu SMP Wed Feb 22 14:14:39 UTC 2023 x86_64
Management connection MSS 1368
Time: Thu, 13 Apr 2023 02:28:12 GMT
Connecting to host i5-12400-1-wg, port 5201
Cookie: 6rtbzadj2on7igc7bt2hfhphdg2ebfgwxzim
TCP MSS: 1368 (default)
[ 5] native 10.9.9.3 port 58036 linked to 10.9.9.1 port 5201
Beginning Check: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second take a look at, tos 0
[ ID] Interval Switch Bitrate Retr Cwnd
[ 5] 0.00-10.00 sec 5.65 GBytes 4.86 Gbits/sec 0 3.14 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
Check Full. Abstract Outcomes:
[ ID] Interval Switch Bitrate Retr
[ 5] 0.00-10.00 sec 5.65 GBytes 4.86 Gbits/sec 0 sender
[ 5] 0.00-10.04 sec 5.65 GBytes 4.84 Gbits/sec receiver
CPU Utilization: native/sender 19.1% (0.6percentu/18.5percents), distant/receiver 1.9% (0.1percentu/1.8percents)
snd_tcp_congestion cubic
rcv_tcp_congestion cubic
E3-1230-V2 over wireguard-go with UDP GSO, GRO, checksum unwinding, and tx-udp-segmentation on:
jwhited@e3-1230-v2:~$ sudo ethtool -Okay enp1s0f0np0 tx-udp-segmentation on
jwhited@e3-1230-v2:~$ iperf3 -i 0 -c i5-12400-1-wg -t 10 -C cubic -V
iperf 3.9
Linux e3-1230-v2 5.15.0-67-generic #74-Ubuntu SMP Wed Feb 22 14:14:39 UTC 2023 x86_64
Management connection MSS 1368
Time: Thu, 13 Apr 2023 02:28:58 GMT
Connecting to host i5-12400-1-wg, port 5201
Cookie: lod6fulhls3wvtqy7uoakmldifdtcc3nbvfv
TCP MSS: 1368 (default)
[ 5] native 10.9.9.3 port 46724 linked to 10.9.9.1 port 5201
Beginning Check: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second take a look at, tos 0
[ ID] Interval Switch Bitrate Retr Cwnd
[ 5] 0.00-10.00 sec 7.68 GBytes 6.59 Gbits/sec 2 3.12 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
Check Full. Abstract Outcomes:
[ ID] Interval Switch Bitrate Retr
[ 5] 0.00-10.00 sec 7.68 GBytes 6.59 Gbits/sec 2 sender
[ 5] 0.00-10.05 sec 7.68 GBytes 6.56 Gbits/sec receiver
CPU Utilization: native/sender 25.6% (1.0percentu/24.6percents), distant/receiver 8.0% (0.3percentu/7.7percents)
snd_tcp_congestion cubic
rcv_tcp_congestion cubic
That’s a 35% improve in throughput with {hardware} UDP segmentation offload enabled, and practically a 2x improve over the baseline.
Conclusions
Persevering with on our journey to enhance packet processing overhead led us to find and use comparatively younger Linux kernel options. We made use of UDP generic segmentation offload, UDP generic obtain offload, and checksum loop unwinding, enabling us to succeed in a brand new milestone — surpassing 10Gb/s over Tailscale.
Due to Adrian Dewhurst for his detailed evaluation and suggestions, to Jason A. Donenfeld for his ongoing evaluation of our patches, and to our designer Danny Pagano for the illustrations.