I'll talk about UDP and how it can make quick, a little bit quicker, in particular, on optimizing
Firefox's HTTP3 IO stack.
Quick introduction about myself, a max software engineer at Mozilla, I'm working on the
HTTP3 and Quick Stack in Firefox.
You can reach me with Mailid Max-Maisin.d, but also here in the hallway track or anywhere
around.
Cool.
MX-in pretty much everywhere, you find me on GitHub, and then you might have seen me here
talking about Piotrubian networking or Kubernetes and Prometheus in a past life.
Okay, so what does Firefox?
You don't have the time for this.
What is quick?
Quick is a general purpose, transport protocol.
It's running on top of UDP, and the big thing here is that quick comes with its own encryption,
and it's the encryption both the data itself and the metadata, so the protocol data.
We're just very powerful, and we'll go a little bit into that to anybody.
It does connection establishment within one RTT, so you can send your first request after the first
RTT, and then in ideal cases, even on consecutive connections, you can sometimes do zero RTT,
which is wonderful, especially in a web context where they can see really matters.
It is stream-based for those familiar with HTTP2 streams, for example.
It does not have the problem of head-of-line blocking here, as the stream mechanism is built into the
network protocol itself, and not built on top of it.
It has fancy features, like, for example, connection migration, so let's say you're at home,
in your Wi-Fi, and you're going outside, and you're switching to your 5G,
then the connection can migrate between those two networks.
It's easy to evolve.
One big thing here is that a lot of quick is encrypted, especially also the transport part,
and those boxes in the middle cannot make assumptions on a lot of properties of the protocol,
and thus cannot bake it into their processes, and thus it's easier to evolve the quick protocol at the two endpoints.
And relevant for this talk, it's often implemented in user space, on top of UDP, always on top of UDP,
most cases implemented in user space.
There are kernel space implementations, the folks that Microsoft do that, but yeah, we'll focus on user space quickly here.
Cool.
Let's put this a little bit in the larger picture of web protocols.
Most of you are probably familiar with the HTTP semantics.
Now, in the early days, that will go over HTTP1.
Optionally, that traffic would be encrypted with TLSSL.
That would then be on top of TCP, and then yeah, that would be shut down into UIP stack.
Later on, we had HTTP2 encryption mandatory with TLS1.2013 running on top of TCP, and then again on top of IP.
And now, the new thing here is saying semantics still for your application.
Under that, we have HTTP3, and that is then running on top of quick,
tightly integrated with TLS on top of UDP this time with IP underneath.
Cool.
So, why is all of this relevant?
Well, quick is already powering a big chunk of the internet.
Obviously, there are many perspectives.
When it comes to the internet, one is, for example, a cloud for a radar.
They're roughly seeing one third of the traffic, currently a bit quick.
Firefox roughly sees 25% of the traffic being H3, and then quick.
But yeah, different perspectives on a very complex system.
So, coming to the subject of the talk itself, quick and user space,
unsurprisingly, if you run a transport protocol unoptimized in user space,
this will not be as efficient as the heavily optimized TCP stack in the various kernels.
Especially since the TCP has been optimized all the way down to the NAIC,
we're going to get the base card.
So, yeah, very optimized stack, very hard to compete,
especially when you're in user space.
Now, in the unoptimized case, you would think that the user space
quick implementation would do one Cisco per UDP data brand.
And if you think of the internet within the MPU,
so maximum transmission unit of 1,500 bytes,
that's one Cisco for every 1,500 bytes, both receiving and sending.
That's a lot of Cisco's.
And then, just one other thing here is that in the worst case,
you get one Mac for every second packet.
So, again, a lot of Cisco's to send and receive those facts.
So, in the early days, folks at Google,
I'm posted the link here at the slides online,
so if you want to check out the papers,
folks at Google, roughly estimated a 3.5x in terms of CPU cycles per byte
when you compare a quick to TCP in the unoptimized case.
So, that's a lot.
That's a lot of CPU cycles that you're wasting by using a quick in that case.
Cool.
Yeah, I should go into this since we're covering the adorn.
On the right side, you'll see this diagram at this diagram coming up more often.
The basic case, Firefox has a quick stack.
That quick stack exchanges data grams with the operating system,
the operating system exchanges data grams with the nick,
and then the nick sends it out on the internet, right?
Very simplified version of everything.
How is this affecting applications like, for example, Firefox?
I don't expect you to read any of this, just to give you a little bit of a ballpark.
What you see here is the socket thread, the Firefox socket thread,
Firefox drives all its I-O with a single socket thread.
This socket thread here is doing a one gigabyte transfer on loopback,
transferring data over quick.
And the circled area down there, what you're seeing here,
is Firefox just allocating a receive buffer,
and then passing it down to the operating system for the operating system to fill it with EDP data.
Again, the perspective, that's a lot of CPU time in the unnoticed case.
So how can we do better than this?
The holy grail in this is segmentation offloading,
and most of you probably know this from the TCP world.
Linux and Windows both support this also on UDP.
The idea is that instead of sending one small list of 1,500 in the ideal case data gram,
down to the operating system, how about I give the operating system a very large one,
and then tell the operating system where it should divide them later on.
So where to segment them.
And then in the ideal case again, Firefox in its quick stack,
passes down the very large diagram to the operating system,
operating system to the Nick.
Then the Nick segments it into separate data grams,
puts the headers in front of it, and then sends it out on the internet.
Yeah. Linux and Windows support this.
We have some, we have zero, and actually also euro,
but that's, there are some caveats to this on Firefox nightly today,
and looking at the metrics in the wild,
reversely see on the 75% I have we see two or more packets being read.
So that's already powerful as when you read,
instead of one, two, you basically like the overhead of the Cisco,
like diminishes by 50%.
And then in the 95% I have we see 10 or more packets being read.
So on large, like, large throughput transfers,
this is already giving you a lot of benefit here.
If you compare this, oftentimes once as like,
packet trains move through the internet,
ideally they arrive together at your Nick on the receiver side.
And yeah, a lot of CDNs do like a GSO,
so segmented send of 10 packets,
and so it's unsurprising that we see the number 10 here in our 95% now.
Yeah, in terms of a kilobite, it's 2.4 on the 75th,
and then yeah, in the 95th again, we're reading quite some large chunks here.
This all gets us close to a gigabit on benchmarks,
which is, I think, quite nice for browser running on low-powered hardware.
So when we come to the holy grail of segmentation offloading,
we fall back to multi-message source calls.
And the idea here, again, depicted on the right,
instead of sending a small datagram that send multiple small datagrams
down to their operating system and then to Nick and so on.
We do this on macOS.
There's the send message and receive message calls,
but they're not entirely supported,
but they are there and they do work.
It's similar to what Linux offers with a send many message,
and they receive many message,
so it's called.
But on Linux, we have the fancier segmentation offloading.
Combining the two has not been shown fruitful so far,
so that's why we only do segmentation offloading on Linux,
for example, in a multi-message on macOS.
And here again, if you are a CPU bound,
so on a CPU bound benchmark,
receive roughly like an 11% performance improvement,
just by using these multi-message receive calls on our quick throughputs.
So that's quite massive that really shows that those source calls are very expensive for us.
Okay.
Another optimization is PLP MTUD,
that's the packetization layer path MTU discovery for datagram transports,
for those not familiar with it.
And MTU is also an abbreviation,
so it's actually the packetization layer path,
maximum transmission unit discovery for datagram transports.
In short, it's RC889.
The idea is, let's see whether our path can send larger datagrams,
and let's try it out, basically,
and fall back to smaller ones if we can't.
So for example, if you're tunneling through a VPN,
your MTU will be smaller, right?
But if you're not tunneling through a VPN,
you can send larger datagrams.
So ideally, like we support the VPN case,
we start small and then ramp up eventually.
So here, again, the picture on the right,
we have a datagram that is just slightly larger,
and we pass that to the US and vice versa and so on.
I think you get the idea.
We hope at least that we get like a 10% improvement
in terms of like the amount of bytes.
We can send per datagram,
and then, again, the overhead per datagram,
is significant being the introduced.
Yep, again, links, those lights are online here.
Then, early on, I said,
Act frequency or an XR.
We do a lot of test calls for X.
There is a draft at the ITF.
It's called a quick acknowledgement frequency.
The idea is, let's not send so many X,
but obviously in a controlled way,
so our congestion controller, for example,
doesn't suffer from it.
And then a contrived example,
just to give you an idea of where the problem is.
Let's say you have one gigabit transfer.
Let's say you want to transfer that to bytes,
so divide by eight.
We have an empty U of 1500,
divided by that, divided by two for every second packet you act.
So that gives you 40k X per second.
And in the worst case, you read every single of those X.
With a single Cisco, right?
So that's 40k Cisco's per second,
just for the acknowledgement.
It's not actually for the data, right?
All your worst case scenarios,
quick implementation will optimize that by default,
but then there is coming up the quick acknowledgement
frequency draft,
which will allow the sender to propose an act right to the receiver
in this way, basically the receiver sending less X
and then the sender having to receive less X.
So that's coming.
A couple of additional wins,
they're not directly tied to performance.
As part of this project in Firefox,
to optimize the IO path, the UDPIO path.
We've been looking around and instead reinventing the real.
We are using Quinn UDP for the UDP source call there.
And Quinn UDP is actually part of the Quinn project.
Quinn is a different rust, quick implementation.
And so we're collaborating with them,
and yeah, basing everything on their code here on the IO path.
So that's very nice.
And now Firefox's quick stack already in Rust,
does all of its IO also in Rust.
So in the memory safe language.
In addition, using all these modern source calls,
we can now get more metadata.
When we send and receive UDP data grams,
and one very important one here is ECN,
explicit congestion notification.
I'm not going to introduce that today for those.
There are already familiar with it.
We see roughly in the wild, so Firefox 90 already does mark.
And read ECN.
We see roughly 50% of paths being ECN capable,
which is very promising and great for us.
And in the 75th percentile,
we see roughly 0.6% of packets being marked.
So that obviously congestion in itself is not great,
but packets being marked shows us that the boxes on our path
actually are able to manage the queue with ECN.
And then lastly,
we have been working a bunch on our memory management.
I showed you earlier.
There is a big chunk in our CPU flame grass cruise,
the of allocating memory just to receive something.
And what we're doing now is we have a single 64K buffer
for the entire Firefox process for all quick connections.
That's allocated once on the first connection,
and then use for receiving throughout the entire lifetime of the Firefox process.
So this entire chunk of CPU that you saw earlier in the profile is gone.
What is nice is Rust Sporechacker,
gives us a certain check of this of our memory reuse at compile time.
And yeah, it does show a significant CPU time reduction.
So what's next?
Well, we have one more quick talk.
I'll fill this here later today,
and then Lars is giving another talk at the Mozilla.
For those interested and want to learn more about quick.
In terms of what's next for Firefox,
we're rolling out PM2UD.
Then ECN is already in Firefox nightly,
and hopefully it's going to make it into better and release soon.
And then we're looking into the ECN graph
to get our current implementation,
which is an older version of the draft up to date.
The optimizations I talked about today,
I introduced an generic version both send and receive.
We have mostly focused on the receive path so far.
That's unsurprising, like the browser mostly downloads things.
But long term, we would also try to optimize the send path,
so introduce QSO, USO, which is the Windows equivalent send message X and so on,
and also ideally have a long live send buffer
to reduce those memory allocations.
Other things around condition controllers,
like for example, high-start for a cubic would be nice
and various other things.
So in case you want to get involved,
as I said, a lot of this is already in Firefox nightly,
so check out Firefox nightly, and you're already running those optimizations.
Some of them are already in better and release.
Check that out.
Then in addition, maybe I can convince some folks here
the quick implementation of Firefox is on GitHub.
It's entirely written in Rust.
And so if you want to help make a modern transport protocol faster,
and thus also make a browser faster,
come over on GitHub talk to us.
There's also a matrix room, and you can reach out to me.
That's all for my end.
Thank you very much.