OK, good morning, everybody.
Well, I'm one of your tools.
I'm Federico Ezzi, working for Google Cloud over there.
I do mostly application modernization and performance
optimization.
And today I'm here to talk about VPP together with...
So hi, everyone.
I'm Mohammed Hawari.
I'm a software engineering tech lead at Cisco Systems.
And I'm working on the VPP project.
I'm a computer and this project.
And I'll be talking about VPP mainly.
OK, so before going to all the greeting details
about what VPP, what this work all about.
And around the last year, Spring, I set me
for myself, a challenge to test the maximum performance
throughput achievable with in a single zone deployment
on Google Cloud.
I wanted to validate a back-to-end
the latest machines available.
I come from the Telco World, I have about 10 years experience
in fast-utter path, specific DPPK.
And so kind of comfort zone, let's leverage that Telco application
like VPP.
So let's see what are the numbers we can achieve
in the last three months during the question.
Can Google Cloud be a robust platform capable of handling
typical VNF?
Use cases at high-pocket rate.
So for this application, we're going to talk
about software-based fast data path.
So let's mention quickly the technology can enablers.
Mainly there are enablers in Linux, which consists of the ability
to implement user-space drivers.
So UIO, VFI, all these kind of things.
And minor improvements in the Linux kernel
were done over the years for to get better real-time properties,
like the tickless kernels, huge pages, et cetera.
So this is one side of the enablers.
The other side is actually DPDK.
The pull-mode drivers in DPDK
that actually expose device drivers in user space
and allow you to implement networking application user space
with a decent amount of performance.
So now let's talk about the Vector Packet processor.
So BPP, it's an open source project.
Sorry, let's see.
But it has become open source for eight years now.
If you fast open source, you just
based at working in data plane.
It relies on kernel bypass thanks to DPDK among others.
It's a feature-rich.
It's extensible to plug-in.
So you can implement your own network programming logic
in it.
And it's especially optimized for high throughput.
So being throughput, meaning you can process lots of packets
per second.
And it will, so I'll explain how we do that.
Actually, what are the details allowing us
to achieve a very high throughput?
And it's available on Linux.
And since one year, on 3BSD, I'm looking for Tom.
Oh, he left.
He did the VST port.
OK, so what's the approach of VPP
for to get high performance?
So VPP stands for Vector Packet processor.
So it processes packets in batch instead
of having a run-to-completion model.
And by processing packets in batch in vectors,
it allows you to optimize the instruction cache,
because you are doing an elementary relatively
elementary operation on a bunch of packets, which
means that you don't have instruction cache misses
in your CPU.
It relies on kernel bypass in your copy, as I mentioned.
So packets are gma directly
into the user space memory.
And it relies on explicit prefetches
and what we call dual or quad loops.
I'll focus on it a bit later.
OK, so VPP, very similar graph
to a previous presentation.
It has the same ideas.
So this is the packet processing graph.
So you can see you have a vector of four packets.
And it goes through the processing graph.
Like here you have IPv4 and IPv6 packets
so the vector is split between IPv4 lookup, IPv6 lookup.
Then forwarding decision is taken.
Some of the first vectors, some of the packets
belong to first vectors, first vector
are routed to the first interface,
the other to the second interface.
Vectors are reconstituted and then sent over the wire.
And this architecture allows you to have each node
process multiple packets at a time.
So let's instruction cache misses.
Now let's talk about one of the other ideas, explicit pretaches
and quad loop.
So quad loop, how does it work?
So here you have a prototype of the typical VPP node.
So how do we code in VPP?
We process first four packets at a time.
And while we process four packets,
we are sending a pre-fetched instruction to the CPU
to fetch the four next packets.
So the CPU is going to send a request to the RAM.
Get the four next packets to the cache.
And in the meantime, we are going to process the four packets
right now, the four current packets.
We're going to process them four at a time.
And we are going to interleave the instruction,
like instruction one for packet one, instruction one for packet
two, instruction one for packet three,
instruction one for packet four.
Then instruction two for all these four packets,
which allows you to free-field the pipelines of the CPU.
And by doing so, you will minimize the number of cache misses
and you get actually quite good performance.
And finally, VPP is built with performance in mind.
So unlike lots of open source projects,
and actually in lots of proprietary projects,
performance is tested all during the whole life cycle
of each commit.
So each commit in VPP, we have a full test bed
that tests the performance on the variety of scenarios
and hardware.
And you can track the regression, the performance
at the scale of the packet cycle per node.
And each commit that might introduce the regression
is tracked and we make sure that no regression happens.
That's you.
For the recall.
Thank you, moment.
I think it's quite impressive the level of tuning
and optimization and coding that has been done on VPP,
a kind of impressive.
So looking at the testing topologies,
I have two examples for you today.
The first one, it's fairly simple.
Traffic generator and network receiver are just
some test PMD machines.
Generate packets on one end, deliver on the other,
uniflow, excuse me.
Unidirectional type of load layer to traffic.
So nothing particularly weird.
VPP in between routes packet from one leg or from one network
to another.
The Linux systems underneath are deeply
tuned for deterministic performance.
So doing all sorts of possible isolations,
thickless kernel, isolating the user land and worker threats.
And packet size, 64 bytes.
And so I choose to bring you here two specific tests.
The first one is with a single PMD thread inside VPP.
And I was able to achieve about 80 million packets per second.
20, excuse me, 12 gigabit of traffic, roughly.
But the most astonishing details is the drop rate,
only four parts per billion of a percent, which
is, I think, quite remarkable.
Coming from Telco background, these stuffets.
Those numbers are quite crazy, given that this
is on a public cloud shared infrastructure.
Scaling up the number, of course,
and using a much bigger machine with way more resources,
I was able to break the 100 million packet per second
psychological barrier, specifically the 108,
drop rate, mash higher, but still
kind of reasonably low for the type of environment.
And the fun fact, 108 million packet per second
are roughly on a standard IMX, roughly 300 gigabits of traffic.
Second network topology, everything remains the same.
It says the traffic generator is walking out the test PMD
for packet gen depd key.
Package and sensor received the traffic.
In this case, it's the layer free traffic
to the less shades being used, sending
the receiving on both interfaces.
Variable destination ports, variable packet size,
and effectively tens of millions of possible flow
combinations.
In this, again, kind of challenging scenario,
I managed to achieve about 20 million packets per second,
only with 3 PMD threads, 25 gigabits of traffic.
I think I reached the limit of the traffic generator
because although VPP was not dropping packets and was,
but details, this is kind of another situation,
kind of challenging situation where we see the dep optimizations
done at VPP put a good use.
So what are the technologies in Eblar?
First, the first and foremost, huge shout out
to the VPP community and to the development team,
because they did an incredible job actually optimizing
pretty much everything out of it.
On Google Cloudside, forming factors
third generation machine that rely on what we call
titanium architecture, I will go into details in the next slide.
Video methodology, so compute optimised machines,
some Google Cloud expose what is underneath,
directly to the guest.
So memory allocation and CPU mapping
happens on the correct number node as per the hardware.
Jupiter is our network fabric within all our Google Cloud
deployments.
This is a 10 something years, this point effort.
And the latest iteration, we are able to handle
an aggregated capacity of six petabits of traffic,
connects tens of thousands of servers and sub-100
millisecond latency, excuse me, micro-second latency.
Last but not least, well, this is a tight capital deployment,
not deployed on the same record otherwise,
wouldn't make sense, but deployed on one next to another
racks in our zones.
Another interesting bit, I didn't mention on the slides.
Everything that you're seeing here, it's all GA features.
All of this is accessible and usable by end users.
There are no alpha-based APIs or allow list whatsoever.
All of this is standard regular features.
So very quickly on Google titanium, even I mentioned it a few times.
In the industry, those devices are generally called
as a DPUs or data processing unit.
We call it as an IPU in-for-processing unit,
back in the days, smart nicks.
Well, effectively, we don't just do the protocol
for wording, GRO, or check sums.
The entire virtual link runs in hardware and is PCIPAS
through inside the guest.
This allows to achieve a line rate effectively results.
It's a collaboration together with Intel.
We call it titanium.
The product skew on their side is 2000.
And yeah, by the way, didn't mention the NIC.
The virtual NIC is called GVNIC, Google Virtual NIC.
We move the way from Virtio, mostly for performance reasons.
Was built from the ground up with strong focus on low latency.
And the drivers are main land in all the major products
for Linux, so previously, as well as DPDK.
And so thanks for the recall.
So let's give a bit of perspective what we can do with BPP,
especially with this work that proves that we can run
BPP, actually, that proves that GTC, there are cloud providers
that can run BPP with optimal performance.
Not all of them can do that, actually.
You should test.
So VPP, what can we do with it?
First is plug-in oriented.
So it's easy to customize packet processing
for customized networking behavior.
So this is really what we call, what we can call a network
program ability.
And VPP should be seen as a programming framework,
not as a software router, that allows you to implement
drone cloud network features.
You have the variety of use cases.
And this is already deployed.
You can use VPP as a tunnel gateway, for example,
to terminate IP sector models, connect multiple clouds
together thanks to IP sector or other tunnel protocols.
You can also use it for high performance and point applications,
not only routers, but also endpoints.
Because there is an embedded TCP, high performance,
TCP, UDP stack, and VPP.
So actually, there was an NVOI VPP integration
that is an open source, and part of the Android project.
And finally, you can see it as a complement
in the next kernel to do good fast container networking,
to implement a CNI, for example, for Kubernetes.
Typically, we have an example of CNI called the Calico VPP
that actually use VPP to improve the performance
of Kubernetes networking.
Before closing, I just want to mention
that if you would like to read more about this effort,
everything has been published over a medium
at the Google Cloud medium community.
Excuse me.
It goes, of course, much more details, actually,
even removes VPP just to test the overall performance
and see if is a bottleneck, spoiler is not.
And well, I'm also open by the end of this month
to publish an updated version with latest software
versioning as well as machine types.
And thank you so much.
Thank you.
Any question?
Thank you.
So in the beginning of your talk, you showed
this very low packet loss numbers.
Do you then have an idea where this packet loss
still comes from?
Because then at some point, if it becomes so low,
it could come from anywhere, also from hardware,
or should I come to have an idea?
Yeah.
So actually, I spent quite a bit of time
understanding packet loss is in VPP.
So there's a slide that we skipped that explains how you
optimize the Linux configuration so that you are basically
packet loss has come from the fact that the CPU does
something else at some time.
And then VPP lose some packets, misses some packets.
And this can happen from a variety of software reasons.
Like the Linux kernel is this schedule is the VPP.
The VPP process.
But even if you optimize absolutely everything,
there are inherent hardware reasons that forces the CPU
to do something else during a few microseconds,
called system management interrupts if you heard of those.
And basically, because of that, you can never have zero
property.
Yes, time.
Great presentation.
Thanks a lot.
You mentioned you can implement endpoints using the plug-ins
on top of the built-in network stack.
Do you have example of successful endpoint implementation
in production?
Open source or not, either.
OK.
So in open source, we have NVVP.
NVP can be seen as an endpoint.
Even if at layer 7, it sees as a forwarding,
I mean, still an endpoint, right?
And then internally, we have at least two projects, three.
Yeah, two and a half.
One of them is being finished.
That uses VVP as a termination for TCP or DTRS, actually.
We'll actually do DTRS, TLS, Man in the middle, VVP.
Thanks for the talk.
In the last slide, you'll see VPP as kind of an extension
to VVPF.
Isn't the kernel bypass something as an alternative
to VVPF or TCP, for example?
I don't want to let me think about my answer.
In some cases.
Thanks for the talk.
So did you measure the same performance on a bare metal server
when you compared with the GCP cloud?
Did you see any difference with respect to the performance of VVP?
Back in the days, when I used to do another job,
let's say, as in the TTRQ industry.
Yeah, I saw sometimes comparable results.
Back then, I also had the opportunity to be in charge of the entire chain.
So I could see and understand all of the details of the platform.
So performing specific optimizations or troubleshooting
to reduce loss was obtainable while on Google Cloud,
pretty much, well, you follow the blueprint published on medium
and it just works with those results.
So of course, you are not being given the control of all
the hardware and everything underneath.
So performance by, you don't see any difference with respect
to bare metal server running the VVP.
I think now, this is on bare metal.
You can achieve much, much higher performance than this.
Again, this is still public cloud, this is the shared infrastructure.
The network devices underneath are, if I'm not mistaken, 200 gigabytes.
But only in the latest generation C4, we give customer access
to the 200 gigabytes throughput.
And it's just one network interface at 200 gigabytes.
On bare metal, you can do much more nowadays.
OK, thank you.
Expeaker, I think we're on the other time.