My name is Alfredo Cardelliano, I work for Rantop.
And today I want to talk about Supernix, which is this new family on network adopters, actually
it's not really new, but then I have invented this new bazename for this family of adopters.
So my question is, are those adopters really are game changers?
So are they really introducing a lot of, at least for the performance point of view, are they
introducing a huge improvement, you respect to other adopters or not?
Of course, I will focus in the field of natural monitoring, traffic analysis, because we
do natural monitoring at end of it.
Let's recap what is anic, as Martin and the Supernix.
So anic is a standard adopter that you have in a standard server, so they usually provide
just data transmission, so Rx and Tx, and over the years, they introduced a few features
to improve the performance, which is like RSS, the received side scaling, features
or a lot of balance and leverage on multicores, and other features like filtering, etc.
And they are available most adopters today, especially on Intel, Melon Ops.
And then a new family of adopters have been introduced, which is the smartnix, so smartnix
are more advanced at the standardnix, they are usually specialized in some natural activities
like packet capture for traffic monitoring.
Most of the times they are based on FPGAs, and they provide nonsense features like packet
parsing, load balancing, like RSS, but more advanced, more packet filtering, more yields,
more flexible, and some programmability, for instance, if you have an appetite adapter, you
can use NTPL, for instance, to program the FPG to do something.
And then, for instance, they usually optimize data transfer, so they don't transfer single
packets on the bus, but they transfer chunks of data.
Then they introduce these new family of supernix.
So they are supposed to be more advanced than smartnix, and they have a lot of nice features
like other accelerators for specific tasks like networking, of course, encryption, the
feature, compression, storage, so they are directly connected to enemies, or to GPUs,
to accelerate AI.
They usually have also a number of CPUs, like on NVIDIA adapters, blue fields, there is another
CPU by the same for others like the AMD pensando, and a lot of other market stuff that
changes to the pending on the vendor.
So let's focus on the blue field three, which is probably the latest one.
This includes an AMCPU onboard 16 cores, DDR5, memory, up to 400GB connectivity, PCG5,
many other accelerators, all those mentioned previous slides, and this is programmable using
NSTK, which is Docker.
And these exposes network interfaces using a connect text interface, and also internal
is using the connect text 7.
You can use these adapter in queue modes, which is at a standard link.
So you just see on your host a connect text interface, and you use it as a standard network
interface, or you can use these adapter in DPU mode.
It means you can use DR, which is embedded into the adapter, to process the traffic in
the adapter.
There is a bit of which is side, which is up the accelerator with a bunch of accelerators
for networking, or for other stuff, like compression, encryption, etc.
And they expose the interface using connect text.
So NSTK, you can program the adapter running your software on the AMCPU, and process
the traffic.
So I say that this adapter is programmable using Docker.
Docker is a bunch of libraries that can be used to program the accelerators on the adapter,
and there is, for instance, one, so it's divided into libraries, one per accelerator.
For instance, if you want to accelerate networking, you use Docker flow.
And this is a supported bot on Bluefield, if you want to use the DPU, or some of the components
are also available on standard, and playing connect text adapters.
And this Docker flow, NSTK provides APIs to build pipelines for processing traffic.
So you can build pipes that do something, I will show you what.
And then you can change those pipes into a pipeline.
So this is an example, for instance, you can receive traffic that goes into a root pipe,
and then from the root pipe, you send the packets to other pipes.
For instance, you can add a pipe hay, which is matching this traffic.
If the packets match some entries in this pipe, then this traffic is sent to another pipe
if it's not matching an entries and to another pipe C.
And this traffic then can be sent to a physical interface or to an application using a recess.
So each pipe consists of a few entries that you can program into the pipe to match the traffic.
Then if the traffic, the packet matches an entry, you can run some actions like you can monitor,
so count, or meters that is rating the traffic, you can modify the packets,
or you can forward the packets to other pipes, or again, network interfaces, applications, etc.
So the question is, are superniqued better than it's Martinix, from our point of view,
so for network monitoring.
Okay, probably this is the odd version of this right anyway.
So let's focus on the network monitoring use case.
In network monitoring over the years, we've proved the performance of the applications
processing traffic with several techniques, like kernel bypass,
for instance, peer-freeing of the PDK implementing application-level DMA to accelerate the packet
capture, or by load balance in the traffic to multiple cores using a recess, for instance,
or using other offloads.
But the leak speed is going and growing, the CPU is not going as fast as the leak speed,
so we need to invent new techniques for accelerating the traffic analysis.
A typical activity that a network monitoring application is doing
is keeping the status of the network communications, usually in a flow table,
which is a data structure in memory where for each flow you keep the status of the flows
that can include counters, statistics, like packets and bytes, but also layer 7 information,
for instance, if you run in the packet inspection, you keep information like the HDP, URL,
or the CIP, YP, caller, or other information.
Now, so we were looking for a way to fuel the absolute rate, the application performance,
by accelerating this flow table.
So, not for that, for instance, or other mathematics, actually,
not for that, is the only one problem today that supports this,
and introduce the idea of the flow manager.
So, a way to keep track of all network communications in order, in the doctor.
Of course, you cannot afford everything, because if you're running DPI,
you need to still inspect the packet payload in software.
So, you need to accelerate the software with the combination of hardware and software.
So, you keep usually the status of the flows in order, but at the beginning of the
level of communication, you inspect all the single packets in in software and then you have to load the flow.
So, out at work. So, you capture the packet in the application, you start the fight
so you get the flow key, and if you have DPI, you run in DPI in the payload,
and when it's time to upload this flow, that can be at the first packet, if you don't have
DPI or something similar, or it can be after a few packets, one DPI has done.
You create a new entry and you are designed to the hardware.
Then you periodically check the statistics in the adapter to, if you want to, for instance,
to export packets and bytes for the flow. So, our question was, can we use bluefield,
and this Docker flow to do the same, to upload a set full packet processing to the adapter?
And it seems that the Docker flow CT, which is an extension or Docker flow, is a good fit for this,
because it's actually a connection track used for routing, for instance, in the adapter.
So, we decided to build this proof of concept to test Docker flow, the features and the performance,
to see if it's really working as expected. So, Docker flow CT, in essence, provides all the ingredients,
we need to provide the ability to create entries in the pipe that we've seen before,
using the five top all, and so the ability to add an entry, a mobile entry, update an entry,
get the statistics, when needed, and handle the flow aging. So, when a flow expires,
you can automatically remove it from the adapter. And this is available both when you run in the
bluefield, the queue, or when you run on using the bluefield as network adapter, or even
no standard connecting adapters. Then we faced with a few problems, let's say, so the documentation
is, let's say, missing some details, and the line documentation. So, it was not really the
simple to to implement such a proof of concept, especially also because the API changes from
mesh activation, and from adapter to adapter. So, the other remove features on different
adapter models. Even the examples that didn't have too much because they have the sign to be
really small examples, like capture one of the packets, we are getting of releasing the embarks,
for instance, to stuff like that. So, I expect to see something working to run just a proof of
concept, but I have to rewrite it from scratch. Even configuring adapters, it's sometimes
complicated, you follow the instructions documentation, but it doesn't work, so you have to figure
out how it works. But, when everything works, you have a lot of fun because it's really flexible,
and it's, so we came up with this tool, which is script tonight, available on GitHub.
It's basically in the link, okay, there is a link at the end of the presentation with the
source code if you want to check it. This is implementing the flow of load that is described
using local flow CT. It's also implementing a software for table, which is synchronized with
the prototype on the adapter for several reasons. For instance, of course, you want to keep track
of the flow of loader to get the statistics. Actually, there are also other reasons,
more on implementation side, but I don't want to bother you with this, but you also need a shadow
flow table because you want to keep the PIN information that you cannot store in the adapter.
This tool is also able to export periodically the flows, it can work also in line, so you can
configure the adapter to there is an option to, for what the traffic. Of course, it provides
analysis statistics because it is mainly a benchmark in tool for us because we wanted to
prove that this adapter is fast enough for what we have to do. So, this is the pipeline
implemented by this application. This small application is a one file application, so the example
that I was looking for, in essence, which is a good example if you want to start using the adapter
for doing this, so you can take it and extend it. So, in essence, this is the main pipe
created by this tool is this connection track pipe, which is keeping track of all the flows.
So, when the packet is coming, hates the anatory, this is just discarded if you are passive or
it is provided to the various port if it is in line. If there is no heat, this packet is sent
through an RSS pipe to the application using an RSSQ. So, the application can run DPI on this
and when as down with DPI adds an entry in the connection track to a flow this flow. So, you will not
see other packets for the same flow. This is the test bed that we use. So, we have anic using our
PF sign to send library traffic and then on the other side we have this blue field with tiptoe
night and we run this both in the DPI or on the side we are using blue field the nickbot as a
standard connectex adapter. So, the results, this adapter in our test, the maximum we have been
able to configure is 2 million entries in the connection track. So, it is able to keep up to 2 million
flows. Our traffic generator in this specific test bed was able to generate one leg
a bit with standard by packets or for for a gigabit with 60 by packets. So, we run the test
to measure the performance of the adapter and this adapter is able to forward all the traffic
full rate in the pipe in the other pipe. So, no bottleneck there. Of course, if you
follow this flow path. So, if the packet is sent to the application because at least the first
packet you have to process that the application then you have to load it. Then of course, the
R UCPU or the intensive PU becomes the bottleneck and we have seen that you are able to create
up to 2.5 million flows per second in the adapter. So, when you hit this creation rate
you start for, so you have more than 2.5 million flows, new flows per second. Of course,
you each the some bottleneck and you start dropping packets. Also, if you exceed 2 million flows
total, everything goes to the CPU and start dropping packets. So, not everything, but it is exceeding
2 million. So, last slide, smart tick versus supernick. If you if you compare smart tick with
supernick for this specific case, smart tick you can upload up to 140 million flows if you consider
not protect and there is a floppy ratio rate which is 1.5 million and flows per second using one
stream if you use multiple streams this increases to 3 million and you can run selected actions.
So, what is possible using the NTPL? On supernick, on blue field, you can upload up to 2 million
flows. The creation rate is more or less what you have on smart ticks and here we have,
let's say, full programmability. So, the main difference here is the capacity of the flow table
which is 2 million versus 140 million flows. Again, this is the limit high heat
configuring the adapter, probably there is a way to increase this limit, I don't know.
So, conclusion, the pros of supernicks is for sure hyper-mobility, hyper-formans and the ability to
flow the wall application in the CPU course. So, you can really run everything in the adapter without
using the server. The cost is the number of sessions which is lower with the respective smart tick
as I said, the configuration is not really simple, you have to really struggle on that.
You don't see if the pipeline is not enough for keeping all the sessions, you go through the
flow path that can be a bottleneck and the main issue for me is that this is really tied to the
PDK. So, you have to use the PDK to use this, you cannot use other frameworks.
The source code is available on GitHub. So, if you want to take a look, use it,
it's contributed and it will be a welcome. Thank you, sorry, thank you. Any question?
Hi, thank you for the presentation. So, on this slide that I talk about the experience,
trying to understand and learn from the examples, I feel the same way where you hand through the
same thing. So, could you elaborate a little bit more how you found out how to use the APIs,
how you ended up, what was the process to actually find out how to use and make the example work?
Yeah, so the example, in the example to have 90% of the work done, but the 10% is the
phase that you cannot find the documentation. So, at the end, we ended up because we have a
contact in video that told us how to do that fine-tuning and how to use that specific, you know.
Yeah, it's crazy. Unfortunately, it's like that. Thank you.
Yeah, one more question, then I think we have a lot of time.
Yeah, thanks for your presentation. So, did you manage to get your hands on a bluefield for
already, and you have some experience there? I'm fortunate enough. We have the three in our lab.
Thanks.
Something. The pipeline looks a lot like P4 match actions. Do you know if there is any plans
to support that via P4 instead of something appropriate, are you?
Yes, we'll be honest today, don't know.
Thank you very much for being here.