So, we get started again. The next talk is the budget talking about extending AFXDP for
first collucated packet transfer. So, thank you. So, I am going to talk about AFXDP and some
extensions that I did in my PhD work. This has been collaboration with many of the students
who are working with my professors. So, let me just get started first. So, how many of
you have used XDP? Okay, everyone. You know what it does right. First packet processing
implement your own stuff runs in the driver very first. For example, if IP is something
drop it happens right. But one thing that you might have experience is that there
are many constraints. You cannot do everything. If you say, okay, I am going to hold
off this packet and I will say like say in the packet later on you cannot do. There is
not such things right. So, one other alternative that we have is AFXDP. So, here what
we do is, so let me again ask, who have used AFXDP? Okay, fine. So, here what you do is instead
of do the processing in the XDP layer, you just run a XDP program which will say, okay, instead
of processing your, I will say in the packet to some other application that is running
in the user space. And what it can do is that it can deliver the packet in zero copy
manner. So, basically the driver can directly, the nick can directly delay in the packet
into user space memory and it is really awesome. You can get like some good performance
out of it. Now, what like we would like to do is that we can, we would like to have multiple
such applications, zeros, zero copy applications, running in the same machine. Because
of efficiency reason or you want to come up with a strategy where you, you are breaking
application into smaller parts and you are then like joining them together to give some
service right. This is the goal that we want to do. But the, as, as the problem is that
being and loss of it, like if you want to transfer the packet, the packet goes out through the
nick. You, you cannot say, okay, I want to send packet from application one to application
to, it will go out, right. How do you solve that? So, what are the existing solution? You
will say, okay, SRV, I am going to use the hardware support that is there in the machine,
there in the nick and then I am going to use that to redirect the packet from one application
to other application. But as it turns out, the support for AFXDP on such drivers is very
scarce. For example, virtual function support is only available for Melonox MLX5 and
sub-function support is only for ISN MLX5, right. And also, if you try to implement this in SRV,
since there will be packet copies from one, one application to the other applications.
The throughput will decrease because of, of such copies that will happen. Because ideally,
if you have multiple such application and they are running in the same machine, if few
somehow can share memory, between these applications, you can just pass point of
one application to other application, right. So, other thing that you could have
take a thing of like a possible solution is like, there are there are works in the PDK,
like open adsim, which tries to solve this problem. By having a user space application,
a manager kind of stuff, which orchestrates this packet transfer from one application to other
but the problem is that now you, there is a very tight coupling between such libraries as
well as the applications that people are going to make, right. And also, you have to change all
this AFXDP, whatever applications you are making, you have to change them with new APIs, right.
So, what we have done, what our work is that we have implemented packet
reduction from one AFXDP application to other AFXDP application inside the kernel itself.
Therefore, if we want, for example, if the app one wants to send packet to other app,
the only thing that you do is just send, normally pay nothing else, internally the packet will
be sent to the other application and we named it as flash, like fast linked AFXDP sockets for
high performance training. Now, like, let me just give a brief idea of what this AFXDP thing is
and how things are working, right. So, what is happening here is that the application has some
user space memory, that is, that has been allocated and this allocated, the memory is being shared
with the driver as well as AFXDP socket. So, this is where the DMA and other things are happening.
And now to pass the control of where to DMA or you, the driver wants to say, hey, I have DMA,
the packet here, please look there, they are like multiple drinks, single participation, single
consumer drinks. For example, if you look at the purple lines, these are used for receiving
packets and the orange lines that are present are used for like transmitting packets.
And if you look at it, the transfer of packets is now split between user space and the kernel space
where receiving and transmission is actually happening in user space and the processing or sending
the packets out to the neck is being handled by the driver itself, right. And if we again go
much deeper and look at how the redraction happens. So, first thing that it does, the user space
application does is that it tells the drivers that, hey, these are the locations where you can
DMA the packets by putting it in the filtering. And now, the driver says to the neck,
you can do that. And whenever DMA happens, it goes and then the exhibit program runs, it says,
okay, it looks up which AFXDP socket, I should send a packet now. If it sounds that and it
puts the descriptor of that particular location into the RX ring, which the AFXDP socket uses to
get to know, okay, here the packet has been received, right. Similarly, for transmission also,
the PXing and completion rings are used, right. Interestingly, there are two application with
the two threads that is working here, right, or the driver thread and the AFXDP user space thread.
The driver thread is actually in truck driven, whenever packet comes or it tries to send the
packet out, out through the neck, it is scheduled whereas, AFXDP socket is under your control,
how do you, how do you want to run it. Now, there is a like, other kind of mode that,
actually you can use, where the AFXDP itself can initiate the sending and reception of the
packets by using send to and receive from system call. So, it kind of behaves like a busy polling
that will happen, right. So, now, the idea that we want to implement is this, right. There is
two application, whenever application one tries to send packet to the next application,
we just want to put descriptor from the TX ring from one and put it in the RX ring of other
application, but there are many challenges. First of it, first of it is the rings, right.
The rings of AFXDP are single-produced single-consumer. So, if you just try to put the packet there,
it will violate the semantic, right. So, first of all, what we try to do is, we try to
use a single-produced single-consumer ring only by having a single case of target trade to process
all of them to serialize them. We found out that the speed that we are getting where, where,
very low, when we increase the number of sockets. So, instead what we went ahead was to implement
the multiple, we updated the rings to have multi-producer, multi-consumer support.
While doing so, we ensure that from user space, it remains single-produced single-consumer. So,
what we did, if we look at the code here, earlier there was producer consumer and flags.
We have added two more values here, producer head and consumer head. Now, these two are used
to serialize multiple-producer or multiple consumers. And while doing so, we internally used
components to have operations. And, and we have abstracted all of this implementation with some
simple APIs, instead of AFXDP, some subsystem itself. So, for anchoring, you give the descriptor,
it will thank you for taking you also, it will do that. So, now, if you look, if you have this
system, right, now you, you can have like multiple-producer multiple consumer. Now, what you can do
is that, if you look at the figure, when you try to transmit, you can check where, where to send
next, you, you can directly put that buffer into the RX thing of the next socket.
But, by doing so, what is going to happen is that application 2 will have one more buffer,
and application 1, one is going to have one less buffer. So, there is like some mis, like
misbalance of buffers between the applications. Now, to do that, what we do is that, we pull
one buffer from the filtering and put it into completion ring. So, what is going to happen?
From the perspective of application 1, it will feel as if I have sent, I have successfully sent
one buffer. And from perspective of application 2, it will feel as if I have received a new
buffer, from the, from the Nick, whereas, it just internally, in the same machine, it itself,
the buffer has went from application 1 to application 2. And also, if this is, this is when you
have a, like single human region, right, where you can just, like pass the pointer from one,
application to other application. If you have two such applications with different human regions,
what you will, you will do is, it will just do mammocopy internally, right, instead of doing just
do passing. So, that support is also implemented in our system.
Now, while doing so, we realize that, now, we are using, components of operations.
And it, it was happening very frequently. The performance was like decreasing, like, from the
baseline, the base implementation, the performance was well-low. So, what we try to do is that,
we, we hold off whatever, the packets we are getting, we do not thank you. We hold off until
for bad size, some bad size, which is, I think, we have used NaPipose bad budget size itself.
And then, in a single, in a single turn, we push all these buffers into the, into the rings.
And for that, also, we have introduced a new APA for that. Now, the problem is that,
okay, I have 30, 30 buffers that I want to push into the ring. And you do not know, right,
what will be the sizes of the ring in the future. But because we have updated these rings
from single-producer, single-consumer to support multiple producers. For example, I have looked,
okay, in RX ring, I have two buffers. If you, like, take that knowledge and try to,
and keep two buffers into RX ring, it might happen that some other producer has already
filled this RX ring. So, that problem, like, can occur. For doing that, what we have done
is that, we have ordered the ring updates in a manner, that it will ensure that the PX
packets that we are sending will always be guaranteed. It's like, the thing that we have done
is already like, there is a TX batching option in FXDB. For implementing that, they have already
like faced some kind of problem like this, but only for a single application.
Now, since we have two such applications, it was way difficult to do that. We have taken the
idea, make sure that the order that we are sending the packets, ensure that there is no packet
drops. And while doing so, we have found out that we might have like pulled some buffers from
filtering, and we cannot take in, and it might be like laying around in our hand, because,
for example, if we pull three buffers, and only there are two, what to do with that extra
buffer, feeling buffer that we have got. So, what we do is that, we hold off for the next
iteration and then reuse that. And by, and we have, we have some checks and we have done that,
and we have then, we have tested that, it ensures that no packet drops and other such things happen.
So, there is no corruption or such things. The third thing that we face is that, so there are
many applications and libraries like CNDP, which tries to employ some hybrid pulling mechanism
to like increase the efficiency of FXDB application. So, ideally, what it does that initially,
it starts the application in a, in a interrupt mode using pole system call. So, your application
will not take up CPU cycles when, when no packets are coming. The moment packets comes,
like comes coming, it transitions into busy pole mode, starts processing packets. And again,
if there is no packets, it rewards back to the interrupt mode. But now, the problem is that,
if you look at the figure, the packets are only coming in application 1 and application 3,
the application 2 is doing nothing. There is no interrupts coming in application 2. And hence,
the kernel will think that, okay, there is no processing happening, I will not schedule it.
So, by default, if we implement whatever we want to do, it will, it is going to be messed up.
The brings will get empty or some deadlock situation might happen. So, what we have to do is that,
we have to implement, we have to implement a hybrid pulling mechanism, which also schedules the
application whenever packets are redirecting from redirected from one application to other applications.
Also, like, whenever there are applications which are sending packets from one NF to other NF,
and the, if the second NF is congested and is not able to process packets, the other
application is the first application. Continuously tries to send, like buffer again and again,
waste wasting CPU cycles. So, also what we implemented was, senders can, if they see that,
okay, my ring is full, I cannot send more packets than that. They can just do a pole system call
with pole out, which will block the application. And when the, in the, in the receiver application,
when there is, this congestion goes away, it can just call a receive from system call to just,
inside, internally inside the kernel, just wake up the other applications that I am ready,
I have now space, you can now try sending application. So, we did implement this also.
Other implementation is that, I think it is, it is quite easy to do this.
If you have multiple applications running across different containers, these are different
namespaces, you cannot directly access the Q. What you can, you can do is that you can have some
application running in the host namespace, which handles, passing the control of the,
Nick to the application that is running itself. So, what we did was we implemented a flash monitor,
which communicates with the applications running inside, inside containers using some,
using some UDS rockets. And by doing that, you can initiate, create as well as share user
memory between applications, across between applications that are running in different containers
very easily. And it ensures that previous tasks and all accesses are like taken care of
by these privileged applications. And also, you might ask that, if you have two different applications,
which are, which is sharing memory, like application one can corrupt memory of other application,
what, what we will do about that. So, what we did was we implemented a rust library. So,
if the applications are using our rust library, it ensures that, if application one has sent
a packet to other applications, then the control of that packet is gone, you cannot do anything
about it. This is the fourth thing that we did. But the point is that, okay, you have this
system of passing packets from one fxdb socket to other fxdb socket. But how do you specify,
like which socket next, right? So, because these rings and all these terms that I am talking
about are structures inside the kernel, right? And which is associated with some fxdb socket,
which has this point like, struct xdb socket. And these are like process local, right? You
have fds, fds points to this xdb socket, which then points to this rings, right? So, if, for
example, fxdb socket 1, once to send fxdb, send packet to fdb socket 2, how do you, like get access
of those rings, right? So, to understand first, like, how does xdb does that? We looked at how
xdb does that. So, what xdb does is, is they have a map, which has some kind of index,
these index can be like, user can specify what the index is, by default, it is QID and interface
that you are using. And it actually stores this struct xdb socket. So, whenever a packet is coming
to the device and to the Q, it looks up into the map, gets this ok pointer and then from there,
it gets access of this rings, where it and Q's packets, right? So, now, we want to do the same thing,
whenever socket 1, once to send packet to socket 2, we want to get access. So, first we thought
ok, maybe we can have some xdb program running in, in the egress path, but before that we thought
ok, let us go an implement a straight forward solution. So, what we did was we implemented a
c surface interface. So, whenever a, a, fxdb socket is created, in this interface c's kernel
flash, a index gets created. So, for example, socket 1, one is getting created, for two, two
two is getting created. And then now, two set of redirection dosing that, I want to send packet
from 1 to 2, you can just do a equal to 2 c's kernel flash 1 next. So, internally whenever
now, these packets are getting transmitted, the map will be looked up, is it ok? Now,
it is the packets need to go to like fxdb socket 2, it will get this structure that is stored
in the kernel, get the access of the rings and then do this magic that is happening. So,
that is how we are doing it. So, now, the point is that, all things that we have done like
how much change do we need to make, it is like, is it drastic, does it change, what is the default
path of fxdb? That is no, we are not changing anything, we are only augmenting stuff with
whatever is there in the fxdb system, it is around 700 lines of code that we have added,
most of it was for the c surface interface. And in the driver also it requires only minimal
patching, because now, when driver checks if there are transmission, if there are packets
ready for transmission, now you need to say to the driver, hey, there are packets ready, but you
do not need to transmit it, you just do your stuff. So, the new lines that you have to add it
are those 2 or 2 only. So, if there is no transmission, it can just return, I have done some work,
the nappy should work as usual, but do not do anything, do not DM a packet out. So, these
code is needed for supporting different drivers, we have tested our implementation in
four different drivers, we have tested and it was working fine.
And yeah, the performance gains, if we look at it, the axis says the number of applications
that we can change together in 0 copy manner, and the axis says the throughput as well as the
latency. As you can see, if we use the standard as a way, the performance decreases, because
the copies and different stuff. Whereas, if we use our flash technique, then it stays almost
similar. And if we compare it with other, like use a space training technique, like open
medium, it is almost similar, where the throughput is almost similar, whereas the latency is
really low compared to if you think about the latency that we are getting out of like open
Ethereum solution. And yeah, in summary, I have shown you that AFXDP has a problem, when you
try to have multiple AFXDP applications, colocate on the same machine, it cannot send packet from
one socket to other socket. So, for solving the problem, we have extended the subsystem, so that
redraction can happen. For that, we have updated the rings to support multi-producer multi-consumer,
implemented batching buffering, smart polling for efficient packet transmission, and
C surface interface to set up these redraction rules. For now, talking about what can we have
this technology, what we can do with it. So, first thing is that we have now this C surface
interface, I think if you think about programmatically, it is quite a bit strict, you can say
that you can say in packet from one to another. If you want to say that, you can have
application one can change from one to other application, as well as other application. You can do that,
but the application now internally has to say to the kernel that I want to say in this packet
to something else. So, what you would like is that the application will change the packet as usual,
and whenever it comes to the kernel, the kernel can automatically redirect packet wherever you want.
So, the answer will be obviously, a EBPF program, right. If there is a EBPF program at the
address point, then you can decide this rule, right. For this particular packet that is coming
instead of sending the packet out through Nick, I want to send this packet to other application,
or you can say that, okay, I want to send to application C. So, that logic you can implement.
There has been interesting talk about implementing HPE grace, there is a talk, there was a talk in
LPC, I think this fits really well with this idea where they were trying to see what observability
and other things that they can do it, right. Now, we can have this HPE grace hook point where
this decision of sending the packet to the next NF can also be incorporated, right. So,
I think it is really great. So, also we have tested this changing solution with different applications
like load balancers, firewalls and other stuff, right. I think generalizing it to other other
such applications can also be interesting, right. For example, you can have a dedos mitigation
application. So, nowadays, IP tables are used. So, maybe I will replace IP tables with some
0 copy applications that is actually doing the routing and then I will have this dedos protection.
Maybe have some key value store or web server running with this entire stack and I can redirect
packets whenever it is going through this stack, through this bypass stack, right.
So, that can be very interesting. So, I would really like to listen from you like what do you think
about this implementation. We are also currently developing a 0 copy support with TCP,
MTC user space, MTCP. So, that we can also like deploy TCP applications over this whatever we
are working on. And I think since we have updated the rings of AFXDP to support multiple
production and multiple consumer. Now, like there can be different scenarios that people can
think of. For example, whenever a packet has come to a very Q1, you can in the XGB program.
Now, it can only deliver packet to whatever AFXDP socket it is bounded to, right. If it is
if it is bounded to Q1, it can go there only. Now, because the rings are only support single
predecessor single consumer. So, now, you have multiple producers. So, you can say, okay,
from Q1, I want to send to Q2, those kind of things also open up because of the implementation.
So, I also, I am really excited about what can we do about like using these single producers,
single multi consumer and multi-producer single consumer rings in AFXDP. And also, for I talk
about flash monitor little bit, I am also we are also working on rewriting the flash
monitor as well. It was written in C, we are rewriting completely in in rust right now.
Thank you. Any questions?
Okay. Great work. I have two questions. The first one, how do you deal with the red
direction of the, I mean the AFXDP rings that can be of different size, right. And then you could
potentially one process redirecting to the other one could block, right. Like, have you thought about
sorry can we be doing? I mean like the AFXDP rings that could be of different size,
where different sockets, right. And if you redirect from multiple other sockets to a certain socket,
it could like fully block the receiving socket. So, like have you thought about?
So, what we do is that internally, when you create redirections, for example,
A1s to redirect packets to B. We, in the in the kernel, we create some temporary buffers,
which is size of socket 1. And when you try to redirect, it will, maybe you are trying to
redirect the whole thing. It will, it is going to try to and keep all those packets. If it is not
possible, then it will like ensure that other packets stays there. It will not block until this is done.
And then my other question was, have you measured the overhead, like for like a regular AFXDP
application with new multi-producer multi-consumer? Like how big is the overhead? And you mentioned
in your talk that you gained the performance because of batching. Yeah. So, yeah, can you give
some more details on that? Yeah, yeah. Actually, we did test it with with our
multi-producer multi-consumer implementation. We did, we have done some experiments. The work actually
has been published in SCC. In the paper, we have the results comparing it. And it was not much.
The latency, I think the exact numbers I don't remember, but it is not, it is not
drastic. It is almost similar. And it was because of the batching that we were able to
like have the same numbers as normal implementations. Yeah. There is question.
So, is this already like, is there a patch set for this? Are you going to upstream this to
kernel or is it already in? Because I know, yeah, it is open source. So, we have two components
with that web developed. One is the kernel changes that we have done. The other thing that we have
done is that we have a like whole user space library where people can go build their application.
That is a big thing that that I didn't discuss at all in this in this talk. Both of them are open
source. We are looking, we are trying, we are in the in a, I will say timeline with talking with the
people who are working in this field to see what they think about this, do they think that
this is very important and should go into the kernel. So, I think coming up with applications
that can benefit from this technology is also very interesting thing that I think we can
thought think of. Amazing. Thank you. Yeah. Thank you.
Have you considered about like just sending our FC patch set to the kernel milling days just to get some
feedback as well? Yeah, I have been thinking about that. I wondered how you handle the performance
when you go cross CPU because there is a lot of you lose a lot of performance when you do
a cross CPUs. Yeah, definitely. I think because of batching of the performance,
there is if you look at a number right, there is a decrease little bit decrease when you go from
like one to two. It's not much, but there is a decrease. So, I think because of the way that we
are doing the batching, the like the difference was not that much. I didn't discuss about batching
a lot here, but yeah, because of the, I think because of the batching the difference was
like we were able to catch up with the, when we were doing like components of a
preparation across CPUs CPUs. Yeah. Any more questions?
What is the platform name that you didn't discuss so far because I'm actually interested?
With the name of the. No, not the flash one, but the other one. Sorry, I didn't
have to develop the applications on top of it. That was also open source.
Yeah. So, the user space thing is named flash only and the kernel space, I think we named it flash
Linux kernel because we wanted to get like pushed into Linux kernel. So, we didn't name anything
interesting for that. Thank you. Right. Thank you. Yeah.