Thank you.
Nothing,
just everyone's going just for this time.
So you don't have to switch during the talk?
Thank you.
Have any interference?
Hold it down.
It's on.
This is always on.
And don't shut it up.
Put it somewhere else while you're not speaking.
I will do this.
And then there are questions.
I'll take this microphone and walk around.
Okay.
Perfect.
Thank you.
Okay.
I'll get it.
Okay.
All right.
Thank you.
I appreciate you coming through our discussion about dynamic resource allocation for networking.
My name is Doug Smith.
I work on OpenShift's networking and a member of Network Pulling Working Group.
And this is Miguel, who I'm joined with.
We also collaborate on a number of things in this area.
Okay.
So I'll take it from here.
So the first thing, let's walk through the agenda.
We'll try to define the problem.
We're trying to solve and which problem we're facing right now.
We will try to paint the landscape of multi networking in the Kubernetes ecosystem.
And we will afterwards provide the context for DRA.
Explain what dynamic resource allocation is.
I mean, I guess you guess that has to do allocating resources, but we will go throughout a little bit deeper.
And kind of contextualize what does that mean in the context of networking in Kubernetes.
Or especially multi networking in Kubernetes.
We will then finalize where the tour of the resources of a community like what do we have at this point in time.
And we will then finalize where the call to action.
So DQR and problem and the alternative to that.
I'm not sure.
I assume most of you are used to Kubernetes and are using it.
And are quite aware of the Kubernetes networking model.
So pretty much what you get is like all your workloads and the cluster will get one single networking interface that is totally managed.
It connects every workload in the system.
You don't want that.
Put network policy on top to kind of drop traffic.
And that is what you have today.
But question is what you need what if you need more than one network interface.
So we're a connection to more networks.
Let's say you're implementing a firewall or something.
Well, you won't do that with one single interface.
What is Kubernetes has made it.
Let's say this smart news, but it totally avoided answering this problem.
And left it out of there like use cases.
Thing is the community.
Join forces and like there's an initiative called the network plumbing working group.
They came up with something called multi-CNI, which kind of tricks the system and provides you an API that you can request more attachments.
To different networks on your workloads.
So it pretty much is an out-of-three solution.
This is not integrating Kubernetes at all.
Kubernetes does not know anything about this.
And while that is extremely flexible, it looks extremely bad.
So it's the two things in the left.
If you look at it, it's like a network attachment definition in the bottom left.
It's pretty much like a JSON encode string where you put whatever it is that she wants.
That will make sense to your plugin for it to do what you want.
And in the bottom, we have how you request an attachment to that.
You just put in the annotations.
Again, a JSON encode string saying which runtime parameters you want to have it.
So it's not very comfortable.
It looks bad, but does the deed.
Now, the something came up like recently a couple of years ago, probably,
called the Kubernetes native multi networking initiative and they propose something different.
So you would have a generic CRD called pod network in the bottom right.
Where you would define your provider.
You would pass a couple of parameters and all that.
And then in the upper right, you would have like,
you would get to refer to these things in the pod specification.
So you would have to mutate the pod specification as it is today.
You would need to change the pod specification.
If you have like a network array where you could put like which attachments you want to have.
Now,
a signal network, a Kubernetes signal network looked at this and said pretty much.
Nope, you're not going to touch that.
You're not going to change the pod specification.
Find something else.
And this was like there were pretty adamant about this.
This was like a non-negotiable kind of thing.
And it kind of left us, well, left the community in a position where we do not know how to proceed.
Like doing this implemented out of three, we already have that's not what we want.
How do we put this in?
So the question essentially is,
how do we come up with the way a Kubernetes native way to request like a generic resource that happens to be an attachment to a network?
That's the only thing we need to sort out, I guess.
And Doug, we'll take it over from now.
Awesome. Thank you, Miguel.
So now that Miguel has laid the groundwork,
you might be wondering, well,
what the heck is DRA dynamic resource allocation?
Let's imagine this scenario.
You've got a workload and it's going to use a GPU as you may have heard
artificial intelligence and machine learning is fairly popular these days.
Dynamic resource allocation is really designed for this specific case where you may need a GPU resource.
So if you've got a workload, it wants to use a GPU.
And you've got a cluster and there are so many GPUs available.
Well, what DRA allows you to do is to inform the scheduler that you have a number of those available.
So if you've got this workload, which node should it arrive on?
Well, as you can see here on the middle node, there are no GPUs available.
So it would leave you with one or two.
So DRA allows you a way to express that you want to,
you want to express your intention to get some of these hardware resources.
So what about in networking?
Well, you can say the same thing for a virtualized Nick, for example.
So SROV, single root Iovertualization, which is a hardware technology to have high performance networking.
You have exhaustible resources there as well.
So the same thing applies.
If you say I'd like to request a virtual interface,
well, maybe there aren't any available on a particular node.
If you've been in this space for a while, you may be asking yourself,
well, don't we already have device plugins, which have been in Kubernetes for quite some time?
And believe it or not, even back in Kubernetes 1.8.
This was also designed for AIML scenarios as well.
And the thing with device plugins, and we do use these for SROV, for example,
is it's just a integer counter.
That's all that you get, is just a count of resources,
1, 2, 3, 4, decrement the counter if you use one on a particular node or not.
In DRA, you have a way to express this with Kubernetes resource,
resource class and a resource claim, which actually gives you a richer structure to express those.
And additionally, you can also, instead of saying one pod claims this one resource,
it's exhausted, you can't use this anymore.
DRA allows you this particular opportunity to say,
share this resource.
So why would we use this for networking?
And I would say that the problem is really actually compounded by a problem that a lot of people are trying to solve today,
which is actually where you need GPUs and nicks at the same time.
So if you've got a scaled up training scenario where you're using GPUs and you're using nicks,
because you need to have a high data transfer rate between nodes to share the checkpoints as you're running through training,
you're going to need to allocate both the GPUs and the nicks,
and you're also going to need to get, particularly like a numa alignment.
So you have like the same compute nodes in order to, in order to get that high data,
to get that high bandwidth that you need.
And so that's one part.
And the other part is we kind of still don't have a common Kubernetes native way to express our intent to use a network resource,
as Miguel is mentioning, we've been struggling with this for years.
And the question also is, why not CNI?
Well, CNI is used widely for networking in Kubernetes,
but CNI in some ways predates Kubernetes as designed to be container orchestration agnostic,
and then Kubernetes one.
So it's the container orchestration engine that people talk about.
So you have two different API.
So if you're somebody who's building Kubernetes controllers,
and you use Kubernetes API all day every day,
and you go to use CNI, all of a sudden this looks like a total non-secward RQ.
So sometimes people want to kind of walk around CNI,
which removes kind of like a common place that we use for problem solving.
And are there downsides to DRA for networking?
Yes.
And we are trying to work through these, and here's the thing.
CNI itself is actually very elegant and very simple.
I'm a real big fan of it.
DRA has a number of moving parts,
and those moving parts will probably be easier to interact with programatically
if you're building Kubernetes controllers,
but as an administrator of a system,
as a human interacting with a system,
it's more parts to line up.
And at the end of the day,
don't be afraid that CNI is just going to disappear
as we know, APIs kind of are forever,
and CNI will continue to be used.
And in fact, there is a project that is out there,
very, very fresh, that's a CNI DRA driver
that's built to bridge these worlds together.
So let's take a real quick tour of the community.
A few of the working groups that you may want to visit,
there's a number of stakeholders in this between the INL side,
the networking side, and otherwise,
one we have the device management working group,
which was looking to build out structured parameters for DRA.
This is particularly to solve a shared resources problem
where let's say you have a GPU,
that has 48 gigs of V-RAM on it,
and you schedule a 12 gigs worth of V-RAM usage on that GPU.
Well, you just have like reserved the whole thing,
and now you've wasted the majority of the V-RAM.
So this was intended to help share that.
There's also the Kubernetes multi network working group,
as Miguel mentioned,
to come up with a Kubernetes native way to express your intent
to have multiple network attachments.
They're very interested in this,
and last but not least, is the network plumbing working group,
which actually has been working on the Kubernetes
multi networking problem for eight years.
And it's hard to work around the number of the problems
and knows the pain points of these very well.
There's also a number of projects out there for you to take a look at.
One of my favorites is one called network DRA,
which is a POC by Lionel Join,
and kind of does some of the like initial exploration
into this idea of using DRA for networking.
Lionel also spearheaded the next bullet point on the slide,
the CNI DRA driver.
This has exactly one pull request merge to it,
as of today, it is spanking new.
And also out of the device management working group,
there's a number of examples that use SRIOV as part of their examples
for those AML training scenarios with high speed networking.
In closing, what is the future of this?
And part of the question is,
we don't know, we're kind of seeing the POCs right now.
And so that means it is the ultimate time to get involved.
Your voice can really be heard.
I also think it's an incredibly important time
for the community to be involved
so that we don't necessarily see the 800 pound gorillas in the room
running away with it and taking it for themselves.
Like, if you have a use case,
if you develop CNI solutions,
you can actually get your voice heard right now.
And I think it'll be really important to the long-term life cycles
of these to have you there.
And to me, the answer is really why
why DRA for networking is because we need a Kubernetes native way
to express how to consume network attachments.
We don't have that today,
and this represents potentially a common place
where we can start to standardize that.
If you would like to learn more,
please check out this blog article of mine.
It goes into depth, and also gives a couple pathways
to put your hands on this technology
and run through a tutorial to actually see it in action.
So I really appreciate that,
and thank you for your time.
Any questions?
Thanks for the talk.
That was really interesting.
I know there are some people from IBM research group
that were working on the CDI concept,
which stands for Composable Disaggregated Infrastructure.
And for me, it looks like the DRA
that we have been talking about through the presentation.
So my question here is, are you familiar with it?
And if so, are there any similarities
and differences with the DRA concept?
Great question and a very informed question.
Actually, if you bring up some of those POCs,
you're actually going to see that they also use CDI as well.
So you're going to see that that is actually an important way
that some of that information lines up
at the container runtime level as well,
because there's sort of like Kubernetes layer of expression,
and then what's going to happen at the runtime
to get that information down into the containers
and CDI is used.
So you'll see on some of Lionel's POCs that he's saying,
hey, enable CDI.
No problem.
Anyone else?
No? Okay. Thank you very much.
Thank you. Appreciate your time.
Oh, yeah.
Oh, you were? Okay.
Okay, cool.
That's awesome.
Oh, yeah.
Thank you.
Oh, yeah.
Thank you.
Oh, thank you.
Oh, no.
I'm just trying.
So, I have to speak to myself.
Yeah.
Oh, yeah.
Thank you.
All right.
Who do you are for student?
Who do you work for?
No.
Hello.
What are you doing?
Yeah.
But we don't.
We don't.
I'm not using it.
No.
I can't.