All right, if I may have your attention, please.
So this is a lightning talk.
It is 10 minutes.
It's going to go fast.
Questions?
You can seek me after work.
My name is Machidenway.
I work at Aficious.
I'm here to discuss the problem of figuring out
from a container perspective.
How many CPU can I use within that container?
If I'm on a vast machine, where I'm actually constrained
to a small subset of that machine.
So there are various uses to use the notion of knowing
the current CPU number, the maximum number of CPUs
in user space.
Some of those use cases, tracing ring buffers.
That's a personal interest of mine.
Memory allocators, decimala gmailac.
The Gnuci library mallac.
Caches, such as the NPTEL,
Treads.cache in Gnuci.
Schedulers and user space statistics counters.
And user space generally using the observable number of CPUs
to automatically scale the number of threads.
So the context of this problem.
It becomes much clear when you look at modern machines
with 512 or more hardware threads.
So the memory footprint that it brings,
when you use CPU data structures in user space within
CPU set constraint container.
That's an issue.
And also scaling the number of worker threads.
So I'm a maintainer of the rest-artable sequence
system called in Linux.
And in Linux 6.3, I've contributed an additional field
that allows figuring out an index that is indexed by the
thread number within an index bound by the maximum number,
the maximum number of country running threads within the process.
So the current approaches to bound that maximum number of running threads.
So one approach is to use CPU set within your container.
So this provides a memory user per bound when you limit the
containers to CPU set.
But it's not ideal to describe constraints in a cloud native way.
It's bound to the machine topology.
It's hard to compose containers.
That are expressed with CPU set constraint.
And it's tricky with big little key core e-core CPU.
Because then when you wire yourself on a specific CPU,
you're actually telling, well,
I want to use a performance core or an energy efficient core.
So this is semantic.
You may not want to convey in terms of
restricting your application.
Another approach you have currently when you want to limit the bandwidth
or the quantity of CPU time you use within a container is to use the CPU
C groups.
This allows limiting containers to specify portion of time slice.
For instance, the CPU max interface file allow you to say,
I want to use 2,000 microseconds per 1,000 microseconds slice.
That means 200% of CPU time, but you can be migrated anywhere on that system.
So it can be either 2 CPUs running the workload 100% of the time,
or 200 CPUs running 1% of the time.
So that's not a good way to let your system know that you want to be
run on a subset of the machine's cores.
So the proposal I have, and I discussed that at LPC 2024,
I'm bringing this again to gather feedback.
It's to introduce a new CPU max concurrency interface file.
That would define the maximum number of concurrently running threads for the C group.
So in terms of what needs to be done within Linux is to extend the Linux scheduler,
migration and load balancer to track the number of CPUs that are concurrently used by the C group,
and to constrain migration to the currently used set when the number of concurrently
CPU reaches maximum threshold.
There are some additions that would need to be done in the C groups that are structures,
such as counting the number of threads in each run cube belonging to the C group with
CPU counters within each of the CPUs.
Count the number of CPUs in a global counters within the C group,
and keep track of the set of CPUs in the CPU max within the C group.
So the tricky part here is to handle errors and combining users that want to set their scheduling affinity
and use CPU sets in combination with the max concurrency limits.
And so this is where I have some ideas, but I also have some things that would need further
class.
So some of the ideas are to make scheduled affinity and CPU set fail.
If the request for a given set is going beyond the available concurrency limit,
decreasing the max concurrency could fail also.
If decreasing the max concurrency under a certain threshold cannot be fulfilled,
given the current set of constraint.
Increasing the max concurrency should never fail.
It's just giving more room for scheduling opportunities and migration opportunities.
So that's not such an issue.
What I expect though is that we may have some worries about,
basically, what I suspect is an NP complete problem,
which is basically if you have partially this drawing sets of affinity.
So that's where I'm not entirely sure how we want to tackle that problem.
Because I want to handle this at the point where the user are modifying the affinity
or setting the max concurrency, so that it's basically just an impact on the set,
then for the allowed CPU mask.
So we don't have any impact on the fast path of the schedule.
It's only load balancing and migration that get affected with additional constraints.
But in order to do that, so if we have this joint set,
that's a part where we may have to rely on some statistics.
So feedback would be good on this point.
So someone saying, well, I want to find my first thread on CPU zero and one,
second thread on one and two, third thread on two and three,
then what's the set of one or two CPUs that would allow doing your scheduling of all those threads.
So at that point it becomes a bit trickier.
I suspect that typical case would be having some kind of a general super set.
So of all the general mask that allow free movement within a given set,
and then subsets, so pinning specific task to specific CPUs, that's one easy case.
So those might be kind of the two main use cases,
but I'm interested to hear about how there are many uses of those kind of these joint sets,
and how should that be handled.
So that's what I have.
We even have three minutes for question and I guess setting up the next speaker.
Okay, are there any questions?
Is there any overhead of exposing this new field in the,
because you have to track this concurrence field all the time?
Have you calculated the overhead of this?
I don't expect that to have much overhead because that would be limited to migration and load balancing.
So there would be some overhead on that,
but I would be very careful about not impacting the scheduler fast path,
because as soon as we start adding constraints that need to be taken into account in the fast path of the scheduler,
that would be maxed by the scheduler of maintainers.
So that's the target.
Do you have time for one or two questions?
Do you already have patches?
Not yet.
I'm actually, so I've started discussions with some,
well, it's not being at the top of my priority stack at the moment.
All right.
All right, thank you.
Thank you so much.