Hi everyone, that is the corporate disclaimer. My disclaimer is that I'm about to tell you
a lot of lies. Yeah, you're welcome. That's because if I told you everything that was in detail
that was absolutely strictly accurate, we will be here all day and we just don't have time
for it. So if you know better, if you think, hey, he's oversimplifying this, yes I am. I'm
going to mislead you. I'm going to give you enough information that you understand why I'm doing
what I'm doing, but don't don't rely on what I'm telling you if you're working on something else.
What I'm telling you is true for the purposes of this talk. So, Mem, why do you want to shrink it?
Well, it takes 1.6% of your memory at boot up and some people think that maybe they have
better uses for it. I think of it as being 16 megabytes per gigabyte. So if you have like an 8 gigabyte
phone, then that's 128 megabytes of your RAM that's being used for all mem app. And maybe you'd
like to load one extra tab in your web browser or maybe you have a 1 terabyte server and that's
16 gigabytes of memory that maybe you could use to rent out one extra VM on your server. So it's
not a trivial amount of memory. I think working on performance, stuff for a while and it's
never been big enough to get on my radar, but there are other people with other use cases for whom.
Yeah, this is enough that it's worth spending a bit of time on and by a bit of time I mean we're
coming up on 8 years. So we've got some other benefits on the way. So what is Mem map? If we're
going to shrink it, we have to know what it is. And it's actually just an array of struct page.
And there's actually there are actually three opinions about how big struct page ought to be.
Some people think it's too large because that's 1.6% of your memory that you could be using
or something else. There are people who think it's too small. They want to add extra stuff to struct page.
There are constantly proposals for, hey can I just add this one extra thing to struct page?
The most audacious of these was something you want to double the size of struct page.
So they could put different things on different cash lines. And that was like, wow, no.
There is an extension struct page x, which exists. It's absolutely horrible, don't look at
it and it's slow. If you don't put performance stuff on it, you put stuff on it that you've
got nowhere better to put it. And I hope we can get rid of it. And there are people who say don't
change a thing. Struct page is exactly the right size. It is 64 bytes. 64 bytes is a magic size on
current CPUs. It's the size of a cash line. So if you do one thing to a struct page, it doesn't
affect the struct page that's next to it with somebody else that you're using. So it is perfect
size, don't change a thing. So we're going to bear these three perspectives in mind and get
back to them at the end. So how do you get a struct page? We call the page allocator. What's
the page allocator? It is the fundamental memory allocator in Linux. There were lots of other
memory allocators which have other uses and you almost certainly want to be using one of these
other memory allocators, but for some things you want to use the page allocator directly.
And all these other memory allocators sit on top of the page allocator. They all get their memory,
but they hand out from the page allocator to begin with. So when you call alloc page, which is
your interface to the page allocator, that gives you not just the four kilobytes of memory,
it also gives you the struct page that controls that four kilobytes of memory. And you can use it
because you call alloc pages is yours. With some arcane restrictions, I'm not going to go into
because we don't have time. So when I came to struct page, this is what I like in late 2017.
And you're looking at that as the thinking that is a wall of text. I like to use this as a
scare people. You can't tell what most of this is useful. If you want to change anything in
struct page, if you want to use struct page, you look at this. I don't know. I have an unsigned
long. I want to sort in struct page. Where should I put it? Well, there's all these counters.
Counters is an unsigned long. I'll use counters. And then you realize that actually you just
overwritten riff counts and everything breaks. So you try to do something else. And it's all for
right. You can't really use this without spending a week trying to figure out where are the
holes, what can I use, but what do some of the else we're using for? So this is now late 2018.
These are the long time support reasons I went through found the right years.
So I've rearranged it and now you can tell because there's comments.
There were comments before. They weren't very useful comments. But now the comments actually tell you
who uses these things. What are they useful? So if you've got something to use for slab,
it's in the struct slab. These are all anonymous unions because that made everything keep working.
We could have gone through a named all the unions, but that would have been a lot of shurn. We didn't
really need to. This was good enough for what we were doing at the time.
So yeah, this is now a little bit more understandable. And that paved the way for what came next,
which was moving stuff out of struct page. You can see. We could actually go this into two columns,
maybe even one column. We would have been able to get it shrunk further, but we also discovered
page pool was using some of the elements that were here. It was a hidden user. The page pool
user was using some of these elements and we didn't know because nobody had gone to look.
So it's been made as official and we said, okay, you've got your own substructed
struct page. But what we've done is we now have a separate struct slab and we have a struct
and we have other structs. They overlay struct page. So you can't see them in struct page anymore,
but struct page is still being used for those purposes. And this is a necessary intermediate step.
It's not a great step because it's kind of nice to look at struct page and go to see where do
these how do all these things line up. We have build time assertions to make sure that things
that do need to line up still do line up and scary comments saying that there are dragons here,
please don't touch. And you'll see that this is a kind of temporary state of affairs.
Okay, so now we've organized struct page. How do we shrink it? I mean we shrunk with a
finished struct page and there's fewer lines of sea code, but it's still occupying 64 bytes.
So we need to identify redundancy. So what do we have in struct page that we could get rid of?
And the primary thing that we're doing is dealing with multi-page allocations. So it's not just
alloc page, the alloc page is actually just a front, it's just a macro. It's, of course,
alloc pages which fascism order. Sure, that's exactly the kind of comment I didn't want to hear.
So alloc pages gives an order. So order is the log to base two of the thing you actually want.
So if you want to allocate four pages you call alloc pages two and that gives you two to the two pages,
which is four. So when, when you do that, all, we, we, we talk about the head page,
the first page. And then the tail pages, which are all the ones that come after it. So if, if you
call alloc, alloc pages three, you'll get seven tail pages and one head page. And the only thing
in the tail page is this thing called compound head. There's a little bit more to it than that, but
simplify. So if all we've got in the tail pages is a pointer that says, hey, that guy over there,
he's the real head page ask him not me. We can shrink that because using 64 bytes to store eight
bytes is wasted. So instead of pointing to the head page, we now, we're going to shrink
drop page dynamically allocate the data that was stored in that 64 byte head page and then have
all eight of the pages point to that page. So eventual goal is an eight byte drop page.
That's a lot of work because we have actually encouraged people to use stroke page. If you're
calling alloc pages, then you get to use stroke page. That's the contract, that's what we told people.
And now we're saying, yeah, could you stop doing that? So, you know, it is on us as, as Mary
Management people to give them a path forward, right? We're not, we're not, it is up to us to
fix their code for them because we've we've changed the contract not that. It's going to be a lot
of work to get to eight byte struck pages. So the 2025 goal is a 32 byte struck page. So we're
going to heart half it. We're going to give you back eight megabytes per gigabyte instead of the
14 gigabytes that we're promising. So it's going to be much much smaller. It's going to have flags,
Mendesk, RF count, and a private. So you can allocate extra memory and hang it off private
or maybe hang it off Mendesk because that's going to be up to you. So what we need to in 2025
we to finish the folio conversions and file systems. So there's still a lot of code. There's still
some file systems which are using the elements of struck page. So they're doing page arrow mapping
or page arrow index and they should all now have been converted to folio index. I think
Seth is so outstanding. F2FS just finished. There's a few other things but most mostly we're done.
I did have a goal of getting rid of page index this much window. It was always a bit of a stretch goal
and we didn't quite make it but we got almost all the patches that we need. So next murder window
page arrow index is going away for sure. I have a bunch of patches queued up to get rid of page mapping.
I started a patch series to get rid of page Mendesk gd data and we're almost there. I need to go
back and we visit page LIU. So despite the fact it's called page LIU it's just a list head. So
if you want to put pages onto a doubly linked list you use page LIU. So well that's not an LIU.
Well but it's the right type to be a list head so we're using it.
Okay we will fix this. We will find a way that you can keep track of your pages that doesn't
involve using a list head because linked list or evil and I've given rants about how evil
linked list are before. So a lot of independent projects here like everywhere in the kernel that you
see page LIU or page mapping or any of these other things being used that's somewhere that if you're
inclined to help you could picture. If you do please we have a THP cabal call every other Wednesday
and just let me know if you want an invite and we can call out we can coordinate and make sure
nobody's working on the same project because there's a few projects that there's a few uses of
page LIU still out there that I'm not entirely sure how to fix but we'll figure out a way.
So we have some places which do implicit casting between struck page and struck folio which is
all fine for now because they're the same data structure but we're going to separate them then
we need to stop pretending that the same data structure. So one of those is again within file systems
we have this thing called the buffer head buffer head used to have a well they still have a
B page pointer and the points to the struck page that the buffer head points into. Now there's a
B folio as well and we need to get rid of all the mentions of B page and make sure we're
pointing to B folio. There's a couple of other examples like that but that's the kind of thing
we're talking about. We need to remove the uses of folio arrow page so one of the things I did to make
this all easier is that inside the definition of struck folio it says oh yeah there's a struck page in
here as well so it looks like the struck page is within the struck folio. So and that was always a
deliberate sign that this was code that was going to have to be properly converted at some
point in the future. So I made it easy to grip for and we've been getting good progress
I'm getting rid of uses of folio page. This was particularly handy for file systems because there
was so many interfaces that file systems used that as we progress through the folio conversion
we needed to have this kind of easy escape hatch like oh this this is all stuff that hasn't
been converted yet let's just convert back to struck page and we'll deal with it later and we're
getting to a point where that's just about finished almost every API as a file system wants to use
has already been converted to folio there's some weirdo stuff like FScript that still hasn't been
converted and that's just something that needs to happen and it's something that I haven't done yet
because I'm not all that comfortable with FScript I don't know the code that well and I'm just
going to have to learn it. The page pull out a key to need to be split out of it so I'm struck
I just talked about that a few minutes ago and then the months we've done all of that we can
we can dynamically allocate all these struck folios lab CS mallet pts page will
I'm calling all these things Mendesx generally so that memory descriptors name is hard
particularly since there's there's no struck Mendesx everything has its own unique type
so we don't really need a good name we just need a name that we can all use
when we need to use it or should we rarely so is everybody happy with this
we're making struck page smaller so we're getting memory back sometimes I could book
or make more money I mean so that we can charge less per virtual machines
struck pages now rarely modify when we get to the 32 byte struck page
very little it gets modified at allocation and free but at runtime while the struck pages in use
we don't often modify anything so we don't have this problem with interfering with neighbors
and particularly if you're doing large allocations which was kind of the whole point to the
of the folio project was to make it easier for the page cache when in my most memory to do these
large allocations the pages that you're modifying will also belong to you so you're not
interfering with anyone else but it's all your stuff anyway so that's satisfied the first two crowds
the third point of view that but I want to add struck stuff to struck folio so that's okay you
can add you can add stuff to struck folio and it won't affect struck slab and it won't affect
struck page pool well vice versa you can grow struck slab and not care about how's that going to
affect struck folio because the real problem growing struck page was it affected everybody
affects the entire system and now everyone's got their own allocations you only have to justify
it to the other users of struck folio or to the other users of struck the SM allock which is
actually an easy SM allock so the the the the the the the last radius of your change has has
much decreased which makes it much much easier it's that a good thing I don't know probably
I mean people want to do these things for reasons right so okay it's all the downsides here
well at least for the 2025 edition if you're still doing order zero or order one allocations
we're going to use more memory in total so we're going to have like a 96 byte struck folio
and let's say we're doing order one allocation two 32 byte struck pages such 64 byte plus 96
by such 160 bytes which is more than the 128 bytes that we use to use so that's the net loss
but once you get up to doing order two allocation you're using less memory in total so that's a win
the moral of the story is that if you know a file system that isn't using the
isn't large folio enabled yet you you should you should talk to the fastest and developers and say
why haven't you done this yet because I tell them that and they're listening to me so
I shouldn't say so many many many many files systems have now been converted to use
not just folios but large folios and more are in progress
the fuse is particularly interesting because they did it and then they found a performance problem
and so they've had to undo it at least temporarily which is unfortunate but these things happen
so one thing that might make people unhappy is that we may be getting cash misses when we go
from pages to folios and and back again there's not necessarily going to be a lot of
call to be going from pages to folios I'll give you an example when when when you do an ODIRECT
I know we have this thing inside the kernel called get user pages so you say hey I want
I've got this trunk of my address space that I want this this this set of memory addresses I
want to do I owe to and inside the kernel we walk the page tables and we say oh here here with
the struck pages here here here are the page table entries for that range of addresses and we will now
get what will now get the struck pages for each of those and we will convert those into the
folios and then we'll send it off to the file system and so that's going to be slower because
there's going to be this this in direction right we're going to take one extra point of
dereference to do that and maybe that's going to be a problem and maybe it isn't because you're
doing the IO maybe it's just not the big deal now what what may make some of these things better
is that we can actually shrink struck page for it with more work we can get more benefit so we can
go to 16 bytes relatively easy I think that's something we get done in 2026 I don't think we have
time to finish off in 2025 but we can certainly you know try hard going down to 8 bytes is going to
be really painful and may not actually be worth it because there may just be so many places that
need to be changed that we don't want to do it I mean all of these things may not be worth it right
we may we may get to it and say hey the performance loss that we get by doing the page to
folio conversion in the back end this whole thing just isn't worth doing and that would be very
sad I don't I think I don't think it's going to be a problem or you know wouldn't be trying but
if we if it does turn out that we just need to give up on that side here well at least we've made
the kernel a little bit smaller a little bit cleaner we we no longer have quite so much type confusion
and we'll have a simpler to understand kernel and that will be worth it so I I use the kernel
newbies wiki as my development notes so I just just other people can see what I'm working on
that's the current page I'm using for for this chunk of it this has not been a one-man project
there's about 14 aims on that list and I've probably missed a few people who have helped out
throughout this project it it it has been progress for about eight years we've we've accomplished
really quite a lot in that time I mean we're we're not on the real time when it's 20 years
plus kind of schedule but we've got all the code merged already and you know we've really been
structuring it that way that every step along the way there has been some measurable benefit
to somebody to some workload I mean we we got a doubling in right bandwidth of NFS for example
and you know that does that does just an amazing performance improvement but I want to thank
all of these people and I want to thank you for coming to the talk
all right questions hi so in the 32 by the version of the struct there's a memdesh
and a private if the future intent is to shrink it even further why wouldn't you make the second one
online long reserved and not permitted to be used in first place you already have
things in mind that users will are going to need that for yeah there's a bunch of different ways
we can get to a 16 by the struct page the important thing is it's not worth going to a 24 by the
struct page we want to keep it a pair of two so we need to get rid of ref count and then we can
get rid of I think flags I think we're going to get rid of ref count and flags and just have
memdesh comprise that there's a bit more to it right I I've been I've been it's been simple
flying things here but I think we can get to as 16 by easily if we get to 8 by it's just memdesh
right that's the only thing in it and memdesh is actually a types point so the the bottom
four bits are used to know what kind of memdesh we're pointing to and then the flags or the flags
move it into the memdesh there's a lot to it and it's actually all up all on the the wiki page and
a lot of detail so I take a look and if the wiki page doesn't explain it properly then drop me an
i don't know you no doubt.
Ok let's see how we do
Well we're just a person below no it's alright ok yes wands simple question if we share one page
which track between one holo can, I would say,
CPU user pages for tracking access.
And we have follow with one page, physical page,
hot and other code works.
It happens, in this case, all follow will be marked
as hot usage or do we have a chance to split
and follow to hot and cold and safe memory.
OK, yeah, that's a great question.
It's more part of the folio project than part of the Mendes project.
But this is kind of the trade-off that we're
choosing to make that we have decided
that it is not worth managing memory in four kilobytes
trunks.
Four kilobytes is just too small to be managing memory in.
And that means that we're trading off the precision
of managing 4K for the bandwidth
of being able to manage in 16K, 64K chunks.
I think it is an excellent improvement to try
and detect this kind of situation where we
have made the wrong decision, where we
have the situation that we should not have merged these multiple
pages together into a single folio and track them individually.
And we have the flexibility to do that.
We just don't have a mechanism to do that today.
We can split folios.
That part works.
But we don't have the system in place to say,
to know that has happened and do something about it.
And if somebody wants to do it, that will
be a fantastic improvement to learn next.
And I think I hope somebody takes on that as a project.
And that actually works.
Because it's always been in my mind that something we should do.
It's just I don't have time.
And I don't necessarily have all the best knowledge
about how we could do it, what the right way to do it is.
So yes, you've raised some excellent points.
And I hope somebody does do that.
All right.
Well, after time, thank you very much.
APPLAUSE