Hello everybody, welcome here at the Lightning Talks in Brussels here at Firstum. I want to introduce
to you Phillip Stanna. He's talking about how to clean up a 16 year old Linux kernel API
give him a warm welcome and enjoy the talk.
So, hi everyone, I hope you're the good for us them and not too many beers. So, I'm Phillip,
I'm currently a Linux kernel engineer for graphics cards, mostly, and infrastructure they need.
And since this is Lightning Talk and time is of course resource, we jump right in.
Just one disclaimer, we are looking here and quite some old code that is quite bad.
And I just want to point out we just criticize the code right and we don't want to condemn
the offers, because you know what, it's like people are sometimes on the time pressure are doing
things for the first time, sometimes we have 10 years more experiences, yeah, it was a better
of you actually. As an user, I can hear a quote that I like quite a bit from a former colleague of mine,
who said, yes, it's often developed, you know, at first you think that the others can't code.
And ultimately, you realize that no one really can. I think approximately it's true,
maybe quite well. And so, we are looking into a specific subsystem of the Linux kernel,
the PCI subsystem, and notably the PCI Express port, and PCI Express is probably the most important
boss on your computer, like virtually everything your computer does, and I know it's multiplexed
through PCI Express lanes ultimately. And it's quite old, but I think it's from 2003 or something
someone in 20 years, it's quite an old code, probably older than many people in the audience, right?
And the subsystem in the kernel, where this code lives, currently is only one full-time antenna,
which is not that that many. And as far as I know, it has like two free APIs that are kind of broken,
should be repelled with a very difficult throughput. And we're looking into one of those APIs,
which courses will overflow, some potentially only find behavior. And we don't have to understand
that much about PCI. All that's important for our talk here is Jonathan DeFolling. So in PCI
of PCI devices, for example, your graphics card, and the card consists of so-called bar
spasic address, which is still some memory regions, basically, physical memory regions.
The best example probably is your video ran, that's a PCI bar. And now as a driver, when you're
at a graphics driver, for example, you want to move shaders and stuff into your video memory,
so you need to access it from the CPU. And to do that, you need to do something that's called
an IO remaps. You need to map the physical addresses into your virtual address space, which is
in the right set here. And then you get an IO pointer, and with that pointer, you can access
the device and copy stuff on the school. And then PCI does several functions to do that, like
depending on what precisely you want to do, the ranges you want to use, and other requirements.
Those details are interesting for us, but we're looking into those IO remaps functions in this
talk here. And so one day, I was tasked with some GPU stuff, and I wasn't expecting anything
able to happen on that, and I was just about this function here. I was just called PCI and I
am at table, which quite an interesting prototype. We now see languages, sometimes a bit
nasty with those, and then I was like, okay, what's that? And this is a function, it takes a PCI,
there was this parameter, and the returns are points pointed to a constant pointer. And then
really get what is it doing. And then do some good blame, because it's from 2007-2009, right? It's
quite some old thing. And if you don't understand how something works, then the best thing to
do in the kernel is look for users. Look who is using that function and then try to get, and
alternatively could also read the documentation, but who does that, right? So, so we look for users
and then you just cover the following. So this function always occurs as a pair with another function,
which is PCI and my own imaginations. This precisely is the own met function, which does the O remapping
into the virtual address space. And, well, what it does is, it does the O map, and the
memory request. And for some reason, you specify which bar you want to map with this bit mask.
So you set the 0, bit, and you're going to get the 0 of bar. And then, for some reason, the actual
IO address, your driver obtains through the set O map table function. And you do it by indexing
over the function. Well, compile people would beat me. You can't actually index over function,
but you index over the function returns and so on. But you get the deal. I had never seen something
like that. And see, I didn't even know it was possible. But the miracles are covered by themselves,
I guess. So, and then you ask us why did the implement that, like 16 years ago, it that way?
Well, why? What was the background? And the background is the following. I assume I don't know
for sure. But it seems the offers, intention was that this bit mask functionality would allow
for mapping many bars, more than one simultaneously, by setting several bits. And that now,
seems to be the reason why the second function became necessary. Because, and see, there's not really
a smart mechanism to determine several IO pointers or pointers in general to the caller.
I mean, you can do some tricks, I know, but, you know, and hit a graphic shows you what's been
done. So, first you do the IO map. Then that function enters the IO pointers into some global
table that's administrative somewhere. And then through the home map table function, you index
over that. And then you get the IO address. And you can pass it to your driver and then the driver
can do both of the stuff. Yeah, and that's bad. Why is it bad? First of all, most notably,
this awkward function index mechanism doesn't allow for bounce checking. So, neither the function
can check the parameter you're passing here. And to see long, which is for us, I mean, we're
also as an mechanism to check that. So, if you had a few beers too many in the start hacking,
and you can just randomly overflow this table, we just saw the other graphics. And then you
have the archetype, the par excellence for undefeated behavior and see because you just
get some random point of a memory positive to the driver, the driver might try to write into
the random point somewhere in the memory and everything explodes into your face. Well, and the other
things, PCI currently supports like a maximum number of PCI bars of six. So, most of us are
like three, but a one bar, or maybe three. And now, the entire mechanism becomes questionable
because a bit more, the bit more is an integer, we're like 32 bits at most, it doesn't even
extensible. If PCI wants at one day one day introduces like one thousand bars, you know.
So, that's not good, we don't really want that. Because, you know, Colonel APIs,
centrally prepared, APIs should be robust. It should be as robust as possible because
drivers, for example, are not always written by people like Ken Thompson and Super Geniuses,
there's sometimes really written by interns and all that. So, the more robust you central
infrastructure is better. And then other things is that there are people who like to be clever.
That's actually code from the Colonel, here's the file, that's what this is going on,
that's only part of the code, by the way. So, here he is, it's okay, x3 because someone was
too lazy to set the bit in a readable manner, and then you know, okay, x3 is like
the lowest to bits being set. So, it should be the first two PCI bars that the drivers
requesting, but there is being shifted with a base and I'll tell you that on how I code is in a loop.
So, you have no clue what the code is doing anymore. And we don't want, at least I'll
say we won't want people to write super double clever code, you know. We wouldn't, we could
blame the offer now here, but we don't want to encourage it. And if you want your children to
eat green vegetables, then where they shouldn't do is you shouldn't give them two pieces, right?
You don't have to, it's not a good strategy, you shouldn't, shouldn't provoke the problem.
Yeah, and then when you search for the Colonel, how's it actually being used?
There's when I started working on the Earth, there's like 131 users, almost all of them
because one bus, it is at one bit in the bit mask thing. And as I said, the maximum puzzle thing
would be six bars, and if you're really in six bars, then there's a revolution in a
recurrence of the programs called the loop, then it goes just a function that's a single ball
like six times, and that's it. So our conclusion is this RPAPS overengineered, we would like to replace
it with a simpler alternative, when I own a region without S function, where you're just pass a bar
index as a number, and that returns immediately to pointer, and then you're done.
And now, okay, you can say you can easily remove it, can't you? But there's some obstacle,
so now we saw, there's 131 users, that means this code is scarred around dozens of subsystems
and drivers, and the Colonel pitches a typical merge per subsystem, so you need to know knowledge
and act the approval of each of the maintainers, and should be reviewed under reasons to the
kind of tricky. And so effectively for Colonel political reasons, it's not possible to replace
the API in one go, that's very difficult, and despite the way something that's surprising
for people who are new, the Colonel is not only able to acknowledge it, it's a lot about
process, about discussions, about Colonel politics, you could call it. Anyways, we want to
get rid of it, how would you do that? So that's the pre-solution state what it was like
one year ago, then you have all the drivers up there, they are omaped their PCI bar, this function
unpasses the outputter into the magic table, and then the obtained the address through this
magic in the expansion thing stuff. Okay, and how can we phase it out out of the grammar?
You can do it by providing our simpler alternative, then I just address which is prototype,
and that would be a function that just requests our IONF's one bar, and where do we implement it?
So the way I did it is I implemented this as the new base of the omegreations function,
it just returns one mapping address, and the only user at the beginning is the regents function,
the broken one, and the rest is not changed. And this is the great advantage, Colonel process
was if you blow things up and call the regression, it's very easy to revert, because you have only
one place, the PCI subsystem, where you can repair the function, and to get revert and you're done,
instead of 141 places. So that's already good, the other thing is you immediately recognize if it's broken,
because all the users will use your code, which is also handy when then later convincing,
maintain us to merge more of your code because you can say, hey, you're already using it,
below the surface. And now that we have this simple alternative, we can expose this new pair,
maybe make a public, if a new driver like F here comes along the upper right corner,
it can immediately use the new function, obtain and smapping directly, we felt this awkward table
index thing, and now that the new alternative is public, we can deprecate the old API,
and say, please don't use it anymore, and programmers always read documentation, so it will definitely
not happen to someone who will use the old API, and it will definitely not happen two days before
my talk at Boston. Yeah, and then due to the step by step, we'll report more and more drivers like
D&D, and one one day, 131 users later, you're done, then you can delete the old code,
and if one clean function that does this one simple thing, net the stuff, return the address,
or an error if there's more so problem, and classic unix philosophy, you know, do one thing and do it well.
So, down some conclusion for, because many people would like to contribute in the current,
maybe some of you would like, I would recommend this guide here, I would recommend browse code,
did you interest them, preferably code that you have some experience with already, that would good,
and if you see something that looks broken, then it actually, most likely, is broken,
because my reaction with that code, for example, is there must be some genius reason why it's that way,
I haven't understood it, but that's what it's often the thing when you're doing things in the
current for the first time, because some of the things, the feeling some people seem to
perceive the current engineers a bit like the elves in order of the rings, you know, the most
noble and wisest of all beings, but if it looks broken, likely it's broken, so I'll recommend
try to repair it, and in case of that, ask on the mailing list, ask the maintainers,
is the maintainers duty, in my opinion, to guide you, to guide new contributors, through a patch series,
if the maintainer doesn't do it, then he doesn't have to write on my opinion to complain about burnout,
yes, so yeah, don't hesitate, repair things, also that it's good for your
reputation, in my opinion, because it's good to be known as someone who repairs stuff, right,
because many people only want to implement new features, because that's the exciting thing,
but I think it's good for us to be in the reputation, so I repaired something that was broken for
16 years, it's a cool thing, so, and then you have success, and that's good, and in that sense,
if you'd like to jump into the kernel, into kernel work, would write, ask, and if you have
a good remaining form Lewis.