All right, I think we're good to go. Let's learn about rest on Linux.
Yeah, welcome to my talk on Restful Linux. I'll give you an overview of the project, what
it is. First one, my name is Anis. Anis Astier. The live blogger of the Canon Recipes Conference,
a great Canon conference in Paris, and actually to check it out. Small disclaimer, I'm not
a restful Linux contributor. Just this talk to learn more about restful Linux, I invite you
on the contrary to look at Miguel Oreda stock yesterday, it was in Jansson. It was recorded, of course,
a very interesting presentation. So what is restful Linux? First of all, it's a meta project,
just like the Canon, you know, the Canon is a group of multiple projects that merge every 10 weeks,
and restful Linux is just that it's a set of multiple different projects that want to use rest
in the Linux Canon. The goal of the project is to make rest second the main language for the Canon
in general, not just for drivers, but in general in the Canon. And upstream using, making sure
everything is upstream the core goal of this project. A quick history of restful Linux, it started
in 2013 with a demo, it was a small, a lot of module, rest.co. You can basically, basically,
we can click the links if you go, check out the slides. There are online on the page, so every link
you see is kickable. Then, fast forward in 2019, Alex Gainor and Jeffrey Thomas, give a talk
at the Linux Security Summit and inviting kind of developers to use rest for security reasons.
A year later in 2020, there was a discussion at the Linux Plumber conference with many people already
involved. A year later, Miguel Ojeda sends the first pull request with many, many contributors.
He did the project announcement along with the pull requests, and the restful Linux experiment
was merged into Linux 6.1, so it was three years ago, almost three years ago.
And ever since 6.1, at every release, of course, there were new changes. At first,
at the beginning there was nothing, there was just the infrastructure to be able to build
thing with rest, but there was nothing to build. Now there's a bit more and we'll talk about it.
First of all, why rest? Usually when you pick a mainstream programming language, you have to
make this trade-off. You have to pick two of those. You want to be able to do dynamic memory
allocations. Is your program going to have everything statically hard-coded or not? Do you want to
have memory safety? I prevent the full of flows, data races, things like that. And do you want to
have a garbage collection or not? And the language that can be free running, you can run without any
runtime at native speed. Well, with rest, you don't need to pick, you get all three, and that's
a part of mainstream language. There's no other one that comes to mind, that has exactly the same
properties. Of course, memory safety is just one aspect of the equation. And the last XDC
lead poll, she's a kernel developer, she said that memory of safety is a list convincing point.
Linux kernel developers, they've been using C for decades, they know it's unsafe, so it feels like
they're being talking down to when you keep repeating the same arguments about memory safety.
And ergonomics are what she says a bit more interesting, because rest can be a kernel
material. You can encode properties of your kernel subsystem of your drivers, of your APIs,
in rest, which you can't do in seeing, you have keeping your minds because of the many features
in rest, you have an image pattern matching trade, you have ownership and like time tracking,
ball-checking. Of course, there are reasons why one would not want to use rest. For example,
it's a new language, so you already know C learning text time, and if you've been programming
for a long time, you know usually you can pick up a new language pretty quickly. With rest,
it's a bit different, like you find walls, you find things you are used to that you can't do.
So it takes a bit of time. There are other reasons, of course, for example, the Linux kernel
supports 22 or 23 architecture families, sorry, just in general, and resty has about half of those
up to tier two targets. So if you want, there's a bit more, but it's not exactly the same number
of architectures, so some architectures, supposedly by Linux won't be able to build rest code.
Until we have DCC at least. So right now for the main architectures, you can build the kernel with
clank and DCC, and this is support for rest code this coming via two projects. So there's
DCC RS, which is working with upstream to add rest support directly into DCC,
and there's rest code gen DCC, which is another project whose goal is to change the rest
C backend to use DCC for cogeneration. So instead of LLVM, there are other by the way rest C backend,
like crane leaf, but yeah, it's south of scope of here. What is the strategy of the rest for Linux
projects for working with the Linux kernel? First, the project wants to lead by example. So it
wants to have documentation everywhere with tests, safety commands. So when you're doing rest,
some path might be unsafe, and every safe there's a link in the Linux kernel should have
a safety comment explaining why it's used. It uses a bind gen for FFI layer generation. So FFI
means for instruction interface. It's when you want to bring two languages, usually right
and interface either by hand or not. Usually not, you prefer to have this automated. So there's
a tool called bind gen that will parse the C headers of C files and generate a rest FFI. So it's
basically rest code with unsafe functions. Those unsafe functions on top of it, the rest for an
project says we should not use them directly in the account. We should build safe abstractions
on top of those bindings. So you don't call the bindings directly. What are those safe abstractions?
It's exactly what we were talking about. It's a way to encode the constraints,
which you have in the C code, the C layer, and you need to understand those constraints to put them
in rest types. That's where you need really kernel domain expertise. Next, please,
read it on the specific kernel domain you are abstracting. So usually it's not trivial to do,
but it depends. In fact, there are already quite a few abstractions in Linux, many APIs,
for example, you might know about work use, device, miss device, which has merit recently,
platform device for embedded developers and any kind of people using platform device.
There's an alloc module to do allocations. There's a PIGNM space, which was done by
a Christian here for a driver for the Android driver. Anyway, there are many others, it's just a selection,
and this is quite long, there are a lot of things you can do in rest, but for many use cases,
it will still be too short. We'll still at the beginning and you might find that you might want to
use something in the kernel, some API that's not abstracted, and this would need to be written.
So where are we with drivers? First of all, in the Linux kernel, there's this rule,
called the node duplicate rule. So if you want to merge the driver, it should not have
another driver, which has the signature for the same hardware. So of course, this rule has been
relaxed in some cases, and for in the rest of the next project, it's being relaxed for what
are called reference drivers. So this reference drivers serve as examples of how to write
a rough code. And one of those drivers is the null block driver, which was merged upstream recently,
so it's a complete implementation of the C button. For now, there are the two ones
that exist in the Linux kernel. There are other things that were merged. For example, the DRM
panic curr code generation. So it's kind of like a Linux blueprint of that, except not. So you have
when you panic, you have a lot of, a wall of text with a lot of information, the stack trace,
the state of the registered row. And instead of taking a picture or copying that by hand,
you just kind of curr code, and you have also data, you can copy test it and send it in your
degree port. And also curr code generation is done in rest. There are two five drivers for
network, social network key and thing that have already been merged. And of course, that's just
the tip of the IPS third. There are many upcoming drivers, many things that are being worked on,
one, which is well known, will be the other Linux Apple GPU driver. It's not upstream yet,
but it's being shipped, if you use as a Linux, it's being shipped to everyone with the
running Linux. There's a re-implementation of the Android Bider driver, which is a core driver
that is extremely Linux. So it's being rewritten with the aim of replacing completely the C
implementation, the same for the NVME. There's an upcoming driver again for GPUs, for NVDPUs,
and many others. Yeah, the list is quite long and many working progress projects.
Recently, in the recent updates, we've had a few changes over the last Linux version. For example,
it used to be that you had to use for to be the rest into a specific error. You had to use the
specific rest compiler version. It's no longer the case since two or three release. Now you use
rest 178, I think. It was picked very consciously because it's package in almost every
district that may be stable, but it's in testing. There are now a few types to wrap the Linux
allocators. So there's KVAC, KBox, which if you know the thing about two about rest,
is just like VAC and Bugs, but for the kernel, they have a slightly different API, and it's
done to be able to pick the type of allocators because they might be multiple allocators in the
kernel and the allocation flags. Another date, which was done recently, was that the rest project,
the rest compiler now builds the Linux kernel in CI. So every PR is tested with the Linux kernel
build to make sure that there is no breakage. The rest project takes rest for an extra
sleet, one of its flagship goals for the second half of 2024, and probably will be for the
first half of 2025. What are our kennel mentions thinking about rest? What do they think?
So globally, I'd say there's a positive outlook. As the last maintenance, it seemed that
it was globally positive. There are many supportive maintenance. I give you the example of
Christian here who even wrote an abstraction to have to have encode the properties of a domain
he knew very well. The Linux talk of the steam right ahead to rest developers, so right code,
even if it's still an experiment, and I invite you again to write what Miguel Okadastok,
who did a really great segment yesterday, and he interviewed many different kind of developers
and gave code to what they think on rest for Linux. Supporting rest for Linux is still optional,
still an experiment, or is it depends? It depends on the subsystem. If the maintenance
is supportive, it's not optional. It's part of the features, and the rest for Linux developers never
said that you can break rest wherever you want. In the last case scenario, if there's a disagreement
of something, then you can reach this point, but it's not something that was never said.
So that concludes the first part of this talk. Now we'll show a bit a few code examples to
why rest is interesting in the Linux kernel, and we'll start with this, which is the direct
copy path from the presentation by Alex Alis-Rill and Carlos Yamaz on the binder driver rewrite.
So you have a comparison between C code and rest code. So you see the rest code, you have just
a closing accolade, and that's perfectly normal, that's because of life time tracking, of ownership
tracking, and the way the drop tray works. And of course the C code is only just a part,
it's not even complete yet. Let's now look at the minimal rest drivers. So this is directly
from the kind of source tree, if you go into the examples, you see how to write a module or a driver.
For that, you will first use a prelude, it will import a lot of few things that the
rest for next developers are important into your scope. What's important, for example, you have
the module macro, it allows you to declare your driver, your driver has a type, it's a structure,
we'll see that later, and it has some metadata, and it has a name, a license, or a description,
everything you have in C code usually. And if you look a bit more, I told you the module has a type,
this is basically a structure, and this is the state of the driver itself. Here it's an example,
so the state is an array of numbers, a dynamic, an array of sign 32 bit integrers, which is a
KVAC, I thought I talked to you about it a bit earlier. And then you need to, for this structure,
you need to implement what we call the tray in rest, so the tray is called module,
so you implement the module tray, and what does that mean means that you need to have this function,
the innate function, that's what it means, implementing this tray. And inside this function,
you will see the PR info macro, it's basically almost the same thing as the PR info Linux
kind of function, so it's actually a function, it's also macro, but it's nothing important, and you
declare your dynamic array, so this will be done on the stack, and then you push things to it,
and you will call the push function, like the stack in rest, except, this function can do allocations,
and you will need to pass the allocation flags, because in the corner you might, depending on the
context, you might need to have different flags, for example, if you insert people context,
so you pass the flags, and of course this allocation is available, so it can return an error,
and you pass this error to the return of the function itself, and then you return the states,
so you return the structure, you declare with the numbers inside. The end of this driver is
to exit, when you exit, it was chosen to use a drop tray, which is a built-in tray,
which means that when it goes something goes out of scope, it's called, that's how you do the automatic
screen, as an example, that was shown before, and so the structure will implement the tray,
it has one function, and this function calls the print again, and you print the state that you had before,
so that's basically it, that's the rest, minimal sample, that's in the Linux kernel,
a source tree. Let's show now a bit more complex example from the SAE Linux GPU driver,
and one of the reasons that was set it by SAE Linux, and even later by Devali and Danielo Kremich,
is that rest is very interesting, because it allows us to abstract the firmware layer,
the interaction between the kernel driver and the firmware, and on Apple platforms, and it's
true, so for NVIDIA GPUs, the kernel developers don't control the firmware, it's controlled by
the hardware window, and with every iteration, they might break somethings. In order to prevent
breaking the kernel driver, there's an abstraction inside Linux to help, and this will be a macro.
This macro, it's a pro macro, we won't show it, we will show how to use it, but not how it's implemented.
First of all, it's not this one, this one just macro to tell the rest compiler, I want this
structure to be represented like in C, so it would be C like representation in memory.
This is a macro, and it has one argument, which is AGX, and we'll come back to it later,
and inside the definition of the macro, there's this AGX version, so it's the same,
it's referring as exactly that, and the firmware can be selected hardware dependent,
or version dependent, so the G would be generation and V version, and you have
like three hardware that are supported, and six firmware versions. Of course, we don't want to
support all of those, it would be like 18 things, and it would grow even faster, so there are only
five that are supported in Linux, for this out of three driver. Let's go back and take a look,
and what this macro allows to do, you can now add an annotation, this will be again,
an entrepreneurship at build time by the macro, and this annotation allows saying I want this
next field to be or this next expression in general, to be only for this version, and this version
it has an expression in evaluator that's run at build time. You see here's a version, it has to be,
if we want to have the counter, it has to be greater than V13, or equal to V13.
So if we go and take a look at what happens with this macro, it will generate each of the five
possibilities we've seen to generate at build time different structures. You see the structures
and I'm slightly differently, it contains a different generation and the firmware version,
and we can see that there's one that contains the counter, which is because that's how it's
declared in the firmware interface, and the other does not have it. So yeah, that concludes the
this example. I just want to show a small possibility of what could be possible in the future in
Linux, usually when you are using Rust, it's not Rust does not protect against deadlocks.
So you can deadlock, it's not one of the properties perfectly safe to deadlock, depending on the
definition of safe, talking about the Rust definition. Of course, using the Rust programming language
is possible to build a structure in a way that you can never deadlock. So this was presented
by Joshua Libre of Feather at the latest RustConf, in a talk called 16 and not safe world.
I won't go into details, but it explains how to build the context with two traits and the
macro, and how to build basically a new text directly, actually graph, so that the other in which
you can take mutex always depends on the previous one, and you can never take it in a wrong order
if it has to be satisfied at compile time. And it has no compile time. It has no runtime impact,
it's just verified at compile time. If you're a bit curious how it might work, I invite you to
look at it. It works for Fuchsia's NetStack, which has 77 mutex's. And that would be for my presentation,
thanks a lot.
Thank you. The mutex verification, do we need to annotate the mutex's or just a compiler
look at all the mutex's and says, well, this one, does it infer that? Or does it do we have to do it?
It's done manually. So everyone, it works for this project. They have like 77 mutex's. They went
through every one of those, and they defined it an order in a graph. And I think it could be done for
some subsistence, something that needs to be explored. Yeah, I mean, I think I'm thinking about
the giant comment we have at the top of our map.c, which lays out the locking hierarchy inside the
MM, and I'm a little bit scared, but it would be nicer to have that actually verified by the
compiler rather than just a comment that can be ignored.
Other questions? Can you explain why there is no duplicate rule for drivers? Why is there
no duplicate rules for drivers in the kernel? Because you don't want to have duplicate work.
You don't want to have too many people working on different drivers for the same hardware.
So how does that work when you have platforms that are not supported by restaurants and you have
code running means? Someone might want to contribute to driver running restaurants and you
are running a map, or whatever, and it's not supported by restaurants.
Okay, wrong example. Sorry. Try the guess. Try to guess one and support the architecture for PC,
maybe? Some old poor PC? Poor prices with all. M68K is not supported by RIS, which we have
gems here. It's not supported either. DCC is coming. And of course, if you really want to have
you know RIS driver running back you to reinvest in RIS or RIS or RIS code reinvest for your architecture.
All right. Thanks all.