So, hey, I'm Daniel Amada.
Thanks for having me here.
It's the first time I'm ever in the graphics bedroom.
I'm here today to talk about here.
There's a new Rust GPU driver that would have been writing.
And when I say we, there's me, Daniel.
Some two people from Google, there's one lady from arm.
And there's some volunteers as well, contributing code.
So, this is a Rust GPU driver as I just said.
So, what I want to start with is by talking a bit about what is a GPU kernel driver,
right?
And versus what is a user mode GPU driver?
The difference is between the two because usually people get it confused
and they think we're working on the user mode driver
when we are not.
And also, I'd like to begin with a little bit of the story behind this project.
So, basically arm approached us to try and get a Rust driver done.
At that point, they were experimenting with having a driver that was basically have written in C
and then have written in Rust.
And that did not go anywhere.
So, we basically start over from a clean slate with a driver completely written in Rust.
And after that, Google also joined the project.
They're also contributing one or two engineers today.
And the reason why we're using Rust is that turns out that it's fairly easy
to compromise a GPU driver.
And then, by compromising the GPU driver, you take over the entire system.
And then, basically, you can steal the user's personal data, et cetera, et cetera.
So, it seems like the GPU driver is really inoffensive,
but actually it's one of the most targeted attack vectors.
So, during the stock, we have to briefly talk about how GPUs work.
Then we will discuss a little bit how the kernel driver exposes the API to the user space.
We'll talk about a bit about job submission.
And that will conclude the first part of what is this?
And how does this work?
And then the second part is, what is ahead of us?
What's because my point here is to try to try to lay out some plan for the future.
And this is like, what is ahead of us?
How is the prototype looking like?
What's the plan for this year and the next year?
So, again, briefly, how would GPU work?
Again, I'm just simplifying a few things here.
But you're basically trying to have the geometry like this one,
where you can have more complex shapes like this.
And this, textures, which will give a shape,
some appearance of a real-life object.
And then shaders, for example, where their programs running on the GPU,
you're going to compile that down to some machine code that can execute on the hardware.
And then everything is going to be, you know, place inside some buffers in GPU memory.
And like the vertex data, the texture data, your command stream has to be place inside of a buffer object as well,
the shader machine code.
And then, at some point, you basically record your commands into a software queue like a VKQ.
And then that has to be backed by something.
And there's something in this particular case for our molly is tier.
Is the kernel driver, right?
So, for an application like VKQ, for example,
VKQ will be, you know, taking the geometry for a cube, the shaders, everything.
By being that thing to Vulkan, Vulkan will eventually, you know,
build a command buffer, place that command buffer.
I'm sorry, the application will place the command buffer in the VKQ.
And then the driver will take over, basically, after this point.
So, in terms of, you know, how the code is structured, much of the code is actually implemented at the user mode level.
So, when I say user mode level, I'm referring to Mesa for those of you who were here in the talk in the,
to talk to go, basically.
So, mostly Mesa, which is where the shader compiler lives, which is where the Vulkan driver lives,
the jail driver lives.
So, the majority of the actual stack is implemented at the user mode level.
And then the kernel mode driver is just a small layer that will do things that user mode drivers cannot do.
Basically, sharing the hardware between different people who are accessing the GPU, for example,
ensuring isolation, bringing up power, for example, or clock, so things of these nature,
allocating memory, things that the user mode driver cannot do on its own,
because it requires more privilege.
So, as I said, the user mode driver will be implementing the API, like Vulkan or GL,
that will be also compiling the shaders, et cetera, et cetera.
And then, eventually, has to talk to the kernel driver thing, where the kernel driver is basically
making sure that the user mode driver is implementable.
How, by basically, again, allowing for the user mode driver to locate the memory,
providing a way for the user mode driver to access the GPU's ring buffer,
which is where commands are going to be placed in, doing dependency management,
for example, there is a lot of most jobs they will depend on a series of previous jobs,
and then it's up to user space to build as dependency graph, and then tell the kernel about
this dependency graph, such that when it comes at the time to execute a job,
a job is not executed before its dependencies are done.
So, basically, the kernel driver is the one keeping track of this dependency graph,
and making sure that the dependency graph is followed.
Power management, debugging facilities, because when things break,
usually the kernel driver will be able to give you some information
to dev core dump, because it has the privilege of just to do so.
When the GPU breaks, I'm sorry, when the GPU goes down, because I don't know,
it's crashed at the kernel driver is the one that can restart it,
and again, the kernel mode driver is just the bridge between,
like a much larger user mode driver and the overall hardware.
So, for example, just to go into a little bit more detail of what's here is doing.
So, GPUs nowadays, they can provide their own vision of,
they can isolate different contexts using virtual memory through the IOMMU,
much like we do for CPUs, for example.
And this is something that the kernel driver is actually managing.
So, the kernel driver is the one programming the IOMMU in the system,
and basically making sure that one application cannot access the memory
from another application, does it be a memory from another application?
Again, we give the user mode driver the ability to say what he wants,
where he wants things to be mapped and unmapped,
but the kernel mode driver is in charge of programming the actual hardware,
and the page tables.
Synchronization, so basically the kernel mode driver is in charge of saying
of letting user space know when jobs are done.
So, again, so that the user mode driver doesn't really wait forever.
Some resources are shared between multiple users like the GPU driver,
the display driver, et cetera, et cetera.
I've already talked about this.
I've already talked about the dependency graph,
where the user mode driver will be building those dependency graph,
and then tier will be making sure that these dependencies are actually respected.
And we're recovery, again, we've already spoken about this,
bar management.
The kernel is the only thing that can like bring up regulators,
and basically when the device goes idle,
again, it's the job of the kernel mode driver to actually lower a little bit
of the clocks, and that's such as what you don't use as much power,
and unfortunately, tier cannot do any of that in the moment.
But it's blend, also debunked facilities we've talked about this.
So, the API for tier is much smaller, again,
than the API for user space.
Because, again, user space is actually covering a full API like Vulcan,
or OpenGL, but tier is basically just this.
These are the main APIs that we have.
So, creating and allocating GPU memory is the top row.
Enforcing isolation between different applications using virtual memory regions,
which is what the VM create and VM bind I often do.
And then giving user space access to the GPUs ring buffer
by creating an execution context.
So, basically allocating GPU memory,
a job submission, and enforcing isolation.
So, here is the overall picture for what a GPU kernel mode driver looks like.
We have the UAPI which is communicating with user space.
And then, again, as I said, scheduling jobs is one of the major responsibilities.
So, we have a component, a shared component between multiple drivers
in DRM called the DRM GPU scheduler, which is doing this dependency tracking thing.
We've been talking about there's one component, one shared component called DRM GPU VM,
which is doing this isolation between multiple applications that we've been talking about.
Allocating GPU memory is done through this gem infrastructure within DRM.
And then, below that, we have the actual hardware.
We have the VRAM, we have the firmware scheduler,
we're going to be talking a little bit more about the firmware scheduler,
and then the actual course.
And on top of that, our Mali has its own firmware assisted jobs scheduling system,
basically inside of it.
And that's basically implemented through a microcontroller unit.
So, this microcontroller, basically in tier, we never really interact with any of the course.
We talked to this microcontroller unit through a shared memory region,
and then in this microcontroller unit, we can basically allocate a ring buffer
where we can place command buffers in, and then we can tell the microcontroller unit to execute that.
At which point it will take over, and then tell tier when the work is done.
So, again, if we go back to Vulcan, for example, we have command buffers being recorded into Qs.
Then tier will back these Qs with what we call CFF groups,
which is just one view into the GPUs ring buffer, basically.
And then eventually once you place your command buffers inside of this ring buffer,
the microcontroller will take over, the work will eventually get scheduled inside of the ring buffer,
and then the firmware will eventually raise an interrupt to tell you, hey, this job is done.
At which point you can tell user space, because again, the kernel mode driver is the one that has to tell user space when work has completed.
So, this is basically what a kernel driver does.
And this is not a specific to rust, but to all general kernel drivers.
So, where are we with the rust driver and particular?
And I don't know how many of you have seen my presentation at XDC, where I presented some to the three months ago,
and basically back then we didn't have a functional prototype, but now we have,
and this thing is actually running tier.
So, this is being rendered using the rust driver at this moment, so it's running, it hasn't crashed so far.
We ran this thing for three days during plumber,
Linux Plumber's Conference in December, and we had, so there's this board here.
I don't know how many of you can see, I'm not going to touch it, because imaging sits.
So, we had controllers, actually let's plug it into the board, and then people were playing supertips cart, you know, the cart game,
and people were playing against each other for three days and they didn't crash.
So, at this point, I'm somewhat, I believe somewhat that this thing isn't going to crash on us, and he hasn't so far.
So, basically, we have a prototype, but yeah, what now, right?
We have two different strategies at the moment, so we have this thing downstream, which we basically wanted to ensure and tell the community
that, hey, we can get a rust driver done that can be, as before, and as a C driver,
and we wanted to show it with, you know, we wanted to show people something real.
So, we did this downstream driver, we, again, we demoed it at plumber, et cetera, et cetera.
We didn't do enough benchmarking at this point, because there's a lot of moving targets on the table,
but we ran a few games and the performance between tier and the C driver called Panther is roughly similar.
And by roughly similar, what I'm trying to say is you get roughly the same amount of frames,
but we did not do actual benchmarking, which would be the right thing to do, but we're going to get there in the future.
And then we have the upstream effort, which is basically the most important thing at this moment, right?
Because it's no good to have a simple downstream driver like this.
Well, we want to do is to have this thing basically be a part of the Linux kernel.
And for that, we will have to figure out how to would stream what we currently have inside of the kernel.
And as I've been saying, I've explained this a couple of slides ago, writing a GPU kernel driver requires a lot of shared components.
I told you guys not when we were discussing GPU VM, the GPU scheduler in the picture that had the basically tier in a nutshell, something like that.
And we need to have abstractions for the shared components for all drivers, not only tier.
And they have to work, of course, for all drivers. So in particular nowadays we have Nova, which is, you know, been written by the NVIDIA people, which is a massive driver for NVIDIA hardware, which is going to support tiering and up.
So I guess, Turing starts from 20XX series of NVIDIA GPUs, if I'm not mistaken. So Nova is going to support that and up.
So a lot of hardware, a very complex driver, there's also a Zahi, who was initially written by Azahilina, right?
And nowadays, the Azahil people have taken over the project because you left.
And they're also trying to upstream the driver. So there's basically three of us, and RKVM, RVKMS, which is a mode setting driver, also written in Russ. So there's four drivers.
And they're all going to use, you know, these shared infrastructure.
So our plan initially was, let's get this infrastructure up, same, right? Because if we don't have the infrastructure, there's going to be shared by everybody.
We cannot have our driver. It's a simple as that. So we basically have most of the DRM stuff upstream, so you can have, you can have a DRM device, a rendering node, show up.
We have clocks and regulators so that you can bring the GPU hardware up because you need power and clocks signal for that.
We need RKVM as I told you before, we need RKVM used to know when jobs are done, for example.
And it's the main way through which we communicate with the microcontroller is through interrupts.
And we also need work used for a lot of reasons.
All of these things are downstream, unfortunately. And this is a problem, because for as long as these things are downstream, we cannot upstream or work because it works depends on this.
So allocating GPU memory nowadays is not possible in the upstream Linux kernel.
The reason being does gym, SHM, there's no way to communicate with that in Rust today in an upstream code.
If there's no way to communicate with the gym framework, again, there's no way to allocate GPU memory.
If there's no way to allocate GPU memory, as I said, allocating GPU memory is basically the number one thing that a criminal driver does, you can't do anything.
So to answer a question that somebody may or may not have, where are we in upstream, we can basically probe the device and bring the power up basically.
And not much more than that.
Reason being, again, we cannot allocate memory.
We cannot talk to the IOMM use, so we cannot provide the isolation thing.
I've been talking about, we cannot have, we cannot have fences.
Again, fences is this synchronization mechanism where you can basically wait on a fence.
There's only two operations, waiting and signaling.
And it's how the kernel driver tells us these are space, for example, that a given job has finished executing.
It's how you build this dependency graphs.
I've been talking about where you say, hey, this job wants to complete, it's going to give you one fence.
And then this other job will depend on this fence, so it's how you build these dependency graphs.
And we cannot allocate fences in tier because there's no way to talk to the current C code that does dot in Rust.
And this GPU scheduler, again, one of the major responsibilities of a kind of driver is scheduling jobs.
Again, by noticing which job has its dependencies met, such that this job can execute at the moment.
This is done by this GPU scheduler component.
There's no way to talk to the staying upstream at the moment.
So there's very little that can be done, however, work tackly, all of these things.
So, viewed Paul from Red Hat is working on the first bullet point.
And I'm pretty sure his series is going to be merged soon.
So soon we're going to get the first one.
Second one is probably going to be merged in the next cycle.
And then again, there's a person, Philip Stoner from Red Hat, who's working on the scheduler and the fence stuff.
And just last week he put out a RFC.
I'm not really sure how this RFC is going to perform.
And it's not really your M GPU scheduler, but a new thing, but we're going to talk a little bit about this.
But the good news is we have a prototype driver, so we can just take whatever code anybody puts out.
And then backport that into the prototype and then test it out on a real device to see how it performs.
Because drivers like Nova, for example, do you're not at a point where they can test a scheduler?
Because they cannot submit jobs yet, because they did not get there yet.
And for the prototype, which is the staying, I mean, I used to have these pictures there before I could present from the actual device.
So I don't think pictures are really needed because we're seeing it being presented.
We can basically watch YouTube play games.
So yeah, gnome and Western are working.
And in fact, let's just see whether we can run something like supertickscart.
And hey, supertickscart is working.
This GPU thingy at zero means that we don't have support for performance counters, which we did not get there yet.
This thing, basically, we're actually only 50% sure that this thing will actually work until the end of the presentations because a lot of things to do is still.
And you can play.
Yeah, whatever.
And yeah, there you go.
You can basically play.
And again, people played this for a few hours during plumbers.
Um, how do I quit this?
And then if we do, like, demas, grab tier.
Tier is basically outputting a lot of things.
Basically, all mappings have traces on them for debugging reasons.
So every time you map and map memory, you bring something to the console.
Let's see if VKQ is actually working.
So yes, it's running on Molly, just 16.
And Vulcan is working, albeit with some glitches because as we will discuss, there's a lot of shortcuts there.
So if you run VKQ on a actual, you know, how do I say this?
On the driver that has actually been deployed, let's put it this way.
It doesn't stutter like this.
This is where the synchronization is in 100% okay yet.
We have taken some shortcuts to get this to work.
But it works.
I mean, I couldn't use that as my daily driver for email.
And, you know, I, as long as I saved all my work.
After, like, every three or five minutes, let's, uh, yes.
All right.
So gnome is working.
VKQ is working.
Super tics cart is working.
This can we run Firefox?
No?
Yeah.
So Firefox is working.
All right.
So let's get back to the presentation.
So anyways, as I said, some parts will need more iterations before being ready.
As you can obviously see, there's a few things that this thing does not have.
For example, it doesn't have any power management code whatsoever.
So the power management strategy for this thing is probe the driver,
clock the GPU to the max clock frequency that you can, and then just leave it there.
So what it was is not good enough, like, as I said, synchronization is not good enough.
So I'm not going to be showing those, well, like, if you run super tics cart in windowed mode,
it's actually glitchy and there's, uh, a few, um, how do I say that?
Yeah, well, glitches.
As you see, red and two different colors popping up every now and then, um,
what I was just not here, um, oh, um, error recovery.
So if the GPU crashes in this thing, you're, you're done.
You have to review the computer.
Usually, when the GPU crashes, the first thing you have to do is, hey, is this crash recoverable?
Because some crashes are recoverable.
And if they are, then you should rewind the state to, you know,
you should basically recover and put the GPU in a state where the user can resume, uh,
using the, the hardware.
But this is not the case here.
So if this crash has a crash has, and then you reboot the system and bye bye for all of your work.
Let me see if I can remember something that is not implemented yet.
Oh, um, you can only run H programs at the same time.
For reasons.
No, really, um, I said that the microcontroller can schedule jobs automatically.
But that's only if you give it up to eight jobs, right?
If you give it eight jobs, like from zero to seven jobs,
it can pick between these eight and automatically scheduled them.
But if you have more than eight, you have to have a software schedule on top,
which is more code that is not as nice as nice and shiny to show people.
So it's not there yet.
Um, but what do we want to stay for?
As we said, going forward now,
we have a driver where we can test the code that other people from,
from NVIDIA, from Red Hat, or any other contributor,
we can test their code in a real device, right?
Uh, we can basically backport their code into on top of our prototype and then run it and then collect data
as does working, as does before meant or whatever.
And if it works through tier, there's a good chance it's going to work for Azahi and for, um, and video as well.
So again, how can we, you know, get this prototype and upstream it and, you know,
get rid of the shortcuts, that's where we are at the moment.
So we are focusing basically upstream now.
We're going to get a lot of the code that we have on the stain.
And then we're going to see you, hey, where's, um, where do we take any shortcuts or what can we improve?
Or can we design the Rust API to actually be better or safer,
or more performing, et cetera, et cetera.
So this is where we are.
There's actually a big discussion nowadays about how
job submission is going to look like.
Again, job submission is, let's say, one of the major responsibilities of a criminal or
driver.
And we are, this thing is using the GPU scheduler.
Uh, the DRM GPU scheduler.
That's written in C, however, as I said, this thing has a firmware.
Uh, a microcontroller unit that can automatically schedule jobs up to age jobs.
And then it doesn't really make sense for you to have this direct GPU
DRM scheduler on top.
And this is also the same thing for Azahi and the same thing for Nova.
So it's looking like we're going to be writing a new scheduler directly and Rust for,
for job submission, which is not going to schedule anything because as we said,
the GPUs nowadays, they can do that on their own.
It's only going to do the dependency management part, right?
So figuring out whether the dependencies are met before executing work.
And Red Hat is basically working, or spearheading does effort.
They're calling it the job queue.
And we're working together with them to help them test it, give them input on the design.
And so on and so forth.
So as I said, we're going to start with a clean slate.
Try to upstream whatever you have here.
Try to upstream whatever the dependencies are.
Still, not upstream in order to get a driver upstream.
And then we're going to go, uh, focus some effort on benchmarking.
Basically.
So how performant is this thing compared to the C driver?
We're in a couple of games, but we need more data than that.
We're going to run CTS and ensure that CTS is passing, you know, that sort of thing.
On the upstream driver as we get there.
So basically, this is what I had to show for today.
It's sorry I did not bring the controllers, otherwise your guys could play for 10 minutes.
It'd be cool.
Do you have any questions?
Which talk are you using in your demo?
Arcade 3588.
The question was, which SLC we're using for the demo.
And it's the Arcade 3588.
Hi.
Thank you for your talk.
And you said that you don't need scheduling or up to eight jobs,
but then you need some kind of scheduling.
And then you said, why?
Let's get early in front of the stuff.
Let's get you.
I won't choose by that.
All right.
So CSF will give you eight.
So sorry.
So the question was, the person was confused because I said that the GPU can schedule up to eight jobs.
But then we need the software schedule on top.
And I told that the job queue is not going to be scheduling anything.
So there was a confusion in all of these different information.
So to answer your question, CSF, which is this firmware assisted scheduling thing,
will it give you up to eight slots where you can basically,
you have eight ring buffers where you can place your command streams in there.
And then it can automatically schedule up to these eight ring buffers.
If you have more than eight jobs trying to use the GPU simultaneously,
you have two options.
Either you say, hey, the GPU is busy, which is what this is doing.
So just return ebusy or you boot someone out.
You stop the world and have a look, hey, who's idle or if nobody's idle,
who can I remove from the ring buffer at this moment to free up one slot to give it to another
job basically.
So this is what I'm referring when I say software scheduler.
It's this component.
Now this component is really only for tier and panther,
because for Azai and for Nova,
they basically can have as many ring buffers as they want.
It depends on the amount of memory, but there's no restriction.
No eight or 16 is lots.
So when I say that this job queue thing is not going to be doing any scheduling,
this is what I mean.
If you're driver, for example,
needs to have any extra scheduling,
because you only have a limit at amount of ring buffers,
then you have to do that on top.
And so far, this is only the case for tier.
Any other questions?
I have a more general question.
You have some of that.
In GPU memory allocation of the course,
don't worry.
But in a scope, in a scope,
modern arm chips, which have tasks of CPUs,
CPUs, GPUs, GPUs,
and CPU core for GPU and stuff like this.
And surrounded by high-binded memory, share memory.
Do we really need to allocate if this memory is heavy to share
it with all types of chip?
Well, all types of course.
So the question was in correct me if I'm wrong,
given that modern SLCs have a lot of memory that's shared
between a lot of components.
Do we really need to allocate memory?
The answer is yes.
So basically,
you need to allocate a portion of the system's memory
to the GPU.
And this is what Jim is doing.
So Jim is going to use the SH,
SHM, FS layer, or anyways,
to actually do the allocation from you.
And it's going to carve out memory from the system
overall memory.
So yes.
I question about crucial routes.
Did you do it only because the memory is empty,
or you're looking to get some components?
And if it's even possible in a framework.
Oh, sorry.
So the question is, are we doing the standard routes
only because of the memory safety?
Or are we also looking to get more performance in rust?
So performance is actually a known goal.
More performance is actually a known goal.
What we're trying to do is to get as much performance as the C driver.
And the reason why is,
it's basically not a goal of the language to be faster
than C in general.
And even if the language is faster,
having a faster kernel driver wouldn't necessarily make it
faster overall.
Because the kernel driver is not really the bottleneck
most of the time.
So having the kernel driver execute faster wouldn't necessarily
need to games executing much faster.
So no, we're only doing it for the safety.
Yes.
You said something about the developer and the as I drive up.
If there anything you heard from people from AMD to you or Intel,
I mean they have to print what's the biggest price of the kernel.
I think that did they show any interest in looking at rust?
Has AMD show any interest in looking at rust?
Or in looking at rust?
Well, no, as far as I'm aware.
I haven't heard anything.
That's my answer.
Thank you.
Yes?
The mentioned feature.
Something from here.
These are implemented in PAN4, I guess?
Yes.
Is there anything else that implemented in PAN4?
Which is one to have feature,
but not master feature,
but implemented in here?
Yes.
The performance comes as I guess?
Yeah, a lot of things.
In tier, basically, in the downstream prototype,
we have the bare minimum.
The question is, is there more things that's implemented in PAN4?
That's not implemented in tier.
So, a lot of things, basically.
We only have the bare minimum here to submit jobs and, you know,
build the firmware and have jobs execute.
So, performance counters, bar and management.
I think the bug facilities, error recovery,
support for more GPU models.
So, there's only supports Molly, just 16.
We're a PAN4 supports other GPUs and other architectures.
So, there's a lot of missing at this moment.
More questions?
Yes.
You know, don't you stop this, PAN4, I guess.
So, I could just pull your budget drive up,
fill it and drive on my own,
RK-8 or RK-8.
Can I, the question was, can I download the driver
and test it on my own, RK-358?
Yes, but don't blame me.
It takes back.
No, yeah, it's public.
But don't do your most important work and then have a crash.
Do you think there's any more questions?
You mentioned the video playback though.
It's a bit of a blind spot for me.
Is that like an entirely separate set of BDIs
of the driver from our virtualeration?
Or is it a different driver?
Apparently, was it the same buffer objects as everything else?
All right, so the, the question is,
how is and correct me from our own,
how is, how is video encoding the code implemented at the driver?
And basically, I don't think arm has any encoder decode engines
in the GPU.
So it's totally separate from the driver.
More questions?
I have a question.
I know it's very close.
You're working on the who's funding it and why?
Because that's all the things I know about.
Oh, yes.
So, who's funding the work and why?
That was the question.
So, basically, arm and Google at this point.
And reason is, basically, as I said, security.
So, making sure that people cannot hack devices.
They may otherwise sell, for example.
And eventually, we plan on, you know,
so there's a seed driver, which is Panther,
and then there's tier.
And eventually, in a few years, if everything works out,
the overall plan might be to move the platform from,
from plant or to tier.
And then, you know, to develop near features there.
This is still too much in the future too, actually, discuss.
But that's the general idea.
Requestions?
Hi.
I mean, that's some stuff that's not ready yet,
because there's no, like, gross funding for the figure you're looking at.
Is it, like, painful to implement?
So, yes, it's, is it quality?
Is it the technical problem?
Or is there a problem with more rust?
Yes.
The question was about the abstractions,
and whether it's the, I said there are things that are missing,
and whether they're hard to implement, or what the problem actually is.
Correct?
They're not hard to implement, but we need community consensus, actually.
So, that's the hard part.
And the number one roadblock, because when I had this slide saying,
what we have versus what we don't have in terms of the abstractions,
I said the most of them are going to be upstream soon,
except for the job submission logic.
And this is where this job queue stuff, replacing the GPU scheduler,
and doing versus not doing scheduling, et cetera, et cetera.
It's something that has to, this, we have to talk a lot about this,
because it has to work for everybody for all drivers.
And getting it to work, and getting it to work correctly,
I mean, we already have this scheduler in C,
and it's plagued with some issues.
That nobody has managed to fix for seven years.
And we don't want to repeat that just so that we have something that works.
So that's, yeah.
And fences are usually the same where it's like a house of cards.
So if you have a bug somewhere, everything comes crashing down,
so you have to be really careful there.
More questions?
Hi.
Have you ever destroyed hardware, Bob?
Thankfully not, have you ever destroyed hardware while doing this?
Thankfully not, not yet.
Hopefully never, right?
More questions?
Yes.
Malay is like, why did you go to an Android?
And I'm going to use the S-turn over for you.
We have plans to back work on this or,
but I have to wait a few years before we can move on to our next news.
The question is, what are the plans for Android?
Basically, right?
I'm not sure I'm allowed to discuss this.
There may or may not be plans.
Yeah.
And it may or may not be soon, ish.
All right.
Yes.
Any more questions?
No?
All right.
Thanks for having me.
Hopefully you guys enjoyed it.