All right everyone, Jen Looker here is going to be talking about
porting GGML to Nux Kernel Development Frameworks.
Are we ready?
Nope.
Just staring around.
There we go.
All right everyone.
I give Jen Looker a round of applause.
Okay everyone, my name is Jen Looker.
The title will make sense as I go along.
About me, while the older I get, the harder it gets to say what my specialization is.
I've been working very skills, but whatever I do, I tend to naturally lift towards
the border between software and hardware, it might do.
I'm constantly working for a RIFI startup, but it's employers, I express my collective
personal way to express my collective madness to the lockdown was building analog synthesizers.
Well, it is my personal project, so I has nothing to do with my past or current employer.
And so about this talk.
So this talk is mostly actually about how to run GGML on a constrain environment.
And by the title, I guess your question is what is Nux, and that's the first things I
will try to answer.
And then once I get what Nux says, like hardware and which is a constrain environment to run
on better.
The kernel, the Nux Kernel is a kind of space component, so you know, it handles interrupt
exception and requests from the user space, and there's a proper user space, but usually
you can, you know, you essentially have what you do with people called program.
And yes, the request of this call and the model that I have at the start from any web
notes is that you just have one user space learning, but of course the kernel can support
creating new software, new programs, new space and go.
Right.
So what's our care looks like?
Our care is again, I want to make it portable, so the way I did it, I made it an alphabet
loader, so health is the binary system, and the two were away to see a Nux.
One is the link view of a NL, which is based on segment, but then there's the loader
way to the loader view of a NL, which is based on program ahead of.
So what I do is that when you load the kernel, there was special program methods that
tells where the kernel is, one's things like the frame buffer or the information page or
the memory map and other data, so it's completely separate as a way, but in this also
very portable, so what I had is our call library, and then I have various machine-dependent
platform-dependent functions, and then you can just like the call plus one of these, you
know, gray boxes, which can move to NFI or to booth on SPI, and right now targeting mostly
it is 564, and it is exported to N64, big, so the second part, so after I load the
bootloader, I do have a kernel, the kernel, what is a kernel?
Well, in the most abstract way, a kernel is executable, that is loaded by the bootloader,
and essentially reacts to various events that happen in the system.
So in order to achieve that and making that simple as possible to build a new kernel, because
there was my goal, I have, of course, like two libraries that are dependent on the platform,
one is the hull, which is up to a station layer, but it's mostly meant to boost up the
system, so I've got essentially jumps into the beginning of the, of lib hull, and then
essentially abstract the CPU away, and then there's the platform, the platform is the part
that controls mostly, the intercontroller and now, and provides a timer, because this is
usually a fundamental part, again, I suppose, it's 36, 32-bit and 64-bit, and this five in
both of them, and mostly, so for example, the SIPI version for this five is coming, but
it's not ready yet.
And then, of course, I have our portal, lib hull, which is the interface, which usually
the custom kernel code that you write interface with, and, you know, it provides all the
stuff, which you need when you programming a kernel, like, you know, memory-allocates of
mapping, you know, they use a stack frame to switch threads, and, of course, the page table
to switch page table and global, you know, the global memory currency, and, you know, panics
which is what you mostly run when you run a your own kernel, okay, so what is like to be
the kernel for people interested in, in nox?
Well, the way I see it, as I before, essentially, is a kernel code that is a seed programming
in this case, but you can be voted to new things, the essential reacts to various events.
One is the timer, the other is the intercontroller exceptions, the Q-sup programming device,
which is a seed-school, and, essentially, in order to write kernel under nox, you just
need to implement these functions, and, there you go, and then you can just boot and put
in things.
Of course, having a kernel without a user space, I can call the user devices over a little
news, although this demo is all about this.
And so, I also provide a way to have a standard way to program the, at least the beginning
of our user program, and so, essentially, there's a lip nox user that just ups out the
C-scopes mechanism that a kernel use, and, yeah, that's actually what I'm doing, yes,
as I said earlier, there's only one user space at the beginning, and, yeah, and so, yes,
of course, as I said, like if you want to run multiple programs, the kernel, of course,
need to have a four-core anexic or as powerful whatever you need.
Right, this is important, despite the minor details, because it would be important for the
test.
The talk, there's an anxangiros, it's a call it, and it's a lip, and by the C, that is a C-lab
rate that I call it together, mostly taking bits of net BSD, and right in the stuff in
a simple way.
It's simply a very small lip C, because in order to create a binary, usually, during
the C-run time, which is certainly C, or mostly, and then C, at the end for the various
constructor, I live C, and then the C. So, you know, you have a main function, the main function,
actually, is called by the C, at the C, or the C, or T in general, and then usually have
the various function, like the interface, and stuff that are part of the lip C. And so,
this is the part that you need to create a binary, and if you go back, everything goes
things that I've explained, from this, from the user space, to the kernel, to the arcade,
itself, the whole binary. So, this is, this is the call, this is really the call,
actually, what makes noxveres it bought.
Right. So, another we have, this idea of what noxes, how do I, actually, run GGML on it?
So, let's start a look at what GGML is, at least in my view, so in the most simple way.
Right. The way I think personally, in GGML, not definitely an expert, is that, in order
to build the most minor, minor minimal GGML, I have built, essentially, you can look at
this components, I don't know how to call, but this is how I abstract them. For one day,
there's definitely a part of my abstraction, which is something that, you know, like GGML's
time, or GGML, all of the functions that actually abstract the actual low level call,
then there is utility functions, which are, you know, there's a lot of functions to do
file I here, also for GGML, for dot, for our files of loading models and things like this.
And then there's the function when you actually build the model, so you do, you build the
graph, then there's a table implementation that is the one that actually start programming,
running the compute task, then there's the GGML's function, and this is the, I'm not talking
about, I'm just talking about, you know, CPU executions, so I'm not talking about back-end
stuff like this, but in general, this is the model that I have for GGML and that I'm going to use,
you know, right? So let's have a look at the software side of it, what are the dependencies?
Well, GGML is written in C and at least this part of it, in C, and I'm very minimal set of C
plus plus, luckily. And so usually this is what you get when you try to compile GGML,
there is some standard C, like there's definitely a lib C, that's of course shockingly,
mathematical operations, and usually the lib C doesn't implement this floating point of
relation, that's sort of standard function called lib M, you need, for the C plus plus part,
you need usually like standard C plus plus library and the C plus plus one time,
uses the title, uses P2, and actually as a cell, I will say later, it's actually quite difficult
to separate both of them, and then GGML, GGML, time, uses a POSIC, so, you know, it's a more unique
specific, of course, to do things, and it's like locked monotonic, just back to the POSIC standard.
The venues that I had, that made this so possible with these, were as the fact that the subset
of the C plus plus standard library is very small, is mostly back to the narrator's end,
you know, going to get the crazy stuff, that C plus plus can give to programmers.
Right, so what does it take to port GML to nugs? Let's have a look again, another, you know,
how the software of nugs is architected, what do you get when you're actually running a system
under nugs, so after you've rooted? And, well, we support the MPs, so we have multiple CPUs,
and in modern architectures after the 361 will say, the focus is mostly on chupy-vierage level.
One is the kernel, and the other is the user. And, as I said,
when we put on the nugs, when I calculate, when I calculate this, nugs will create,
we will have like a kernel image, and I usually match on each CPU, each being the same.
And, of course, you can run different code for HPP, you can specialise it, but like at the beginning,
you give this to welfare, you will be sure that HPP has the same alpha mark everywhere.
Right, so, when I found myself in today's, and I got GML link and say, how would I like to run?
And, this layout is increasingly flexible, because you can do everything, and so, I wanted to create
a compute platform, because I just wanted to say, okay, let me see what I can get, if I can run today.
So, I decided that I actually like the idea to dedicate entire CPU that runs and interrupted,
inside the system, so, instead of having threads that get created, and so, locally, I can just simply
dedicate entire CPU, ask compute threads. And then, of course, it would be very useful,
as if you have this machine that boots and start doing this calculation without the ability
to communicate or to the saved things. So, I can use the bootstep CPU, which
most of the time, it's easier to just, you know, the speed that start in the system to have a
system interface. It's actually at the most minimal, as you can think of, or whatever you want to run.
Unicunners will be a good idea, because essentially, the just use one thing.
And so, this is the plan, this is the model that I want to, for this approach, for time to get
some sense into notes. Right, how do you, and so, here's the thing. So, I'm going to look at
a compute CPU, as it said, we have kernel and user space, and there are two ways to run things.
One, I can just directly run GGLML, and whatever, in the tech computing vector in kernel mode,
so in highest, in the higher privileged mode, which in the nox model, actually that means that
interrupt will not interrupt, you will not interrupt it ever, not even by interrupt. All, I can do
something which is, you know, more say, and usually, which is like a kernel that is just a compute
support, and then the GGLML computer is running in user space, so that you can request stuff, and you
can just, but it would be interrupted. So, usually, the reason to run things in user space is
because there's a corruption in the user space mode, usually the kernel is not affected, so the
sub-each of the system is not affected. The problem is, of course, that it's also easy to put libraries,
because you can just compile way more stuff, and having the compute support to implement the
stuff needed, there's a possibility that by, by interrupting the stuff and also fact that,
you know, that there's some privileged operation, you need to do the system, back and forth,
and this was the demo, so I decided to go for the first option, and so running everything in the
kernel and see what happens. So, this is what end when the, what the software's fact looks like,
we got GGLML first, of course, we need to have the Liban, because let's see, we give them courage.
I'm going to use Libnu's compute is the first library that I used there was on top of LibiC,
which is essentially the part that allows me to allocate thread just by scheduling a CPU,
so the CPU are waiting, and when I'm in the thread, essentially the needs to compute with
set to a CPU, okay, now start, and of course, it stops and implemented the whole condition
waiting for the various CPUs, so essentially it kind of emulates the thread tool, and then
I have to write something that is awful to look at, but it works, which is LibgGLML UX,
which is well, the dirty work is done, and so everything that actually needed that was missing
is both there, so one is the C++ on time, which is like some sector completely function that
we need to implement for the C++ code compiles to run, okay, yes, and yes, so you need that,
and then you need, if see, essentially, to LibiC, because LibiC is minimal, so you know,
sometimes you need, sometimes you need like Q source, and stuff like this in GGLML,
and so I said, okay, instead of adding to LibiC, I'm just going to add another library that just
includes next user headers, and so I did that, and only sanction one there, and then, yes,
and so of course, I have to map it side to no compute, and I have to implement it in GGLML's time
directly to use the Libnux timer, so putting all together, this is what this has been my,
what I did, this is our system looks like right now, so right now, in order to test this,
I, as you can see, I have all the GGLML running in the CPU, in the compute part,
and I needed a quick demo, so I took example GPT2, and of course, the model is loaded in memory
shed, I could just land directly, booting from hardware, yeah, you can find the code there,
is originally, I was trying to put last directly to nuts, then I said, okay, you know what,
I can actually do directly by implementing the tent pool, it's going to be simpler, and it was,
and yeah, so it's a prototype, it's just, I wanted to say, look, we can do it, which was my goal,
and also to learn what it takes to run GGLML on a very minimal environment, what you need to,
to look at the various dependencies in order to understand it. So yeah, the code is there,
it's for anyone who wants to read it, and the final consideration, well, honestly,
porting it was much, much easier than expected, when you, when you see as it was last code,
usually you do not expect that to see that, but actually, it actually is very saying, and
it's, it's, it's, it's still very, very valuable to port in various different environment,
the only thing that I would, I would do is like simple modification, but unfortunately,
they're in two, the core architectures like, for example, GGLML, C contains everything from
file I owe to various plus or plus section definition, and probably separating different file would
be very nice, it was very impossible, very difficult, so that's why I had to learn the
pizza to do the various, to implement the type of, it would be very nice, if the type of
would just be linked in a separate file, it would just change it, and yeah, and again,
like the plus one section is defining GGLML, C, and I would need to do some ifs to define
all those things like the posit head interface, and that's pretty much it, this is where you
can find more information about it, it really was!