Hello, my name is John Luca Guida and there's a reason that's also a backstory and I
will be sure to tell you all about it and then of course some of us know Mark has been
personally a personal session of mine for a few decades but I'm going to attempt at having
got brief introduction to Mark which is impossible because it's extremely complicated and then I'm
going to go and say how it implemented why I discovered the YB4 but like what is the principle
between the implementation and I can you go towards implementing something like this and then
the lesson learned okay let's start so the backstory this is not my first time and I present
to this bedroom I think it was in 2016 and 2018 and presented my macocerna which was called
Moujahak which was all John interface exported was such as the vice diver that you can see the
presentation for the reason why and after 2018 I realized that I needed to get rid of some code I was
emotionally attached to which was mostly 32 bit and it is 6 and it okay I actually need to
port it to modern hardware and so I started this project called nooks so nooks became in the end like
I cannot put a type in framework so it's something that is extremely portable and it's meant to
allow me to put a type kernel which is something that I do in my spare time when I don't do synthesizers so
okay the original motivation was that and when I designed this framework first of all I make it
portable so I now it works in I36 and this 64 it is 564 and originally was meant to run Moujahak
which was a very simple kernel I made it extremely simple for a reason I didn't believe in the mix
between fork and threads so I made it just I implemented forks I didn't implement user threads
and nooks essentially has this implementation if you're interested in about nooks tomorrow I will
have a talk in the eye dev room and I will have to explain more in details over there why I did or
do the architecture of nooks so if you're interested in that we should check that talk.
Now yes so I essentially ported Moujahak to nooks it was quite easy because essentially it was the
same underlying assumption about what a kernel should be and yes and now you know as a result
I still haven't published a code but I got Moujahak for anyone who's interested that works in 64
bit but in Intel and this 5. The problem with data is that I wanted to kind of publish nooks
as a kernel development framework just to help people that just want to. It's an extremely simple
to get a very simple kernel in nooks again tomorrow so it could be more about it essentially you
define a mine a main function and so essentially entry function which is essentially like a event model
it's a simple C file and you get a kernel that boots on SMP on modern hardware. I test it up to
dual Z on with 40 calls so the problem is I wanted to test and test nooks to see how much can
actually goes and really help me to make a big couple of kernels and I briefly consider doing
a unique system far in our unique classical unique system but I say you know what there's
something that is more complicated than a unique kernel way more complicated and that is mark
and so why mark is more complicated so I want for the hardest thing right and well the mark
normal implementation is I heavily use of kernel threads and in nooks I do not support kernel
threads so I wanted to see can I actually statement you will know idea when you'll get out of it right
like pages and pages of mysterious locking out so I actually was very curious why is it written
like this it's that the only way can be written you know I lack you feminism and so I shouldn't
probably say what I think and also there's another reason that I realized in 2025 is the 20th anniversary
of my stomach branch which was a clone of new mark that I presented in 2005 actually I think
it was November 2005 the stomach sorry I had a hard meeting in Madrid and essentially it was
a bit of a crazy stuff but essentially I wanted I tried to solve the problem of the lack of
diversity in the new herd so I plotted a common interface so I could use a skate inside the new
herd instead of doing what I was doing but anyway this is actually history now so there's a personal
and I'm finished business with Mark so when I wanted to to bought Mark I was thinking how can
I do this? Mark was famous for being portable so there's a very well well-defined interface
to implement to port noks to import Mark to an architecture and though I could have ported
noks as an architectural part and the only thing that I actually needed to do which was very weird
was to implement canal threads in noks which I can do but felt a bit unnatural and then there was
the hardest way the one where I can actually implement it from scratch it's a bit insane but the
two of this had enough personal curiosity about Mark and why was it in this way and all the things
I said before I said you know what actually I really like understanding things by implementing it so
it's a bit insane but I could probably try that that I repeat like this is the question that I had
those are completely hard to be complicated and difficult to read and okay yes we all know that Mark
was a pioneer in many modern or I said yes what kind of choice it would have been made today
it's it's much easier after you implement to think about them because you have to solve the problems
solved it's of course I chose this now I'm going to try to I can one cool talk for
hours about this so by brief I need to be I'm going to skip a lot of details I'm going to simply
fire a lot and if you say that I say something wrong I'm probably just simplifying over it just
pretty wrong right so what Mark was famous for it was the first thing that was it was very
famous for this IPC everything was a message pass and it's not a message passing as you find
modern kernels so in the in the documentation they say that they they push for a client server
architecture except in the code they call it a user server architecture which becomes complicated
because there are three ways to kind of messages the first is user space to use a space or you have
a server user space and a user user space you send messages but you can have a kernel server and
so the kernel server the user send a message to the kernel which that's how you get that's how you get
services but of course there's the most annoying definition ever which is kernel user which
of course it means that the kernel request services or send notification to our user and so the server
is in user space and the user is in kernel space. Anyway once you understand that everything
sort of fine but in my make my make branch I change I also allow kernel user to be the
fine as kind of client. Now write and send once is it simple no it's way more complicated than that
because the send write are actually counted the send count can only be one and when I send a
message the easier way to think it's like when I send a message essentially I am giving the right
to the server back and so the left count change well for second clone the right so it's it gets
a bit more complicated but this is the idea and then there's a thing called port sets which are
very annoying to implement in which essentially I can collect a lot of port lights together and so
with a single receiver I can see from multiple ports. So this is when I said that the appc
is nice but this is how it becomes and so you can imagine that then you can have port rights for
kernels services the kernels doesn't really care about rights to use other stuff like this.
Now let's get more complicated shall we? Mark of course introduce our interface definition language
to just call make and of course you can use it on server code again think use it as client
and yeah so these are the works I mean this distance quite say and actually there's an interface
definition there's a musical and some may mark into make generates from the interface definition
I use a code and a server code and so you know like I call it from here and the interface
goes on the other side so whatever I link the server code that's what I'm going to get to think
of course think about this when I have a kernel server the server co-generated by make will
be compiling kernel when I have a kernel user code the user co-generated by make which is a
tool user space tool will be compiled in kernel as well. Now did I say that is oh yeah so I
miss some very important things here kernel defined message so essentially like the message that
the make generates because essentially from kernel code I need to to create a message that then
get sent the the message needs to be known as specific kernel kind of dependent function
why is that because the kernel might need to pass the message so messages are typed in mark
and so for example the message adder will we will link to the port the port write ID of the
sender and possibly other play port and then the value stipes now the two types of messages that
I can send so you know minimum minimum case scenario the kernel when you save a message only need to
pass the header but there's this thing called complex messages and it does need to be passed
all of them even the body of the message because they can contain other range I can just say okay
just move this code copy this code to this process and I've touched this memory to the messages
I can send port write this is our right help it is essentially the capability system of
markets implemented I can send port write support and sometimes I can just directly say okay
I'd apply to this port and this is a port name and stuff like this so what does it mean is that
the fact that the kernel the mix needs to find messages that are needs to make them to find
a message that are actually need to be understood by the kernel it means that pretty much
make and the kernel are tightly coupled so when you implement your own kernel you need to be sure
to implement your own megas well and system that essentially do the physical memory cache
and it just tell the page please save this page so I can create this memory complication that
in reality there are two more complication if you want to know the what I discovered is just
one coupon write mechanism there are three of them and the most annoying one is copy known
which despite the name it's actually copies everything so actually when you switch between
delay to copy normal we use copy known all the page that has worked out needs to be swapped in
by the kernel and then swap it out again and copy and then swap that on both things this is
very annoying and it's only using a 1994 commit by new hard so I'm not going to implement that
shit so oh there's another complication of course you can read that out and not going to do that
so how did that implement mark you know so I hope I convinced you that this thing is complicated
unless they complicated what I wanted to do is avoid looking at the Microsoft code
actively because I wanted to I didn't want to be primed by the mark implementation not so because
I wanted to present this I see what the code look like and what they want to be primed so
the first thing I wanted to see okay the two most complicated system are IPC and VM so
I wanted to implement one first and then the other but of course I can't do that because in
mark when you send a message to my service part of the other space which can be mapping many
VM objects so you can imagine what happened as I said you know this is insane so I
essentially create a special kernel buffer and it's shared memory between kernel user and just send a
message and these are these so allow me to implement IPC first and VM later of course there's
a big limitation that there's a big limitation that the messages are now limited in maximum
size because of course in mark the message that password kernel can be 100 gigabytes who cares right
so anyway I did that and there's a bit of problem in MIG so do the things that I did the mark
is essentially I hit the module and I think this is more expected by this picture so I got the
kernel that only defined kernel object in the interactions they got a kernel module and I leave
module that essentially the MIG output for the kernel interfaces and then of course I got the
but of style that is highly dependent to this so what I have is a way to create multiple
personal inside mark so that we can have a marked thing personality and then having the
you know having various things and testing various things so the way I implemented the IPC
is that I simply have like an external format which is the crazy things that I just got before
and then essentially like I solved the things I solved the messages by by you know checking the
things since only saving the rev component internally and actually this is very fast apparently
and this is the way I implemented the VM I could spend 20 minutes on this alone and I'm gonna do it
but this is all implemented so I'm very proud of that so I essentially have a clock algorithm that
I commit page and this is a layer which in particular is basically the VM object with all the
shadowing part the cache object just be sure to essentially tells the VM object I never seen this page
I marked this page before and the page is page out and so you know the cache object on the page
take longer than marked in the VM object the memory cache essentially the pfn and the clock
the then just the has nothing to do with this