All right. We have resolved the technical issue, apparently. So, up next is Standemire, who has
achieved, did you reenable? Yeah, that should work. Standemire, who he has achieved
to think that we have not yet had before in the two years of new tools, GCC
death room. He has actually achieved two things. This talk is not directly about new
tool chain, but close enough in the layer in the software stack. So, we thought that would
be interesting to hear in particular, I guess, why? Why? It's not directly using the
standard interfaces. And he also has achieved that he has presentation in the GCC
death room this morning and presentation in the LLVM death room in the afternoon. Both
about system-D related things. All right. With that, please get started.
You can clip it on if you like. I can just carry out this one.
Hello, everyone. So, I'm done. I work on system-D as a maintainer and I have been playing
around with fibers. So, like I said, this is also new for me. I've never done anything
in the GCC death room. And I guess it's not unlike the other talks since it's like a
lot higher level, but hopefully this still is useful. So, I mean, the background is pretty
simple. System-D has a lot of demons. The most important one is pit one, but we also have
like system-D resolve the for DNS. We have network management and so forth. And these
demons need to handle requests concurrently. When you're starting a single unit and pit
one, then you have to be able to do other operations. You can just refuse any operations
while that is going on. So, we need concurrence. That's the basic background. Then the
interesting question is how do we do that? So, generally when doing concurrency, you have
two options. Either you have threats or you have an event loop and then you run a non-blocking
code in there. And because we're talking in the context of C, if a non-blocking code usually
means callbacks. So, threats are not an option for system-D. And most in pretty much all cases
where we're running a demon. So, we end up doing the other thing which is an event loop
with callbacks. So, why exactly are threats not an option for system-D? So, this is where
the nature of system-D comes in. This is not a general problem, generally you can use threats
for concurrency. But it's when you start combining threats with forking that things become
problematic. So, the trivial example is you have two threats running in your process.
One threat is doing malloc, it acquires a global allocator lock and then the other threat
forks. Only the threat that does the forks copy it into the new process. But all the memory
is also copied into the new process, including allocated locks and everything. So, you end up in
the situation down below where you have the second threat that tries to do a malloc as well.
The lock is still taken from the memory that was copied from the first process and you end
up in a dead lock. And so, this is solved by kilopsie basically telling you that you can only use
a-sync signal-safe functions after forks. But there's severely limits what you can do after forking.
And so, we end up not doing it in system-D because we generally need to do more, we rely on
being able to allocate memory after forks. This is not a problem when you only have one threat
because there's no other threat that can be taken in the lock, but if you have multiple threats,
then it can happen. And this isn't just theoretical. We used to use threats for a few purposes
in bit one, primarily to asynchronously close file descriptors because closing file descriptors
is a blocking operation. There is no non-blocking interface for it in the kernel. Except maybe
these days for IOU ring. So, we spawned off a threat to do that asynchronously and we did get like
when I was still working at meta. In production, we saw various crashes related to
this specific case. And so, it's not just related to forks because you could ask yourself the
question, why would it be, why could you only use async signal-safe functions after forks because
it's signal. There's no fork in that name, but it turns out that signal handers have exactly
the same problem because the signal hander can generally be invoked any time during the execution
of your project or your application. So, what could be happening is that your threat is executing
malach takes the global lock and then the signal hander gets invoked before it releases the lock
and then your signal hander tries to do malach and you have exactly the same problem and that's why
you can also only use async signal-safe functions in signal handers and like the fork has exactly
the same problem but you need to be using more than one threat. So, that's why threats don't work in
pit one and why we end up doing the event looping. It turns out that unfortunately event looping
callbacks also leads to pain and so to explain that, this is like a very simple example. We have
our event loop running and the event loop, I mean see it as a thing that it waits for events and
it invokes callbacks when those events happen. So, we have one or callback one and we have our callback
two. What I callback one this is it allocates a string fubar then it registers a callback for another
event and then the stack unwinds. Right the callback is done so the callback function will return
and the stack unwinding starts and we use this extremely useful TCC extension in system the right
the clean up attribute to effectively emulate C++ instructors and so what that means is that when
the stack unwinds we will end up freeing our s-for-riable with our string fubar. That's done our callback
two is also past our string fubar because we needed in the second callback. Eventually our second callback
gets invoked it tries to use our fubar string and we get a segmentation fault because the string
has been freed. So, pretty simple error. So then the question is like how do we fix that?
What we could do is we could just not use the clean up attribute. We do exactly the same thing
and then eventually we end up in our second callback again we use the string s. Everything works fine.
The second callback returns and we leak memory because nothing freed our fubar string.
So that doesn't work either. What could we do then? We could say that the second callback
is responsible for freeing our string fubar but what is the problem that you run into then?
You do your first callback everything goes fine. Your second callback never gets invoked because
there's a third callback that stops the program. It exits the event loop. The program exits.
Your string s was never freed by the second callback and so you leak memory again. So that doesn't work
either. So another option is we have this in an system. This event loop does the way to set
a destroy callback for an event source. So what you cannot do is you can set the free function as the
callback for our string fubar that we give to the when we register the event and that will
be invoked when the event source is cleaned up and that's generally a will ensure that the
thing gets freed anytime you need it to be freed. But the problem with that approach is that it
doesn't really scale. Imagine you're in a function five in an asset function deep in a call stack
and you need to wait for another event. You don't just have one piece of data. You don't just have
a string fubar that you want to pass to the second callback. You have like I don't know five
six pieces of data that need to be available in that second callback. So because like the way
it works when you do callbacks you can pass one pointer generally right we call that the user data
pointer. So when you have five or six pieces of data that means you need to create a struct.
You need to define a cleanup function for that struct. You have to put all the data in the
struct to pass it along. You have to define how the data in that struct gets freed and generally
you will not know how the data needs to be freed. If it's just a simple fubar string of course
you need to know you call you need to call free on it. But if you're getting like for example a
path you might just not need to free the memory you might also need to remove the path from this
or not. Right effectively in a lot of cases you don't know especially in generic code
which one you should take. So like the color of that function would have to provide you with the
cleanup function that needs to be called and you can start to imagine how this is not exactly
a fun way to write code or a robust way to write code. So this works for simple cases but it
quickly becomes like inadequate. And so effectively what the problem comes down to is that when you
do a thread and you do blocking code you get a stack on winding or you know you do not get
stack on winding and that's the good thing. So you do your read call it will block and then when
your read call is finished you process the data and you can be sure that your stack has not
been wound. Now your none of your data has been freed and it's all available for processing
whatever your read call returned. When you do the event loop your call back will return your
first call back everything goes like your stack on winds everything gets freed and you need to go
through extreme like hoops to make sure it is available for the second call back. And so the question
is can we have the best of both worlds? Can we have separate stacks because that's what we want so
that we can avoid having to unwind the stack but without the separate threats that's what it comes
down to and like you could imagine this as like instead of passing around like our string
food bar can we just like pass around the entire stack between callbacks and avoid the problem
that way. And so this is effectively what a fiber comes down to right and so it's the alternative
to callbacks which does allow us to do or does allow us to write synchronous looking code
without having to use threats. So what does that look like in our previous example? Our stack does
not cover our fibers so it's only there for running the event loop or primary stack and for each
fiber we allocate a separate stack and so what that allows us to do is we can do like the fiber
reads which I will explain in the next slide and that will that will suspend the fiber we will
I will explain what's suspending is later we go back to the event loop it can resume another fiber
which can suspend and go back and the key difference here is that when a fiber suspends there is
no stack on winding and so you don't get all of the clean up running and it means you don't have to
allocator strikes and everything to pass data around. So implementing this like implementing fibers
implementing the resume implementing this suspending that's where we use the youcontext.h hat
are from CLIPC so well you're the experts so I'll believe you it has three functions get context
make context and swap context get context gives you a reference to the current context which is
effectively the stack the primary stack the make context allows you to create a new one from a region
of memory what you generally end up doing is you am up some memory and then you pass that to make
context and it will give you a new a new stack a new context that you can use for a fiber and
swap context is the interesting one which trips switches between different stacks or different
fibers so that you can implement this resume and this is spent. So what does this end up look like so
or imaginary like fiber read function what do we do in there well first we do the read then the read
returns like the read is on a non-blocking file descriptor so you know it won't block
then we check are we on a fiber so if we're not on a fiber then like there's nothing to do
we just return whatever we got if we are on a fiber then we can check the error code if we get
e again or it would block that means the kernel is telling us we are going to block if we try to do this
if we did not get any again then also easy case right we just return but if we did that get the
e again then we can call that as the event rio function again but this time we pass in a different
callback and the callback is effectively swap context and the user data pointer we give to it is
our current fibers context or stack. What we then do is we can swap context ourselves and so this
will effectively stop execution of the current fiber right it will save everything inside the current
context of the fiber so what is the stack pointer registers and everything and then it will swap
to the main context of the event loop and that will then continue executing eventually like
the events that we registered for will happen in which case the event loop will tell swap context
with the fiber context of the reverse case and we will simply continue executing right after
swap context where we left off and the crucial thing here is that there is no stack on winding
happening so there is nothing getting freed nothing getting like lost and you don't have to
pass things around manually yes so that's what I was going to get into in the second
cancellation is also very important so yeah then there's the cancellation which is also very important
to make sure everything gets freed up but eventually like because we now know that our
read will succeed we don't really because the event loop told us we can call just call read again
it will not block this time and then we can just return the result to the user and this was the
example for read but the exact same goes for sleep for our waiting for a child process to finish
doing this guyo and you can also like trivially implement this or build on top of this if you
have a call back based interface because the callback is just going to be swap context so cancellation
if you go back if we go back to this case here where our event loop gets basically stopped
in the middle by another callback then it's not sufficient to simply free the memory that we have
for the that we allocated for the stack because what the fiber may as done it might have allocated
the memory on the heap right and so if we just free the stack then all that memory on the heap will
be leaked so whenever you want to shut down your project your application you need to make sure
you do you unwind the stack of all the fibers to make sure that they all have a chance to clean up
their memory and everything gets freed correctly and so doing this is kind of simple because instead
of waiting for the actual event to occur what you simply do is you inject an e-canceled error and
you swap context back to the fiber and it's just checks okay do I get the e-canceled error and then
yes it just returns it and so it functions what keep returning the e-canceled error your stack
on wines all the cleaner functions can run and everything gets freed correctly and then you just
you inject the e-canceled error and then you wait for all the fibers to finish and wind their stack
and then you can exit exit program so what I've currently been discussing is a single fiber and
so our single fiber is a sequence of non-blocking operations but often that is not enough you want
to run the non-blocking operations concurrently so imagine you need to talk to five demons at once
and get our day results then having a single fiber is not enough because you can only execute a single
non-blocking operation at once on a fiber so what you end up doing is you end up just spawning more
fibers you create a new fiber you set which function it should run and it will run but you do need
to be careful here because while we're not using threads we're still executing code concurrently
and so whenever you end up having shared state there is all the usual pitfalls that you get what
concurrency which is mutating shared state because your fiber can decide to suspend any time it is
executing it will switch to another fiber and if it is modifying the same data structure then it is
very easy for things to be in an inconsistent state and things will just blow up so you still need
like you still need to concurrency primitives logs and generally good design patterns to make sure
that your concurrency makes sense you can just introduce unlimited concurrency and expecting
things to work correctly sometimes you need to make sure you sequentialize things again
if you're not actually mutating state state state but you just need to copy inputs then
like copying is still like I mean you you keep seeing these talks about Russ where they say
in that one end up just clone the object and that applies here as well right any input
in any inputs you can just copy them to the fiber and you don't need to worry about their
lifetime in the parent's fiber like it can just clean up because you passed in a copy
so like the sounds great of course but there are a bunch of this if like it's not perfect
so like I said you still need to concurrency primitives it's less efficient than call that
because you're juggling around an entire stack it's not possible right because it's effectively
deprecated right because they make context the function prototype is like against the standard
I didn't look into it too closely but it doesn't really work so I found it a bit of a weird reason
to deprecate the entire header but so we we still end up using it and I hope he let's see
one for a move it because it's deprecated in politics but I don't take that well
swap context is not available unless all right because they refuse to implement the interface
because it's deprecated in politics but luckily there is like a little library made by one of the
alpine developers that effectively implements the same interface so we can also use it on on
muscle and for postmarker OS the integration with gdb is not as great as with threads
because with threads you just get you get all your back traces for all your threads built into
gdb and because all these fiber because all this fiber stuff generally is application dependent
there is no generic interface in gdb to allow you to inspect the state of all your fibers
so every project that implements fibers effectively and supreme implementing some kind of gdb
plug-in thing to allow them to inspect their state of their fibers will probably end up doing
something similar not ideal but there's no easy solution here I think performance so
the thing that politics mandates for swap context is that it restores the pretreat signal mask
in our case that's not required any of the other core teen implementations that I found online
because they all focus on performance performance performance for their fibers they don't do that
part because it means a Cisco and that's apparently slow I don't think like we are going to be
using that many fibers that this will actually make a difference but I was still like curious
if there would be a way to make a chillipsy not to this is called if possible but if it's not
possible then it's probably not a problem either and while we do get like non-blocking socket I
owe we still don't have non-blocking disk I owe because I owe you ring is still blocked in most
container run times and and sandboxing environments because it doesn't allow you to filter the
obcoids but that is being fixed by NSX so eventually I will be able to integrate this with I owe
you ring behind the scenes and then we will have non-blocking disk I owe on fibers as well
the current status of this is this is a PR force lip system the I call it as the future of
age because we don't just need fibers we need futures as well but I didn't discuss those because
the talk would become too long it's internal for now but I think this is useful enough that
it will probably become public I don't know how much useful it will be for the wider community
because generally when you're starting new projects that need to do this kind of stuff they
end up already being rust or something but if there is like existing code using lip system
then maybe they can make use of this so it has the basic operations socket I owe child process
waiting sleeping waiting for all the fibers and some other stuff I still need to implement
concurrency primitives but didn't find time yet for that and that was it happy to answer questions
all of them
I was going to, so I'll give her a Peter question.
The question was, there's Peter at the fork.
Why can we use this to solve the problem?
And then, access the thread of the thread of the one thread from the other thread.
There's many ways that fibers can go on to remove, you understand it often.
If you want to do the existing game, you're supposed to have it and want to play,
but just you'll let you have one time, because it's a commodity, it's a complex platform.
Where does the account still have to do it?
Yes, so the question was that when you start doing multiple threads with fibers, things can go horribly wrong in various ways, or the comments was.
But yeah, so generally we use threads, I think, in one more place, some very isolated place and for the rest,
everything is single thread and when we need to concurrency, that runs at the same time we use fork.
And we run a separate set of sub-process.
Just to take a couple of minutes before, right?
I know what it looks like.
You look for it, guys.
You need to read that by the wall.
Let's have a break.
Well, I see this road isn't like it's almost finished.
I'm not expecting the system to go to this place.
But the reason I've written that up is because as part of this year, it's the first time I've written it,
there is a SD-institution abstraction, which is basically some of the things you've got to tell the truth about.
It separates the concerns of actually working across curing operations and getting the most corporations,
and allows you to move things like weight on five things where it brings out to them.
So there might be some design systems from there for regretting these things.
Yeah.
So the common was that C++ has cover teams and the execution and that would fix all of this.
I totally agree, right?
I would love for nothing more than to have a language available that does this for me.
I looked into cover teams as well for C, but effectively where our compiler helped doing cover teams.
So stackless cover teams is impossible.
So this was the best thing I could come up with.
I think it can come from design.
I don't want to add that.
I want to clarify that we don't actually need to spawn necessarily five new fibers for every single non-blocking operation.
It's only when you want to do two non-blocking operations after each other that we allocate a stack.
That's why that's why the SD-instit is called SD-future.age because for stuff that you don't need a fiber for you,
we do have like, we don't do a new fiber.
Like I did that in the beginning and then I really asked,
oh my god, this is hard for you in efficient.
And so now we don't do it for unless actually need it.
Well, future features about the future.
Yeah.
Oh, the time is up, unfortunately.
I can be outside for more questions.
Thank you.