I'm very sorry for my colleagues who have been hearing me talk about
riling over and over again, so yeah, got another three minutes of me repeating everything.
What's one of what's my point?
Okay, the violin guy diseases to my work for a mutable and violin, what's that?
I have to make the same joke, this is not that.
Extremely simple IPC.
It's Jason, former than it was so extreme soccer.
Terminated was a null bite, replies.
Also Jason was a terminate null bite and that's really all there is.
And thank you, this is where my talk was.
But of course, there's more.
There are specific occasions, they're not very good specifications,
but there's something like a specification on that thing.
The binding's available for all the popular languages, including one new one.
And system deep.
It has been around for quite a while, initially found very little adoption,
but we started to make use of it very much, though, and system deep.
Yeah, and before we talk to another video, since they will.
Let's talk a little bit about diva, because if we talk about a new IPC system,
let's suppose to take, like, where we want to move the focus from diva to this thing,
and I guess I need to explain why diva is not the future in my point of view.
As an explanation, I've been around for diva for a long time, because, like, initially,
I was one of the first, like, I worked on a vahee, like, the zero-conf,
demon, and was one of the first heavy users of diva.
And then I implemented SD bus, like, the SD bus library for systemy.
And I think I'm still officially one of the containers of diva's demon.
Anyway, so I have a long history of SD bus.
So I hope this explains a little bit why I think I know a bit or two about diva's.
And if I have an opinion, I'm not a complete idiot by saying this.
So yeah, SD bus is actually okay for many users, right?
Like, it's an IPC system, and you can use it.
But it is serious limitation for many other users.
In particular, it has many limitations for what I, and in the systemy,
we, as a project, have been using it for.
Systemy deeply integrates into diva's, right?
Like, all our APIs, until two years ago, were exclusively available by a diva's.
And this level of integration is not just going away.
But yeah, we will focus, like, move our focus probably more to vahee.
So let's be more specific and talk about problems with diva's.
There are many problems, right?
Like, I could have many, many slides.
But this talk is not supposed to be our diva's supposed to be a vahee link,
so I'll try to keep this short.
One of the main problems as diva's is not available in, until like boot, right?
Like at the demon, it gets started when systemy leaves the early boot phase, right?
Like, it's not available in the inner d, it's not available in the early boot phase.
After the inner d is annoying.
Yeah.
It's, I mean, it's not on my laptop, it's somewhere else here, issue.
So it's not available in, it's not available on the late boot.
Not an inner d, for example, and this is painful for us because, of course,
a good chunk of the complexity and systemy of everything that it does is starting up the system.
And if we have one of the core concepts that we need for this, which is the IPC system,
not around during early boot, things are really, really painful.
Yeah, it's just too late.
It's been probably more, too late for more of the half the things that we do in systemy.
Systemy has a way out so far.
We basically wear our own little divas demon in a way.
So you can talk to divas to pd1 via the divas protocol.
And you get basically a direct connection.
And in a way, we embed a little pseudo divas demon in our code base for this.
But it doesn't have the proper divas emancs because direct connections and divas,
they do exist, but they are weird, right?
Like because there are no signal matching,
there are a lot of other things that a real divas demon has that we don't have if you have such a dark connection.
So it uses the marshaling, but it doesn't really reproduce this amandics,
but it was good enough for us to make systemy work.
So this is actually really annoying to us, because, you know,
there are so many demons that network differ example.
It's a networking thing and we would like to offer a DAPI for this of course.
And then if we make it based in divas, then this would mean that networking is only available in late boot,
but then of course that's not how you can do things.
We need nowadays networking during very, very early boot because you might want to boot from a network source.
So, you know, the way how this works in networking right now is that we actually watch when divas is available.
So first you have this networking demon that is starts up based solely on configuration until the point where divas becomes available.
And then it connects to divas and also makes it serve as available and divas and suddenly can actually talk to it.
But this is really painful because it shouldn't be that way.
You should always be able to talk to network D.
So there's also the other problem which is that it cannot be used by many basic IPC service.
It's divas itself consumes them, right?
This is for example to use a group database.
So divas has this policy language.
It's like this exome alum monster in a way.
Where you can say, yeah, this shall be accessible by this user in this group.
But this user in group stuff is traditionally, like, like, I mean, most of the time it's just at C possibility.
But it's inherently designed so that they can be back and serve as a provider user database instead.
And then you have this problem, like, okay, we have this demon that is supposed to provide the user database or a part of it.
But hence I need to talk to it.
So I want to use IPC, but I can't use divas for this because divas would be a consumer of the user database and then you have a cyclic loop.
The divas uses that IPC service which implement, like, implements a divas service which then wants to talk to divas and then you have a deadlock.
So that's also very, very painful that for many of the core things that system do wants to do in other really low level systems software wants to do.
You just can't use it because, yeah, this was resolved in deadlock.
And if it wouldn't result in deadlock because you're very, very careful with asynchronous programming, it's still a terrible nightmare.
And you can have to figure out, like, special handling to deal with this.
And other problem that people typically don't know even if they know divas very well is that it can really not be used to string data, right?
Like, it's fine for control messages. It's horrible if you have to do streaming data, right?
Like, because it has no sensible flow control.
This is because, like, it's the shared resource, the divas demon, right?
It needs to make sure that all the clients that talk to it do so.
Not flooding the demon was too many messages, because then one of them could monopolize the resources and the other ones could not be processed.
So they, they, what the way out for them is basically they do rate limiting.
So if one of the clients sends too many messages, then they, or it doesn't process the messages of center it,
then they will kick the, then divas demon kicks it off the bus.
But this is terrible, right?
Like, because if you want to stream some data through divas,
yeah, I don't know, you have some huge user database or something.
If you want to stream through it, like, somebody asks you for the user database and you want to census through divas,
you have to careful for this with this, because if, if your user database gets to large,
and you try to do this, then divas will recognize this,
this thing is flooding me with messages on kicks you off the bus.
This is a real problem.
And basically means that whenever you have actual, or even have the risk that it could potentially be huge data that you have to stream through there,
you cannot use divas, right?
People ignore this, and this shows up basically as a scalability problem.
Right where I don't know, systems are used in suddenly in a big scale on some installation,
and suddenly everything falls apart because divas kicks everybody off the bus.
So the people who do understand this, then try to find patterns around this,
and use side channels to get the actual data, or they split up all this in your narration,
they could happen this way in individual messages and do flow controls themselves and things like this.
But this is also inherently racy, because if you split it up, it's not going to be atomic snapshot,
and then atomic snapshot of what you're doing, but it's going to be a series of probably hopefully closely related stuff,
but not the atomic snapshot.
That's terrible.
It's also slow.
Because like this divas broker, this divas divas demon that sits in the middle,
needs to basically process every single message sent on the system, right?
Like it needs to copied from the percentage, and then to the destination,
and then on the way back for the reply, needs to do this thing again.
And this also means that every IPC call becomes really expensive,
because it's at least four context, which is, right?
Like from the client to divas broker, from divas broker to the service,
from the service to divas broker, and from the broker back to the client, right?
And that is expensive, right?
If modern CPUs and model operating systems are good at stuff,
what they are not good at is doing these kind of context, which is it?
So it's really wasteful on the one thing you really shouldn't do,
if you care about the performance latency and things.
So also security model is simply garbage, right?
Like there's a good reason why everybody uses who implement the system,
so it's at least uses policy kits to make security changes.
I mean policy kits is also not the perfect design,
but it's at least a divas security model.
And it's like in essence, if you look around at the security policies
that are shipped on general Linux distributions for system divas services,
they basically let everything through,
and then use the policy kit to make the actual decisions.
And if you look at the desktop stuff nowadays,
with flat pack and sandboxed apps,
they also don't use the security model of divas.
They put a divas proxy in the middle,
which makes the actual security decisions and isolates the app world
from the desktop world outside of it.
Policy kit in particular, the way it's worked out,
was also really, really messy,
because it adds like six more process context features, right?
Like because now we have the client,
talking to divas broker, divas broker talking to the service,
that's supposed to respond to this,
the service then sending a message to divas again,
that is intended for policy kit,
then the diva broker sends that to policy kit,
policy kit back to divas demon,
divas demon back to the service back to the divas demon,
divas demon to the client.
It's so wasteful regarding process context,
which is, and again, those are the things
that make everything slow.
We should avoid them.
Then another thing, I need this goes on and on.
Another thing that is really, really painful is like,
very often you have this pattern,
if you talk to a diva service,
but you create some object on the other side,
I don't know,
and a vahee, this works for example,
the service browser, right?
Like you say, I want to see all the web-deft shares
and whatever else,
there are on my network,
so you create this web-deft sharing
browsing object on the other side,
and it's yours,
and you expect to get signals from it,
so you, like,
whenever something like the shows up,
goes away.
So you want to subscribe to it,
and the way this works in divas is then,
yeah, you first do the message called,
create the thing,
you get a handle back,
which they call the object fast,
and you go to divas broker,
and ask now,
I want to install a match,
how they call it,
so that I get the notifications out of this.
But what's really painful about this,
is that this is necessarily racy, right?
Because you can only ask for the match to be installed,
if you know the object path,
that the browsing object is about.
So there's always a time window, basically,
where the service has already established
this browsing object,
this object on the other side,
where the object path not travels into your direction,
where you then have to send it back to the divas broker,
until the divas broker actually installs the match for you,
so that you can get the signals for it.
This is super painful,
because, in particular, in the avahi case,
this meant that, like,
you would always lose the initial message,
this way it tells you about,
like, the services that are already on the network.
So that is a conceptual,
just complete misdesign, right?
Like, if you are,
have the conceptual rights to create an object somewhere,
of course you should be able to get the signals,
the navigation is about it,
and the fact that this is not race free,
it's just total misdesign.
It's painful.
And then there's, like,
some of the kernel people,
when they look at the divas, they call the stuttering.
Like, if you put together a divas method call,
for example, with even on the shallow, like with divas send,
then it feels like it's stuttering,
in the sense that you,
you specify basically the same string three times like you.
I, like, like,
like, uh, Arc.Three desktop.systemD.
Whatever service you call a call to,
you specify that first as a service,
then you specify it as the object has,
then you specify it as an interface name,
and then you add the method called to it.
So, conceptually,
service object paths,
interface name, are different things, right?
Effectively,
And almost all services, they are the same strings, right?
Like sometimes you write them with dots and sometimes with slashes,
but there's the same, same literal stuff.
And they call this stuttering, and it's terribly annoying,
because it makes everything hard to read, right?
Like because, in particular, for simplest services,
that's really all you always have,
and you still have to type this in everything all the time.
Then, I mean, Shell completion makes this nicer,
but still it's like somewhat visual clutter
for very little value that it's annoying.
That's so much more.
The type system has an all the concepts.
It's not extensive without breaking API.
It's has a, we've done an encoding.
Interspection doesn't know name structures.
It's very cryptic and XML has no documentation string.
Has property model that pushes non-atomic behavior,
so annoying too, right?
Like because they have all these individual properties,
they can have an object.
And typically, they, these, they all change together,
but you cannot get an notification about
that they change together, that you will get all the way,
so if you have five properties, you'll get five separate messages
about this, and I don't know.
The error system is just to naive, I just see connections.
They're not serializable and real-autable, right?
Like this is something really important for us.
Like, just to me, people expect us to do
a system-d, a demon reload, and then we'd feel everything down,
come back, but do not lose connections.
Can't reduce that with dealers.
It's not a there's an delegation concept of containers,
and boxes unless you do proxy filtering.
I mentioned this kind of, it's very web unfriendly, right?
Like, nowadays the world runs web stuff,
and it's very hard to connect these two worlds
because they have very different semantic behavior.
There's no enums, there's a global ordering climate,
makes means that you can never have workers' reds
and things like this.
Let's go on and on and on and on and on.
So I'm not going to go into much detail with this.
So much about D-bus, before it's talked about the good parts.
I don't know there would-
You have a question?
Yeah, quick question about the dimensions
that the D-bus is not really suited to the web.
Could you give an example, use change for that?
So the question was regarding it.
I said that D-bus wasn't really suitable for the web,
and I should give an example for this.
So the way how the web works is usually
like this get a response based approach, right?
Like you do one request and you get one response,
or you push something to the cell and you get a response, right?
D-bus is not like that.
It's inherently stateful, right?
Like you create a connection.
This connection is continuous.
It's multi-plex, even, right?
So you will get multiple requests over this,
and you need to deal with this.
You have to continue to process what's coming on on it,
because if you don't, and somebody sends you message,
you will kick off the bus, because that's the logic always, right?
So very, very stateful, and then they install the matches on it, right?
Like, so the connection actually has a lot of state
attached to it in the D-bus broker.
Like what signals shall we get to it?
So it's very, yeah, the get-based stuff,
that the web people are used to, and this inherently stateful stuff,
they just don't go well together.
People do this, like, for example,
you know, cockpit project from Radhat.
They, they, it's basically a web front-down to configure your server.
It's like, it started even out with a translation layer
between HTTP style stuff and D-bus stuff,
and it's just painful.
It's, you can suddenly pull this off, right?
Like, but the impintence mismatch means that you
have to maintain a lot of state in this proxy
to make this work.
Any other questions at this point?
Um, okay.
So I'll start with the problems about violin,
so that you all forget about them before we come to all the good part.
Um,
I mentioned this that violin uses Jason.
Jason is a very, very popular encoding of it.
It has problems.
One of them is that it's not 64-bit integer clean.
I mean, the Jason specification,
the original one, kind of leaves open how precise,
like, what's the, what the bit with is supposed to be for
for the numbers that they have.
They basically just leaves this open,
so it could be auditory precision.
Effectively, though, everybody, um,
like, because the origin, like, I mean, was derived from JavaScript
and JavaScript didn't have an integer.
It only had, like, floating point numbers, um,
like, set it at, like, 64-bit floating point numbers.
And, yeah, the lot of this, like, if you care about integers,
the largest integer you can store there,
in a lossless fashion of 52-bit.
So, uh, and this, this was an average
and many, many, many implementation.
So, people generally assume that, uh, yeah, 52-bit bits,
you can encode losslessly.
If you want to go more than you risk losing
compatibility with certain implementations.
It's a real problem.
I think it's not a big problem.
Not as big as people think it is.
But, yeah, um, it one of the solutions is to form
our format, large integers, the strings, um,
what I think what's more important there is,
like, at least in system, we, we figured out
that most of the times where we actually wanted to use an integer
above 52-bit, um, this was mostly to mark,
like, to have a niche value, you do, like, a special value
that means this does not apply.
Like, let's say, for example, um, memory limit is configured, right?
Like, on some systemly service.
Um, and, uh, if there was no memory limit,
we would expose this as a, uh, uh,
you in 64 max, like, to, to 64-1, right?
Um, and then, uh, this was our special case.
Our niche value that says does not apply.
There is no limit.
Um, uh, but, uh, this, in adjacent world,
this is much better module and modeled,
if you use the null expression, right?
Like, which is, which is a literal value that indicates
this does not apply here, right?
So, uh, in practice, we realized, uh,
yeah, the 62-bit limit kind of sucks, um,
but, uh, most things in this world, uh, um,
I'm kind of fine with it, but yeah, we,
the way out is, uh, to form a disaster.
So, it sucks a little bit, but it's not awful.
Um, uh, one other thing, of course, is that the marshalling,
uh, sings into Jason, is, uh, is a little bit slower than it is for the
binary encoding, the divas use, um, like, uh,
40% in average, like, if you look at the various distribution there,
like, um, there, like, um, there's a,
z-chan already did, uh, like, measurements of this,
where I tried to figure this out, because,
you know, many people assume that marshalling is where
performance, uh, um, has a big effect, um,
but, you know, um, uh, uh, uh, uh, uh, uh,
it's not, like, marshalling, like, 40% like,
a linear increase of this, or 40%.
It's effectively means that for small messages,
it's not even, like, you cannot even measure this, right?
Like, uh, and the actual latency price that you pay,
you don't pay for the marshalling, anyway, you pay for the Roundtrips, right?
So, um, uh, uh, Márch pelvic....
in my point of view, uh, just ignoring it entirely.
Yeah, sure, it's a tiny bit slower, um,
but, uh, this is offset like by many, many, many, many millions of times
by the, uh, fact that, uh, we can get rid of all the
context, which is. Yeah, so it's around trips that kill you, sliding
wasteful marshalling doesn't. Anyway, so much about the problems of
harling, any question at this point?
Yeah, can you send binary data about the problem?
So, the question was regarding whether we can send binary data
of a volume difference, that's a problem. So, yes, you can,
that two ways, like if you have small data, like I don't know, a hash value
or something, you just form it as string generally, right? Or you can
code it in basic form, like if it's a, I don't know, public key or
certificate, whether it's not going to be like gigabytes of data
than you do this. But, uh, uh,
harling inherently has a mode also where you can switch
protocols. Um, so the idea is basically there's a very, very clearly defined
way way how you can say. Initially, we're going to be
balling to set everything up and then we do a mode switch
to something else. Um, so the idea is basically, you can't even transfer
like gigabytes of, I don't know, disc images via volume in a very nice way
because you'd first do balling to the JSON thing. You set the marker
that after this message, that's going to be a binary data
and then the, this would happen, they just continue writing a binary data.
So, uh, that's really nice, actually. It's one of the
types features of balling that you can do something like this. Because it means you just
do one connection. You do not need some extra side channels with this. Um,
and, uh, I mean, to say this differently, you know, debuts sending binary
data is like because of the, the, the streaming issue, like the,
the flow control issue, you, you cannot do this at all, right? Like you
force you into the side, uh, channel. This one's so much nicer. Um,
if there are now questions at this point, I'm going to talk a little bit about
the benefits. One of the things that balling is really nice is that
connections are not just connections where you can send the data
over, but they conceptually also entail a, uh, uh,
pinning of an object on the other side, right? Like so, you do your
method call and, uh, that conceptually, the idea is that as long as you keep that
connection are open, the object on the other side is pinned. So, with my
avahi example earlier, where, uh, we have this browser object created in
avahi and, uh, we need to have the signals and things like this. The
way, how you would express that involving is, um, uh, uh, uh, uh, uh,
you create the connection to the other side. This creates the browser
and the notification chain and pins the browser's existence on the
other side. And as long as you keep that connection open, it stays that way
and usually drop the connection, everything goes away, the notifications, the,
the existence of the browser object on the other side. And, uh, yeah,
the communication channel itself. Um, one of, uh, uh, uh, uh,
I think this, this is one of the best things ever is there is no
multiplexing involving, right? You never use a single
of our own connection to multiplex connections to multiple different
services. You instead, uh, if you have 25 services, you talk to, you create,
and 25, uh, uh, uh, different connections. And each one of them has
local ordering, meaning that, uh, a message you send on the connection
is always going to stay after the one before it and before the one, uh,
uh, um, behind it. But there's no global ordering. This is, by the way,
another thing, diva and diva is there's global ordering, which has benefits,
but it also makes things very, very inefficient because global, like,
a global ordering basically means that any message received by the broker
can never be ordered after any other message received later, um, in the
entire broker. And that's, that's so painful because it means that you can never,
uh, uh, uh, have threads, uh, that process things, uh,
because that would mean out of all of our execution. That's not a lot of divas. But then,
uh, rather than everything is easy, right? Like you can have a, have a thread for everything,
uh, uh, uh, uh, a connection you have, and the, the focus really in having,
yeah, just local ordering on the individual connection, but have many connections.
And then, uh, you got the best of two worlds. You get ordered behavior, because that's
sometimes what you want, but you also, um, don't force the whole world into ordered,
uh, behavior because that would be terrible. Um, and it's request response space,
right? Like you create a connection. You, uh, uh, uh, uh, uh, send the,
request or when you're going to reply back. That is, conceptually,
very, very copy, uh, compatible with the HTTP world. So there was this question, uh,
earlier regarding divas and the HTTP world. This fact makes it so nice,
because it's not HTTP, but it is, uh,
semantically, very, very close to it. And that makes it very,
very easy to have a proxy that exposes viral and KPIs as, uh, as a true HTTP,
and it will actually feel natural and good in a way. Um,
it's also, like, one of the biggest benefits is simplicity of it, right? Like,
it's, it's just JSON over a socket. I can explain this to everybody,
explaining divas to people, uh, so painful, like, so, so much more complex.
And in particular, the web native folks, and I guess this is what dominates IT these days,
um, uh, know the concepts, the underlying concepts, and can, can very quickly get on the
violent train if they already know JSON and these kind of things. Um,
security, uh, any question at this point? Um,
the security model of our link is, uh, a completely different one. Uh,
is, uh, that there is no security model of itself. Um, we rely on the kernel, um,
to do access control for us, uh, primarily, which means like,
because every violent service is inherently an operating system object, right? Like,
it's, uh, binds a socket in the file system. Uh, it's also means that the access control
on that, uh, uh, entry points socket in the file system is managed by the kernel, so they can do,
uh, like, classic unix access modencies kind of things, but they can also do that all that
max, like, as a Linux and whatever the else they want, uh, on this. Um, it can be, uh,
auditors, blah, blah, blah, blah, blah, all the things all the infrastructure that the kernel has,
because we no longer have the concept of a service as a user space concept,
that is a dependent of what the kernel manages, but it's inherently,
it is just a way to, uh, like, it's exposed to kernel object.
Of course, this doesn't always work because, um, like, it's not sufficient to,
to just rely on that because kernel access control stuff is generally,
non-interactive, right? Like, there's no way how the user can be,
can be asked if that something's okay, and we probably want that, right?
Like, so that for, for privilege operations, there's a way how a user can be prompted,
do you really want to format your hard disk and things like this?
And that's something that policy kit does.
So, uh, yeah, the security focus is different one.
We just don't do anything on our own. We just put the focus much stronger
on relying on the two other, uh, uh, uh, uh, uh, uh, uh, subsystems that do this much better.
What I've added, uh, which is kernel stuff, and, uh, I'm policy kit.
Um, then there is also, uh, it's really nice is that there's a delegation model, right?
Because, um, every service and violin just has their own,
I have unique socket and the file system. Um, uh, we can use this for, for delegating access.
So, uh, uh, we can mount them through into sandboxes and containers,
and every service individually because, yeah,
like, and not sure if you guys all know this but the way how a Linux you can,
like, you can mount individual, bind mount individual files somewhere else,
so you can have the host list that binds to an AF unix,
so get somewhere and slash run, and then bind mount that into your sandbox,
and then that sandbox could also access. Um, yeah.
And then there is, uh, like these concepts called SOP-acred, SOP-acred, SOP-acred, SOP-acred, SOP-acred, SOP-acred,
SOP-acred, SOP-acred, so, uh, uh, uh, to do first,
or, like, identification of who you're talking to.
I am a divest and realize of this too,
but, uh, for us, it's just directly how you do things, right?
Like, um, it's a way how you can identify the other side of a socket
and a trusted way, um, uh, which is one of the benefits of using AF unix
that you can identify things like this.
And, uh, something that's also awesome is that there are no synthetic identifiers for objects.
Um, like, in deepest there was always this object pass concept,
which is, like, synthetic meaning that, um,
the services behind it and the clients, uh, never knew this kind of, uh, identification for an object.
But, uh, you would have to make one up if you wanted to communicate via a divest.
Like, for example, in, in, in, in, in, insisting the, you know,
the, the primary object that's insisting managers is probably the unit, right?
And it has a name, right, like, uh, HTTP-D, uh, dot service or something like this.
When it's, like, talking about this via a divest,
you would always have to turn this into diva's object pass.
So it becomes slash or slash free desktop slash system d slash the units slash HTTP,
and then we are escaping service.
Um, so, uh, uh, this is painful, right?
Like, because every time you pass something over diva's,
and want to talk about it in diva's and have an object on diva's,
you have to do this, um, conversion for some actions just terrible.
We're only going to do this, um, it just assumes that, yeah,
every system has the native identifiers.
And if that's a file descriptor, if it's a, uh, pass in the file system,
or if it's a domain name, or if it's a unit name,
doesn't really matter.
It's that thing that should be exchanged over the wire,
and not something made up that everybody on both signs has to convert for some back to.
Um, and there's so much more.
Like, methods can have multiple replies, uh, which is, yeah, that's a question.
So, you don't have an object.
So, uh, if that's object identified,
how could you service this temporary,
and find out that an API isn't made?
So, the question was regarding, it was regarding, uh,
how we do services coverage if we do not have an object identifier.
Um, so, uh, services coverage to a large degree is debugging concept, right?
You want, you're an admin concept, right?
You want to see what's actually available.
Um, so, and, uh, because, I mean, it's a really good question.
Um, the, uh, the stuff, like, the every,
the violin service that we have is exposed as a, uh,
as a socket and the file system, right?
So, you could just go to proc net unix, right?
And have a list of the, a socket stare,
and it's going to cover all the violin stuff, but also more.
Uh, we're working on getting a kernel patch merch, so that we can tag the sockets nicely,
as violin sockets.
Um, that would be wonderful, because then we do not need any further, uh,
uh, uh, a service admiration, which just can ask the kernel, um,
uh, uh, give me all, uh, a few unix sockets in the file system that has this,
extended attribute marker, as this is violin,
and then that's the list of services, right?
Um, but, uh, this is working progress, right?
Right now you're supposed to know, uh, which socket you have to talk to, right?
Um, uh, it's not, uh, but, you know, like, for a system deed,
basically, we put all the sockets for the system deed, uh,
uh, services in the same directory in Slet Run, so you can just do an LS there,
and we used, like, the reverse domain name notation, so it's, yeah.
But, uh, it's not, it's not like systematic.
I mean, the violin specifications actually know a concept of services recovery,
uh, but we never implemented that in Arco, because we don't need it, right?
Like, because, uh, uh, as long as you only talk about a few unix sockets,
it's kind of, like, the file system hierarchy in a way is your services recovery.
Um, so we didn't, uh, need this.
So, well, they've passed that.
Yeah.
It's, he said, uh, if it's about well-known past, uh, and yes.
Have you thought about implementing, uh, debust library,
that's actually a shame performing, or the differences between the two systems to learn?
So the question was regarding, if, uh, we thought about, uh, adding a, uh,
a library that is a shame around, uh, uh, uh, like, which way did you want to have it?
Like, the violin over diva, so diva's over there.
I think neither of them really, it makes me nervous because this is a mathematical differences, right?
Which is too big. Um, like, diva's is really, really complex.
The violin is very, very simple.
And, uh, it's, it's just the impedance mismatches, just too massive.
I don't think.
I don't think that's realistic, like, I mean, the way I think how this should continue, right?
Like, that, the, these two worlds left side by side, and we slowly move things over.
And, uh, I don't think, um, if you have a diva's interface and that's a good one, just keep it.
Um, but, uh, I know that for our stuff, we could just move the focus over to the violin.
Um, one I think is that the, uh, like, violin and hernity know something, uh, like a concept,
where you have massive calls that receive minor replies.
Uh, this is useful for any narration and for, for, uh, a subscription.
Um, where you basically say, you have a message called, subscribe.
And you do this and then you get one reply that says, okay, the subscription is in effect.
Then you get multiple, like, further replies every time, uh, something changes.
So it's about unimeration and such, like, you know, in unimeration, basically means you get all the replies at this, very quickly,
after another subscription means you have a long running connection.
And it's freebable like this.
Um, it's also really, really compatible with per connection.
This is like, uh, this is, I always say that this is one of the absolute killer features of, uh, of violin.
Um, because, you know, traditionally, if you want to write a diva's demon, um, you're forced into basically using an event loop.
There isn't no other way.
Um, because, uh, everything is multi-plexed over one connection.
Um, you will get, like, the, the instant you can, uh, instance you connect to diva,
you must be ready to process multiple messages and parallel.
Otherwise, I mean, you could avoid it, but then you're not really well-behaving diva server.
Um, so, uh, um, um, yeah.
And because it's coming in from one connection and the global ordering and things like this, you cannot use threads.
So it, uh, the only thing that you can do really is, it's an event loop.
This is very limiting, right?
Like it means things are slow and it seems they're painfully complex.
Because if you do a vet event loop, at least if you do it and see, then, uh, yeah, like, uh,
uh, don't talk about this yesterday about how painful it is to have these complex, uh, systems, uh,
uh, um, where you have to basically pass the context data of what you're doing from one event loop iteration to the next one,
because, uh, yeah.
It's, it's very, very painful.
Now with violin, you don't need this.
Because in violin, every connection is separate from each other.
So what you can do, actually, is you can have a, uh, uh, um,
first of all, you suck at activation, system D, meaning that system D binds a soccer for you.
And then every time a connection comes in, we actually fork of a new instance of your, uh, program to, uh,
a handle that specific connection.
And this is, this is, makes things so much easier, right?
Like because the individual, uh, binary can be single threaded, um, it can, uh,
deal with one connection at a time and process that and then exit.
And then, uh, the next connection will get a separate instance and so on and so on.
This in a way is a lot like, uh, unix tools are traditionally written, right?
Like regardless if you use, um, I don't know, L.S.
Or if you use, I don't know, find them out from utilities.
All these little unix tools.
They always have the same way.
You pause in, uh, like you invoke them, pass in parameters on the command line.
You possibly, uh, uh, provide some input via a studio in an output via a studio.
Now that these tools even often speak JSON natively, right?
Like a, like a, you tell, unix most of them can output their stuff as JSON and,
many of them can even read JSON's input.
So the only thing that's like the parameters to the, to the calls are just in the,
and then, um, um, command line still.
But now you can basically take one step further from a such a command line to that already speaks JSON and say,
OK, now I bind this to a socket and teach it the special functionality that it can also read its parameters directly from the JSON,
uh, violin, uh, function, uh, stuff that is before the payload of the data.
And then you have a violin service, right?
Like so to me, in a way, this is like taking classic unix tools that is already one of the good ones that does JSON for everything.
Doing one more thing, which is like this.
Yeah, instead of using the process command line to provide parameters, you move this into the,
as to the in-stuff issue as well.
Um, and it's already a violin service.
So, yeah, so the fact that it can do things like this and spawn off multiple instances,
each dealing was one, um, uh, a connection coming in.
This is, this is game changer, right?
Like because this, in, in systemd in particular, this led us to an explosion of debas, uh, of violin services.
Right? Like I think we are at 13.
I forget the number of debas services right now that, uh, systemd has,
but we are already at 30 or something, um, uh, violin services.
And the reason for this is like consider a little tool and systemd like boot control, right?
Like boot controls is tool that, I mean, you can do various things,
but the primary thing is it can install a boot loader into the ESP, like SD boot into the ESP.
It's a, it's a command that you have to execute if you want to install SD boot,
but also, uh, it's not really called that often, right?
Like it's called during installation, or if your hack,
maybe you call it a little bit more times, but that's kind of it.
Um, and, uh, uh, the idea, like, it's also something you inherently want to have an IPC service,
because if you write an installer, it's one of the things that the installer has to do while
installing something like this, right?
So, uh, but the idea of turning that into a deemous demon is just revolting, right?
Like because it's this little tool that ultimately does nothing more than couple of copying and files somewhere.
And if you want to do it in deemous, you would have to think about, like, how do I do this copying now
inside of an event loop and deal with all terrible?
And then, for what, even, because, uh, it's not that they're going to be like 25, uh,
requests coming in a parallel to install a boot loader, and almost every possible,
um, case it's going to be one invocation during the lifetime of the system, right?
So, uh, uh, uh, uh, uh, in borrowing, all the things become super, super simple.
We just, yeah, it's already this tool that is a command line tool you invoke once.
Now you make it read, um, uh, it's parameters from, from the borrowing method called parameters
that come in from SDN, and, uh, write out a, I've done my work, okay, seeing when it's done.
And then you bind it to a socket, and there you go, it's a rolling service.
So, this is a game changer, because all these little tools, these unique style tools,
that do one thing or single thread it, deal with SDN, SDD out, and the command line are so easily converted
into a violin thing. So, yeah, I think this is a killer thing, right? Like, because it makes it easy.
Easy, easy, easy to expose, uh, system level tools as, uh, services.
There's a question.
You said you treated you explained why you wanted IPC in that example.
Why wouldn't you just use Unix for X, as you are?
So, the question was regarding why wouldn't Unix for X, I could be enough as a, as a way to call
these tools instead of doing IPC. Um, like, doing IPC, uh, like, first of all, it's very useful
if you get, uh, like, structured data, right? Like, uh, you do not have to care so much about
escaping, do not have to care about error handling, like figuring out what the actual error is.
Also about the isolation, right? Like, the execution context of, uh, of that tool is separate from
the client that makes the invocation. So, you can use it as a security, um, isolation that one
is on privilege, the other one is privilege, and things like this, right?
So, um, one is coming from user contact, a context, uh, one from, from system context.
Like, for example, installer tool, most likely, um, is a UI tool, right? Like, a graphical tool,
which you decay, writing a lot running as an unprovoked, um, uh, a piece of code.
But boot control install and parenting needs high privileges, right? Like, so you can use the IPC
as a, as a security boundary, why, where you have a limited interface that does exactly what,
what shall be allowed, but with fork and X, that you can't do this. But actually,
having said, was that, that one of the other killing, killer features of, uh,
borrowing them, and sure you might put this on the slides here is, um, because it's so similar,
they're actually two ways to, you can call, uh, such a viling, enable tool.
You can either bind it to a socket, talk to the socket, and everything's good.
But you can also just fork it off, um, and use a socket pair for communication,
and still speak viling to it. And we use that, actually, like,
I've recently added this, like, many installer things that just just,
command line and text row to, to system D, and it's just a wrap around boot control install,
kernel install, and, uh, it's just a new report, right?
And the way it communicates, actually, it does fork things off, um,
because that was easy for me to, to develop, but at the same time, it's true to be able
to just not fork it off, um, have this stuff unprivileged,
to have the backends privilege and use IPC for, as the installation, uh,
a point, right? So, and that is just, it's just awesome, right?
Like, because you can have both modes, right? The simple one,
offline mode, we're just forksing something yourself off, where you don't care about
isolation, and the, and the other one, where you do care about isolation,
just bound into a socket. So, it's, it's, uh, it's also killer,
but it's already advanced killer for the people who, uh,
uh, made the deep dive into adopting this kind of stuff.
Um, that question.
Another problem with the most today, I think, is that he was acting
extra, when they tried to make them shut down.
Uh, you have to do this in a very particular order.
You have to look at the files to be in advance,
so that you are bound to shut down,
and otherwise, you might have some, uh,
most of the messages on the screen.
Was this really pretty good about it?
So, the question was regarding, um,
like, in Diba's, um, shutting down properly, uh,
is kind of hard, because, uh, yeah, you need to tell system D about,
that you're going to go down, because otherwise,
it might reactivate you and things like this.
It's, yeah, uh, it, yes.
So, the answer is, this will get better as well,
because, you know, in the, the, the individual connections
are entirely separate, right? Like, so then,
if the client drops a connection, this will immediately be visible
on the, on the other side, independently of any other scheduling
that happens on the system, independently of system,
as a drop connection, right?
And then you, you can, eventually react to it.
So, uh, subject activation makes a lot of things easier,
because you have inherently always kernel concepts
that are direct between client and service,
or that you can tie your life cycle to, um, and, yeah.
Um, yeah, so they were, uh,
question was regarding, um, uh, uh,
deep bargaining, seeing I have that somewhere, um,
like at the upgrade mechanism, let's see.
I mean, I have so many more slides,
and I don't intend to go through all of them,
because it's, it's, it's so much good stuff in there,
but the deep bargaining thing I have somewhere on this life,
and deep bargaining borrowing is, is really, really nice,
because, uh, you don't need any special tools for it,
because it's Jason, you can just ask trace
and figure out what the fuck's going on.
You know, I have, in my life, uh, uh,
uh, looked at so many traces from system,
the sending deep-up messages,
and it's just like, is it's impossible to digest, right?
Like, people write, like, their multiple tools,
that only exists to deal with, uh,
debugging deep-up and tracing deep-up messages,
and even dealing with problems that only deep-up creates for itself,
like all these latency, uh, context-switching stuff, um,
stuff, and those actually exist because, you know,
you know, you don't borrowing, you just do the s-trays thing,
and it's literally all that, and you can read it without any further tools,
so that is, that is just,
another one of these killer features.
Um, any other question?
Yeah, this is like the two very natural models I mentioned this,
and I mentioned the, yeah, I think,
another thing is like, when you use violin,
it's competal versus rat pool handling and process pool handling,
uh, the difference is basically being just pieces like weights thread
or, uh, library process.
Um, uh, this is also kind of nice, right?
Like, because what I was just telling you about processes,
uh, it also means that you can use,
because processes are a security isolation things, right?
So not, right? Like, so you can basically have it,
so that if a client calls to demon, um,
uh, a violin service, then, uh,
it can be handled process by a process that is
memory separated and UAD separated and everything,
from all the other ones that run there.
Um, so that is like, uh,
that's like, that's really really good, right?
Like, because typically, um,
system services will do operations for underage clients,
and, uh, if they're multiple of those,
they will come with different privileges. So, um,
isolating the handling of it is kind of valuable.
Anyway, my time is mostly over. Um,
I have still, like, uh,
uh,
but that's fine.
Let's just say, use what,
violin since, uh, two five seven, uh,
we have a, uh, a library and system to use that one
if you can see if you do not,
you see don't wrap that, use something native.
Um, it's not very well documented,
but, um, uh, uh,
uh, there's so many examples because
systemy tree is basically full of it.
There's a command line tool.
Yeah. Um, there's currently
some things that are, uh,
uh, admitted in our implementation,
but that's even more in there.
So, that file is good for path passing, uh,
and we also do, uh,
uh, I think, you know,
I originally wrote this live,
actually for another conference.
Um, I think this data is not entirely correct.
Um, uh,
I've said 30 and 13.
Yeah, I think it's actually more or less nowadays.
Um, yeah.
Um, the future is,
Diva is not going to weigh in the system.
I, I don't want to send the signal
that we're not going to ban in all of the old stuff.
I'm just trying to tell people that,
yeah,
the new stuff is going to be rolling.
And we're going to add alternative,
uh,
API for much of the,
the Diva stuff,
but yeah,
the old stuff's not going to go away.
Um,
that's all I'd have.
Let's maybe,
how many minutes do you have?
Like two minutes?
Let's fill that with questions and answers.
You said Diva's is not going to weigh in the way,
but I'm curious to your opinion about,
um,
is that the Diva's is going to be difficult
or a new project?
We don't have,
like,
this is my entire idea.
Well, do you think that they're going to use
types of data?
And Diva's is still beautiful.
Um,
so the question was regarding,
uh, if we should consider Diva's
deprecated,
or if there's a future where it is deprecated?
So,
to summarize,
like, you know,
the one thing,
like,
really the one thing that I still think
that Diva's is going for it,
is the fact that it's,
you know,
actually adopt it,
and there's so much more language
binding,
whileing doesn't compare, right?
Um,
so,
I don't know,
I think Diva's is not great,
but also it's not complete garbage,
right?
So,
I,
like,
so,
um,
if you check a look on it there,
you will do a separate implementation of software.
But I think the official,
which looks like to be developed all
or developer of the time,
so looks like you choose to create your own
implementation.
How do you sort of feature in terms of
tooling like,
are you going to create an idea
from the finish to because looks like it's
manual,
and it has,
I deal.
That's right.
It already has an IDL.
It is that it is there, or by the time they need to.
So, the question is, I'm going to call it divert from the library.
It's good to know.
I think the question was about the relationship of the stuff
that we do in system D from the rolling main repository.
So, first of all, like there is an IDL,
like the way we generate it is the other way around though,
like it's a seasing that we have,
and you encode the IDL and sea structures,
and then we generate the textural ID form from that.
So, we will not have a compiler to turn IDL into into a secret.
We have a secret that spits out IDL.
The relationship in general is that we had some specific needs,
that we wanted to address, like regarding JSON libraries and things like that.
You also need to think JSON, and then make your choice with that.
Memory safety and all these kind of things.
So, it made sense to do this on our own.
But, I mean, I think the rolling is not an isolated thing,
like the fact that the system uses a lot suddenly
will give it hopefully a big push,
but my assumption is also that other languages
should never wrap as devaling,
because as devaling is like it uses sea concepts,
and if you use rust or go or anything fancy,
then you should think twice,
like do not wrap as devaling.
It leaves that as a seasing, and it's very sea friendly,
but for everything else,
I would assume that some other people take over the responsibility.
And there is a community about this, like for rusts, for example,
there is that link.
Anyway, my time is over,
if you have any further questions,
let's talk about that.
Thank you.