So, hi everyone, I'm going to talk today a little bit of the amortigration system D. I'm
mostly talking about a bunch of loosely connected features. I'm a true half an hour is
actually going to suffice to go through all the slides, but that's fine. We're just going
to cover as many of the features that we can. The more important ones at the beginning,
the less interesting ones are at the end, so it's certainly fine if we miss some of them.
I usually like my sessions to be interactive, so if you have any questions, feel free to
interrupt me and we'll get to answering them right away. I'm much prefer that we're doing
with all the only at the end. So, yeah, I'm now part of the thing I work at the start
of called Immutable, that we recently found it, yeah, let's jump right in. So, system D, what
was that again? I mean, this is the VM devroom, so I guess system D is not really known for
being a VM thing, but it's a very quick introduction, but I hope it's actually unnecessary,
but yeah, it's a service manager, it's a system manager, I like it not only manager service,
manager system at the whole, and you can see it as a collection of user space components
and some of which we'll talk about in this talk. So, the first thing, system D as a guest,
right? Like, so if you're on system D inside of the VM, there are a couple of nice things
so it can integrate into the VM environment. The first one I want to talk about is, yeah,
simple detection, if you're running in the VM environment. This is exposed in two primary
ways. There's one tool called system D detect word, and if you run it, it just tells you,
I'm running in QE, or I'm running in Hyper-V, or whatever else you have there, and there is
connected to this. There's a unit file setting, it's called condition virtualization,
which allows you to condition certain services to the VM environment you're running in,
right? Like, so you can say basically say that the agent that is specific to some VMM,
only runs inside of that VMM. That's very basic. Yeah, I think it doesn't really deserve
much more explanation, but one thing I would like to know is that, yeah, if you hack on a VMM,
in particular, one of the newer ones, the fancy ones that try to do very, very lightweight,
please implement a mechanism so we can actually detect, but we're running in such a VM,
usually this is done around CPUID, leaves like the specific one, and via SM bias or
device tree, please make at least one of these rings available, because otherwise it's not realistic
to detect that we're running in a virtualized environment. So much about the first thing I think
it's very trivial. Well, let's talk about the second thing. It's a SM bias type 11 stuff.
A ZM bias, I hope, does everybody have a rough idea? Does anyone have not an idea what a
SM bias is? Oh, there are people. Okay, SM bias is this thing. It's like a relatively large data
structure that has been around since bias times, like for long before you if I, which describes
the system, and it's going to be like parameters of the system and passes that to the west,
so that the west can figure out what kind of hardware is there and a couple of other things.
It's the thing that you see if you type DMID code in your Linux shell.
There's one particularly, a particular kind of object in there, which is SM bias type 11 objects.
The type 11 objects, I think they're called vendor strings or something like this,
it's basically a way how arbitrary stuff can be passed into a system.
This is particularly interesting for VMMs rather than a physical device,
because yeah, it's a way how they can parameterize VM, and a really, really efficient way.
Right, because it's just a piece of memory passing. There is no,
like I mean, the traditional way is how you provision a VM system with things like cloud
and an IMDS and an emulated city on drive, so it's all horrible and slow required as the
HCP and is like this massive stack of things. This one's super easy, right? Just some memory
to pass through there. There are free form vendor strings, the ones that a system needs
have a specific form. We'll come to that, but yeah, they are just memory, they're simple,
they're fast. If you want to use them with key, when you just type something like this,
it basically just says, yeah, set another SM bias type 11 object. The value here is IOD
or system need a credential that's simply like how we want them to be formatted so that
system consumes them, they can assign variable food to value bar. For user space, like from
inside of the VM, they show up in a system as file and we can just read this and that what system
it does. There are a couple of them that system defines by default, like the bootloader,
like system to boot for example, allows you to select an additional kernel command line options
or additional boot entries, even the boot menu, or you can tweak the log level with that,
the bootloader has so you can actually see things, can configure the time out and think like this.
Most interesting one is this one, IOS system, the dot credential, which allows passing in
system credentials to the VM. That's a really powerful feature, like that two ways I can do this,
either like there's a simple text form, or there's one that takes base 64 so that you can
pass it arbitrary stuff. To understand what this actually does, let's have a small
admission here about system e-credentials. This is a decredentials as a concept we introduced a
couple of years ago to system the, which is how you're supposed to parameterize the system
as a whole and the service is in particular. It's our answer, you know, the web world likes to
parameterize the containers with environment variables and I think it's a really stupid idea
because they are by default propagated and there's no access control on them, like everybody can
read them and it's just a terrible way to propagate secrets because it's very, like by default,
propagated and very open. So we created this alternative. It's basically small pieces of
information that like we expose ultimately as little files and can carry both secrets and regular
data. They inhabit, in heritable down the tree, like she can inherit them from a host and to a service,
but you can also, from a container manager via that, m into a host and to a service so you can just
propagate it always further down the tree. We define many of them that are well known, like every
service kind of find their own, but a couple of them we define as the vocabulary that always
system the understands. One for the root password for the host name and he thinks like everything
you kind of want. From the service point of view, they show up in a special directory where you get
a environment variable which tells you, look in this directory and you find the files and they'll
have that name. They also have, yeah, so the propagation is always opt in as opposed to environment
variables where the propagation is usually by default and access modes apply. Meaning that, yeah,
the we when we a service gets a credential passed in, we'll make sure that these files are locked
down to that only that service under its user ID can access them and not every everything else.
It also has a hook up to the local TPM so that you can have authenticated credentials like
you can have them encrypted address, you can pass them in to your VMM encrypted, but the moment
the service that it's actually supposed to consume them is activated will unlock them and pass
them to the service and play text. It's our answer to, yes mentioned, environment variables
into these awful provisioning systems that are everywhere like clouded and sync like this,
which have these massive stacks of things. This one is really simple, really like clean in a way
and has all these nice properties that we like. But this was just an information, right?
So if you take the SM files type of live and stuff, you basically that's now way how you can
pass into VM, all these kind of settings, naturally an any system system already out there
and we'll consume them and set a root password and whatever else. Next thing, any question to this point?
No, okay, so the next thing is SD-95 style notification. So SD-95 is a concept in
system deep how originally services could tell the service manager, like system deep,
that they have completed initialization. So this is required to
have a clean boot up, right? Like where one service is started and then another service is started
that might depend on the first one, you need to synchronize this properly so that the first one
has finished thought up so that it is connectable so that the second one that wants to connect
it can do this. So we created SD-95 for this is it's an extremely simple protocol basically
where the service just receives an address of an a-funic socket and then the service just sends
a message to this. It's so basic and so simple and I think an allergen that we just decided to
extend this so that we can also do ready notifications of the VMs as a whole. VMs of course do
not have a a-funics, they have a a-v-zock. So this is what it really is. Like we have like the VMM
can set up an a-v-zock, a socket listen on it and then the payload can send a message when it's ready.
It's extremely powerful. The way how a VMM would communicate this to the payload is by
again setting a system credential. The ones we were just talking about is called VMM.notify socket
and just sets the CID like the a-v-zock address that shall be a cell receives a message and
then we'll connect it and send the the message. This is not just a lot simple ready notification
just about like like as you might think that it will just report when the boot of is complete.
It actually sends up a lot more because it will tell you about like how far the boot has already
progressed. Specifically there's one message that is sent if you have SSH for a sample us up because
for many use cases all you need right like you want to know the moment where you can connect via
a-v-zock to the VM via SSH. We also announce a little bit of meta information there like the
host name machine ID. So this which is sometimes really nice because that means that the VMM
is not as much as the black box to the host anymore but they I mean typically they know the CID
of the VM but with this they also know the host name machine ID really basic information.
There's also announced is a shutdown detail right like when the system goes down again we can
propagate something like an exit status of the VM so the VM starts to feel a bit like a classic
process where you can do exit seven or something like this and this propagated up to the VMM and
can see this. There's also like you know the Linux says this reboot string that you can have
which is used by eminent hardware usually but we generalize this and also propagate this up so that
if you have a VM the other also ends up so it's like a textural form of of an access status that
that you can propagate to the VM of VMM. Questions at this point?
The question was regarding how this information is accessible in the host well the VMM receives
it and how the VMM makes use of this is up to the VMM right so my assumption is which is
provides some API but that's specific to the VMM outside of the scope of systemy.
The next thing is systemy as a guest is we recently added the same where if we like there's a
little generator and systemy generator are these little binaries that can basically extend the
dependency tree of systemy dynamically and locally based on configuration or based on what's
available. There's one that checks if AFV is available like if you're running in a VM where
AFV is enabled then also checks if the SSHD binaries are running in if so it automatically
binds SSH to AFV is. It's extremely simple it's extremely powerful because it basically means that
if you're running VM and you just enable AFVsock you can just connect it without any kind of
network configuration it's very fast very simple it's as secure as SSH is secure and it makes it
so much easier to debug things because you do not have to rely on the unreliableness of of
networking to be able to talk to your VM. And remember we have notifications when SSH is up right
like so you can make use of this without anything like as long as UVM has SSHD around as well and as
long as you enable AFVsock you will get another location basically help it's connectable now
and also remember via the credentials stuff you can also provision SSH keys into the VM so yeah
really really nice way very simple way very fast way how you can configure the SSH inside of it
get notification when it's ready and connected just to make a point here none of this like
the automatically disabled all this logic and confidential computing because then it's more
complicated situation which should maybe not make all this stuff available so that the VM
can basically control access to the VM so the provision says there so that we don't do this the
so we only do this basically if the trust model is the traditional one where the payload has
to trust the VM and then full. Any question at this point okay one more thing here.
SystemD boot you know it's that bootloader of systemD or it's it's actually more of a boot menu
the support has been added to to make it possible to embedded in the UFI firmware
that's really nice because it means that the images that you boot inside of the VM do not have to
contain any bootloader anymore but you still get bootmen you get boot counting you get all this kind of
interactivity but also control an API towards the bootloader pick something without actually having
to include that in the image. This is like the way how you use it risky when you just pass
it by dash kernel and specify the the EFI binary it's not a kernel right it's just how
you can expose this. It will still search the ESP for kernels and boot for minutes it's actually
very nice because that makes means that they discover that the insert just really has to have
the UKI and the root file system. Any question at this point? Next thing you know when you
deal with VMs you also want to have logging so a system to journal which is like the logging
for a framework of system he has this option called forward to suck it initially it was about
forwarding log out for automatic to some IP address but you know what it actually also supports
a WISOC these days. So this is very simple very natural way how you can get live access to the
logs of the VM and you can configure this again via system to credential so you can basically
start a VM and say I want my logging to get out on this AVISOC and you immediately see everything that's
happened. It's simple it's nice it's very efficient very fast very early in boot I mean it's not going to
cover the kernel stuff I mean not at least lives the kernel stuff because after all this is a
journal configuration will only get into effect one's journal these up but yeah. Question at this point?
No. Next thing you system repart this is like this dynamic partition thing and system
need so this one didn't really have any thing to do with VMs but I think it's very useful when
you deal with VMs to know about this system repart is a dynamic repetition and this is particularly
useful in VMs because very often you have like a golden image that you deploy on VM and
then boots up and sets up the disk but sizing the disk is something that shall happen locally
on the individual node way to deploy the thing which system repart you can do is very easily because
it basically means that you can just have to increase the size of the host and they can
let the system repart to the rest where you bump up like it goes the file systems are just
going to create them with keys that are specific to the boot of that VM and things to the actual
disk that you provide it to the VM it's really nice. Again this is not strictly using any VM
concept but it's extremely useful in VMs. Question at this point? The question was regarding which
file systems and it's a typical Linux file system xd4 interface and all these kind of things.
So the question was regarding if it also covers LVM? No. Not a fan of LVM.
What is the question was regarding what is expected from the petition layout of the image that is
booted? Well it has to follow the boot the discoverable image specification which is something
that we put up on your API so you basically have to tag and you don't directly have to that but
it gets very painful if you don't but you have to tag your petition as this is a root file system
for x86 or something like this but it's very like if you are preparing the ms in 2026 there's
a good chance they will be put together like this anyway. Next topic it's second with auto and
if you run a VM and sometimes useful to have like this tofu concept where the image that you
boot you just accept but then you want to want that in specific VM instance so that from that
point on it's only this image that boot inside of the VM and nothing else. And system that we have
since a while a concept of auto and rolling of circuit boot that basically means that if you boot
up a VM was a UFI circuit boot set up mode then the boot loader that we have system to boot
can automatically enroll a key in the db like in the keyring of a circuit boot. Lots of system
down reboot and then goes through. This is extremely useful in VM environment it's not so useful
in physical devices because in physical devices replacing a circuit boot pieces kind of problematic
because you don't know necessarily what kind of hardware you have and if there is a I don't
know Nvidia card in there that really requires signed firmware drivers and then if you kick out
the keys and everything breaks but in the m environment it's very very simple very robust because
yeah the VM m controls what the firmware is and it's typically very simple there are no
extension cars nothing so it's a it's a really lovely concept right like so for me if you prepare
this image you just have to put the circuit boot signing keys that you want to be added in
a special directory and then when a system would seize them and we'll just and figure out that
the system is in setup mode so that it kind of rolled them and we'll just enroll them and reboot
turn off circuit boot set up mode and then you're locked down. It's something I can just recommend
everybody to just do like it's so simple if you build an image just do this because I mean
I personally believe that the value of circuit boot is relatively limited but in this case is really
nice because it's so easy to do and it basically means that you walk down with the m to the
payload that you passed it and if an attacker gets access to the m they cannot use a completely
different payload like I don't know it's install so it's just because fedo I was first booted.
Question at this point? Okay there's more there's a hook up with cocoa I'm not going to go
into this hopefully soon we'll have like the generation ID so that system actually has an
instant and an understanding of when snappers are made and things like this but yeah this doesn't
exist right now. Next I'm going to cover a couple of topics not so many about a systemly on a host
meaning that yeah you have VMs running on the host below the host and what systemly provides
to make it nice. Interfacing with this the first one is something called systemly SSH proxy
it's a trivial proxy for SSH so that you can actually connect via the SSH client to a VM that has
a v-lock and has SSH listening and it's basically the other side from this SSH generated I was
talking earlier about because SSH natively has no clue about how connecting to a v-lock works
on this add this in it's called proxy it's actually not really a proxy proxy kind of suggests to me
that it would actually shift bites around doesn't do this all it does it's a very very short
lift and is a way how you can tell SSH basically when you connect in a v-lock it will return the
actual transport socket back to SSH and from that point on it's out of the loop and just exits
so yeah name is misleading the reason why it's called that way is because SSH names the
that way not because what make any sense it's enabled by default if you have a current systemty
question no question there's one more thing we recently added something called systemly and as
resource the it's basically I mean it does a couple of things like the main purpose of the tools actually
to hand out a username space well I mean you give in the username space which is this Linux
isolation thing that's mostly interesting for containers again it's also interesting for the
ms but it's yeah people don't think so much about it there and it gets a user ID range assigned
but one of the things that you also can do it's can then if you have that assign an app work interface
to it and allow this for unproveletch clients so it's it's very simple while you just do an IPC
call and you get a follow-up to a tap device and then you can pass that to key email and it just works
and there's network d behind it so you actually get an DSCP address and things like this
it's my personal answer you know they're like people always do like with key email like this
user space networking so that they have unproveletch reactions and things like I fucking I don't like this
but this is like our alternative to this you actually get real networking real DHCP was routing
and stuff like this and just works the back end of the networking practice system network D built on DHCP server
blah blah blah blah blah a result is solution as well right like so if you if you have a name
assigned to your VM then you can actually resolve things of the appeared versus go go there
so the question was regarding if we can also do the dynamic VTAP I think I edited for I feel like
I added for one more I don't actually remember which one but if it's if you needed it's
trivial to add right like because we have the code anyway it's just talking them up also an
resource team then this is kind of the last thing I want to talk about that's also
system the machine which simply keeps track of your local machines just I mean it's also
tracks containers but it basically just maintain the list it's very simple it's just you
were just your stuff was it and that makes things like the CID like the AFVs are adders and
think like this available to systemly and we can use it for various things for example system
SSH proxy then can ask that thing for a name so that it can find the CID so in a way it's
acts a little bit like the DNS but for AFVsock on an in a local scope but it's also kind of
interesting because it means then when you can do PS with some options you actually see like
which key move instance actually belongs to which VM it's like this so the less tracking there
this is the very last slide with content any point question at this point I'm kind of impressed
that we actually managed to go through so I didn't expect that this is the last thing like we
added something called system VM star is born to systemly while back we mostly did this for our
own testing stuff because you know all the stuff that I was talking about earlier we got in
integration as these credentials and things like this if you use rock you email you always have
to put together these complicated command lines it's not user friendly it's very you can do it
but it's requires a lot of typing with system the VM spawn what we did is I'm not sure if you
guys know system the n spawn which is like this little container manager that we have in system
d and I don't know I like using it it's like the interface that well I wrote it so
I'm used to that thing and I think it's very user friendly so system VMs won't just
exposes the almost the same interface but it actually just folks off key and does all the parameters
if you want to do ss h stuff and credentials and things like this it will just do the right
thing for you hide all this and just rep a q e-mail register it nicely and things like this so
yeah use it it's mostly not for like production stuff it's mostly for local stuff and that's it
do we have time for question no it's questions one two questions
as a question sorry I don't
like the question was regarding if we can take the a a view of support that we have in
journal d and things like that and make it available for any networking stuff
I mean a system d has it like the socket unit concept and it can listen on a view of
unique sockets and it can listen on IP sockets but it can also always since years it has been
able to listen on a view of sockets so if your application is generic enough that it doesn't
really care about the family ss hd for example is like this then yes you can just use it in
total generic and find anything to it like like a shallow whatever else was another question
sorry I don't
so the question was regarding a system d as a guest if it can be used without root
like I mean if you want to run system d inside of the m but not give the m root I don't understand
the question because yeah
okay so if the like the the functionality that we provide inside of the guest is that can
me like this really depends on what you're talking about like connecting outside
like mostly yes depending on what the kernel policy or things are like on on Linux
like everybody can bind if you like but I don't have any time anymore if you have further questions
let's do the south side thank you very much