The next talk and last one from the bedroom and Martin Luther's going to tell us everything
about magic food and the representation of the container as well as us.
Hello, I'm happy to be here and talk about what we're doing in the confidential
Containers project, I think leveraging a lot of prior art from system D or the whole system
D ecosystem and idea was to present a bit what we're doing there, what tools are we using
and maybe we can extrapolate this kind of architecture also for other similar projects.
I work at Microsoft and the Azure Collinux organization and as mentioned, I'm a
contributor and maintainer of some projects in the confidential container space.
So first context, I think it's maybe not obvious why containers and pods and what does
it have to do with boot integrity. Essentially what we have to do when we want to have
confidential containers or use confidential computing in container context, we have to
put those containers into virtual machines because today the technology for confidential computing
is mostly using virtualization boundaries and Kubernetes container realm,
the unit atom that we deal with are called pods. Those are like co-located processes that run
in a sandbox and they share a few resources, their namespace but usually they run with out strong
virtualization isolation on a note next to each other and what confidential containers has to
do is they have to wrap those pods into VMs. So when we look at the typical container launch,
we don't have to go through the whole state machine here but I think it's obvious to further
there's a lot of complexity going on in terms of the user that is writing their Kubernetes spec
which is like a declarative manifest of which containers are supposed to run which comments
are to be executed environment variables all that. They send them to an API server and the API
server dispatches it to a Q-blet which is Kubernetes component and then there's the container
runtime it's called the container runtime interface so today Kubernetes is not really built on Docker
anymore so there's an abstraction and container D is one of those implementers of this container
runtime interface and then finally container D translates those into OCI those specs into OCI
runtime calls and run C essentially launches the processes that the user intended to run in
a sandbox and for confidential containers the picture looks still roughly the same but instead of
having the run C launching the processes we have a step in between that's called the
catashim that will launch the virtual machine and talk to an RPC or to an agent in the
confidential VM and perform the same jobs as that run C in a user setup would do and additionally
for confidentiality there's a ceremony involved like you need an at a station service you need
hardware evidence from the confidential VM so I won't go into much detail here how this works but
it's essentially like there's more than just in a more ceremony required to launch a confidential
plot then there is in a Kubernetes setup so you need to do a secure key release and you need to
verify the hardware evidence and all that has to be part or has to be bundled in the trusted
that computing base of the confidential VM and if you look at the confidential VM components
there's a part that is dynamic that is where the plot processes run but there's also the static
components I was just talking about that are facilitating the key release that are launching
the containers that are jailing the containers and assigning resources they do even c-groups
in the VM to make sure that there's isolation between the containers in a pot
yeah and for this in this context we will focus on the static components not on the run time
because it's a different piece it's more complex to to measure the run time but for the static
components we could leverage measured boot because the system that is launching those containers
the Linux system it's really just a utility VM that has to be very small for performance reasons
but also because you want to have the TCB ideally very small because you have to trust all those
components in there and we have to yeah the components I managed before like to facilitate
at the station there's a few they're called guest components and we have the agent so the the
run time that is launching the processes and one architectural key property of confidential
computing is that it mandates integrity so it doesn't make sense to talk about confidentiality
when you cannot guarantee what you're running and this is often like a big stumbling drop
as when I when people look into confidential containers they are surprised that they have to deal
with this that they have to first get integrity right and yeah if you want to trust the Sandbox
that means we have to trust the guest rest and all of this has to be measured so they're operating
the guest operating system and firmware components there's options to do this like one of the
options that historically was used with package firmware kernel and the cata agent is bit one
without any root of us and this is rather simple and it's charming because it's simple because
the measurements can be pre calculated rather easily and we have like the confidential computing
architectures from AMD for example they really like this because they don't have runtime registers
really well you can measure it runtime so you you have a kind of safe state that with this
patch at VM launch and this safe state can be pre calculated and this is why this was the initial
architecture the problem problems arise when we see like we now have to deal with discrete
processes like those guest components there's three of those for architectural reasons they're
communicate by RPC locally and so cata agent started to become a process manager like with
and all the problems with locks and everything started to appearing like what does how do we restart
those how do they orchestrated so we have tried to have a discussion about system D how we can
introduce system D into this picture to manage those components because it's also enabled
some use cases so for example we need to have like a one-off job that consumes configuration before
the agent is even started it's pit one then the agent needs to be need also take over this job
et cetera and this really simplifies the agent code and simplicity is really key in those confidential
system because they're more easily you can audit them more easily and it's it doesn't help
if the code becomes convoluted and we can leverage system D's measured boot facilities and I think
to weeks ago so not just but weeks ago probably we got this merged so we have to now cata
using system D as an unit system and we are able to use all the the niceties that we get from
from system D in terms of PCR measurements and one of those tools that have been very helpful for
us has been I'm cozy whatever you want to call it we call it and cozy for the time being maybe
I know yeah but it's I would really encourage everyone to look at this tool I think the motivation
initially was that the project was using Packer Packer changed the license
that was not in compatible with C&CF anymore so it's kind of an external push
to change this and we scrambled a bit and we luckily stumbled over this project and it turned
out to be really really great for use case so it's declarative instead of running a bunch of
patch scripts in a VM you can describe what you want and it's again like easily auditable
you can build images rootlers it supports various distributions we use fedora at the moment but
we also tried other distributions and we also have like for example architectures like S390X
that is using mcosy for those images and it also turned out like it's really fast not just locally
locally it's really really fast because it catches a lot but also on the CI we build images like in the
realm of of three minutes on GitHub actions we this allows us for example to run like smoke tests
against those images that we build in PRs and before they had been like a very long E2E test suite
and this really short and debug cycles and it's a great improvement for what we were struggling with
we also have the setup that the rootFS in our case is really immutable it's we're not
we don't have to change it really the VM once booted it won't be rebooted there won't be any changes
persisted on the VM really so what we can essentially do is just use existing the emberity mechanisms
and this is I think in terms of config it's really more or less just this so we don't do we didn't
have to do anything more and for me at least this was great to see like engineers from actually
implemented this and I was searching for where was the actual magic happening but apparently
it's just that easy at this point maybe a small detour why we talked about TPMs and CVMs
so we can use VTPMs in confidential VMs to some degree today because the CVMs have some
kind of privilege level architecture when they don't have runtime registers it's like a it's
called VMPL for CVS and P that's a piece of memory that is protected from the guest OS which
runs in a lesser privilege level and the host because it's confidential VM so and then
you can bootstrap kind of a playground or you can extend the route of trust into a playground where
the VTPM is running and exposes yeah VTPM APIs to the guest OS and then you can do runtime
measurements and you also don't have to use all the bespoke interfaces from the from Intel or
AMD to record your measurements you can just use existing user land so this is implemented in
paravisers or secure virtual I don't know as VSM it's called it's like a early boot component
firmware is that it's performing those tasks one example is open hdl for Microsoft the other is
coconut SVSM and they're both open source and yeah I'm in the chart that's maybe not really
legible but the idea is that the route of trust is really linked to the attestation key of the VTPM
and the hardware route of trust of the TET and we are using the VTPM then for for measured boot so
there's a nice chart with the PCR registry published by the UAPI group and if you look at it
you see it's pretty crowded already and what we use currently is we use the PCR 11 for UKI's
and we use really with PCR 8 for our configuration measurements because it's used by
grab and grab we don't use at the moment we can calculate those PCR offline or by a software
the TPM also possible we also use system be analyzed a lot because in our case like launch latency
is really critical because those VMs they need to be instrumented or they need to be launched by
the cloud provider for example and then you want to have a fast container or time to put essentially
and so we would go into and see like what is where we have to fix problems so for profiling
system the analyzer was really helpful we are also looking at encrypted work spaces at the moment
to use this so when playing around with this we saw that this is probably also something that
is rather easy to do so you just have another partition that is kind of an that can out of
grow and you just add this piece of code to repart and then you get like an ephemeral encrypted
work space I think we have to probably pondered a bit about the security implications of this
but but from the building perspective it's again super easy with mcosm and system be at the moment
nice seeing yeah there's a few future ideas we want to maybe look into sysx because some
users might have opinionated a base OS like they wanted to do it on rail so we have to kind of
plug the coco ecosystem on top of it and also have it particularly measured
it would also be very nice to have non tpm support somehow in system d maybe this is something
that would be helpful because at the moment we have to inject a lot with low level
Tee hardware in bespoke ways and it would be nice to have an abstraction like
lip tss maybe not lip tss but like on the same abstraction level
yeah and summing up using tools from from system d on the ecosystem really gave us a great boost to
iterate and quickly build on this immutable utility VM and I think this can also be
using other contexts where you have a really static OS image and you run an application on it
I would definitely in the future look at this as a template to go ahead for for measured boot
and that's it from my side thanks for listening
at runtime you mean yeah okay is it possible for the container to contribute something to the
at the station no at the moment no I like I left out the container part so what we measure is pretty
much an abstract notion of the of the container workload but at runtime we currently don't do this
because we don't we don't have the option that every Tee has runtime registers and I think this
requirement will be coming in the in the near future because you have like my rather than starting
engine X containers you run long running machine learning processes and those want to have
continuous measurements so we have to introduce runtime measurements
another question two questions first one is the base in the right there one
building our basic goals it means you don't have to be with the usable image you only have
you still have to trust the build environment yeah basically because you can take the same
yeah I mean this is like this yeah those are the question what they are not reproducible so
we're rebuilding an image like a golden image if you will we're publishing the measurements
for this but it's not reproducible and this is like
given it's for some cases yeah but then yeah there was a completely different from what
produced the video and chaos I just yeah depends on the file system the package management
has to be changed definitely a week old right but before we go we don't have to be in
yeah you have to we have to clean that yeah I have to finish an episode of all package
but it's possible yeah I mean it's definitely something that would make sense but it's
like out of scope for the for the time being because it's deemed not trivial
yeah but next we just learned
no I mean that so the question was do we depend only on PCR 11 so no we had basically attached
the whole PCR quote the whole TPM quote to the evidence and the users can then pick like
what they want we maybe even want to put the lock into the into the evidence so
there's it's more transparent than you can replay the lock but at the moment you get like
all the the the quote collect I think all measurements from the from the PCR
you use like the reference values that are published I mean ideally either would build the
system themselves like on the on their on their premises and then take the measurements
according to the environment because there might be different for example from cloud to cloud so
you would get different measurements from early boot on AWS and on Azure it's not one to one the
same yeah I mean you can do it to a certain degree for PCR 11 I think you can do this but yeah
in general yeah yeah but I mean that's more or less like like you have yeah so the if you're
only right so the argument was if you're only use PCR 11 it won't give you a much guarantees
here so but but the point is that at the moment we're not really in the business of like the
deciding what reference values users want to use with more than one to just move like everything
that we can get to to a validator side so they can make educated decisions based on their setup
that would be for the moment my level of education so you know this is I mean obviously
it's exciting but you mentioned using system the analyze to the look like yeah did you find anything
interesting that maybe it's strange because I want to make our fans faster so I mean that yeah
so whether we found anything when using system the analyzed makes it the startup faster I mean yes
but it was like our own stuff it's we just sometimes clouds of weird ways of setting them
setups up and there's cloud in it and we don't we cannot use cloud in it so this is where we
had us by finding out like when do we have to call home to the provisioning survey etc those kind of
respect what more you're talking to an agent we can say today explain that you have to decide
for the hardware we have to sort the VDPM for the this going to take encryption we know
the laboratories for it is in the case and we have kind of plans to eliminate it okay so the question
was I think the question was and we have to rely on the VDPM for this encryption yeah
so I don't know the context really but I mean it's for unenlightened guests I think this
like unenlightened means in a confidential virtual machine without
any addition that the virtual machine would know that it's running in a confidential VM so that's
why I think the VDPM is convenient I don't think this requirement presents a general
so for us it's just convenient because we don't have to write code I mean this is the VDPM
is present then we can use as I said like a few quantic files and get measured boot
yeah I don't know the context really so that's why
systems going and they just somehow need to get the system the key to the
VDPM and that's where you use the VDPM must have an innovation somewhere else you just
get the VDPM yeah it's too long to to a bit but I think we're out of time so again thanks for
speaker thank you