I will talk about what I learned by writing my own container manager from scratch, going
ruthless.
Some lessons that I learned I want to share with you.
I'm looking at my own 18, I look at it on GitHub and I'm a software engineer in
changing art, lots of open source I like it.
These are my contacts of various things.
So what's little about little is the container manager that I wrote,
because I needed something for my other open source project that I'm on
the time on thing, which is called Disrebox, where I needed a full-back solution
from when our host doesn't have podman or Docker for some reasons.
And I wanted something that was self-contained, so easy to install.
Ideally, just a single binary without any other external dependencies,
being like wait in the sites.
And as fast as possible, I didn't need everything that podman and Docker does.
So just the bare minimum that will let me boot up simple container for my
other projects.
And also I wanted to learn about containers, improve my go.
So that's what I wrote.
So what are containers?
Containers are a way to visualize a system without
recurring to emulation or ritualization of unhaul OS.
So in a VM, you have your host operating system running an
hypervisor, where you run other operating system from bootloader up,
and then in that operating system, you run whatever service application that you need.
With a container, you have your host operating system,
and then you have, you use your operating system main kernel to isolate other root
offenses.
We will see what they are, but basically other file systems where you run your apps or services.
As you see, it basically removes a whole layer of,
between the host and the guest.
That's a pro because then it's very fast, very light, and it's easy to
dispose of containers, just scrap them, create them.
Very easy on the security side.
You're sharing your kernel between your host and workloads.
It hasn't implications, but we don't care about that now.
There are some building blocks of containers.
So you have our root of us or base file system for the container.
You have namespaces, which is how we separate it from the main operating system.
Capabilities, which are what stuff inside the container can and cannot do.
Seagroops, it's a way to conceal and separate resources.
So I can say this container cannot take more rammed out a set value.
Second filters, even more sandboxing.
We can filter out a set of c-scoles there.
We can deny to a certain workloads.
And then integration with BIOS system modules,
the Linux app, or whatever.
The first building block we will see is the root of us.
It's the BFI system that is used by a Linux user land.
In the case of Lilipod, I wanted to tap in the OCI registers,
because this is the most diffused and biggest one where I can find everything.
There is Docker Hub, Clio, GHCR, blah blah blah blah.
Many of them.
When we interrogate OCI registry, what we get is a manifest.
We can see with a little Docker manifest inspect of Ubuntu and Genx image.
We have a set of layers as objects in this JSON.
These set of layers will give you a way to download this layer.
And layers are shipped as tarbles.
And you will have the checks on the layer.
So you can always verify that don't have downloaded a corrupted layer.
This makes it also easy to do that.
So what I did with Lilipod is to use Crane as a library.
It's very handy to interface with OCI Container Registries.
You can pull down the manifest and open it and basically go and download all layers.
What I do is use the checksum as a way, obviously, to know that I downloaded something not corrupt or whatever it was.
But also rename each layer as the checksum itself.
So for example, if I have an Ubuntu image and Ubuntu and Genx image, the base Ubuntu layer will be basically named the same.
I will know it and I can use something I like hard links to deduplicate between various same layers.
So you have a storage advantage.
So how we can use the root of fast.
You can see it's root in that.
See it's root is very old.
Unix is called the lets you change the root file system for a set process.
You can basically enter it and basically it's the new root of that process.
It's good for recycling file system access.
I don't want, I don't know, this process to access my whole host or file system.
And it's useful to bring your own dependencies libraries and stuff like that for asset process.
So we go and see a root, but that doesn't work without root.
Permission the 9 in operation or permitted.
Because see a root needs us to be root users.
And for convenience, we also want to mount additional file system.
For example, a C-Sopherex FFS, a ProcFS, various tempFS or stuff like that.
Well, I want to mount something inside.
What we can use is to root less is the other building block of the containers,
which are namespaces.
So namespaces are set of is a technology provided by the kernel itself,
to have some sort of isolated views of resources only for us a set process.
There are various ways, various types of namespaces.
And it's basically how containers contain, basically.
So we have the mount namespace, which basically gives the process a local copy of the mount tree of the file system.
So it can be manipulated by the process without affecting the mount tree of the whole system,
just for the process.
Same is for the users.
And UTS is for host names, PID, IPC network and timeless basis.
I think the newest one.
So what we can do is call the unshare C-School to fuck the process in a new namespace.
In case of C-School, we need the username space and the mount namespace.
And in that name space, the process that you are launching is able to change the mount tree and the user tree,
because it's just a local modification that doesn't affect the rest of the system.
This is a little example.
When we unshare and where we clone, we can, in the new namespace, map the user to something else.
Like, Alice can become UID 96, WLB data, or Bob can become FOO, number 1000 and stuff like that.
For C-Truth, we just need, for us, Alice to become ROOT.
We unshare the mount namespace, the username space, and map ourself to the ROOT user.
We see it through to our thumb, sorry to our ROOTFS.
And success, we are in our new, very, very rudimentary and simple container.
In early part, we are using the C-Sprocassibutes to unshare the various namespaces and because it's configurable,
we can share something unshare others.
And you will see here that I'm not using C-Truth, but pivot route, which is another C-School,
for similar userfulness, because C-Truth can be escaped easy.
With that simple line of code, you can basically escape any C-Truth.
Because the mount tree inside the C-Truth is not changing.
With pivot route, it's different, because it can leverage the fact that we are in a mountain in this space,
that we can manipulate as we want, to really switch the route instead of just changing it,
and then remove the original route of FES, so it's not accessible anymore.
So you can just escape by C-H-T up up, right?
How it works, so we stop with our route of FES on slash.
And we have a new route, which is what we want to pivot to.
With the C-School, we switch the new route with the old route at the same time,
and you're left with only the old route, and the new route FES is the new route that it was before.
But we can leverage the fact that we are in a mountain in space, and we can unmount the old route,
and it disappears, so it's not accessible anymore from the process inside these namespace.
We go for it, and for example, in Ubuntu, we have problems.
So what's happening here?
It cannot set groups, it cannot set GID and set UID,
and there is these underscore APT user inside here,
because APT uses a sandbox of its own to download stuff for security purposes.
And here we have set groups to deny it, and we have only one user in this UID map.
We only mapped ourself to route, and we didn't do anything else.
So what we need here, we need to be able to map multiple users and to have the set groups primitive enables.
We cannot do that.
The only way to do that is to be route.
So we have to use a little trick, which is using these new GID map,
a new UID map tools that are in the shadow packages.
Those are being that only route can do this stuff.
Those are all set UID binaries, so they actually run as route.
But it's just for a brief moment, only for one thing, which is only map something for the child process.
This is the unshared process.
It's a very small tool with very little codebase, so it's easily auditable.
And it has security checks, where for example, new UID map and GID map can be called only from a father
to its own child and knocking else.
So I cannot just change the map single of whatever PID that I want,
just the father from the father to the child.
So what we will do in the host name space, we have our main.
We clone and share to a new name space, and we have the child process.
Immediately we use our helper that launches new UID map to it.
So it has the right maps and mapings for users, set groups and whatever.
Then we do the pivot route and run our entry point.
It's very basic, so we are literally calling a shell command to it.
That's the command that is run.
But when we do that, the mapings here are changed.
What are these numbers? Let's take a look.
So the first number is the start of the range of ID inside the namespace.
This other number is the start of the ID outside of namespace, so in our host.
And this is the range, so how many of them?
The first line is whatever we also had before, so it's mapping from user zero inside the ID, so route.
To the user 1000 outside of the namespace, so ourselves most likely, for our range of one.
So it's just one of one mapping.
User zero to 1000 outside, so we are actually route inside the namespace, even if we are not route outside.
And then it says from user 1 to 65,000, whatever.
It maps them to the user 100,000 and more.
User 1 inside the container will be 101 outside of the container and so on.
So it's very far apart, so it doesn't interfere with real users on the host.
But we have actually a mapping of all the users to real users on the host.
And now APT works, we have set groups.
We, the underscore APT user can run, so you can download packages and it's happy.
The same thing is done for, can be done for the PID namespace, so I want to make sure that the process inside our unshared namespace cannot look to other PIDs outside.
So it can interfere, I don't know, with my own processes, or with the real PID1 or P22 or P23 or whatever.
So it clones the process tree and maps it back to 1.
So then all the trials of 1 will be the trials of 65 here and our map to 2, 3 and 4 in the PID namespace.
And the same can be done for the network namespace.
It's a little bit tricky here because when we unshared the network namespace, we are just left with localhost and nothing else.
So for now, for now, LilyPod doesn't do this type of networking, but you can create then bridges and interfaces here and here and then control the network access of the namespace.
But that's not enough. Then we have capabilities. So what our capabilities is, there were introducing kernel 2.2, so it's quite old.
But basically, before kernel 2.2, Ripsuser can do everything and non-Rootfuser can do just a subset of things.
After that, they basically split all the privileges that the root user can do in little capabilities that you can also have.
And right now, what's happening is I have this set of capabilities and when the container starts, it's interacting all of them.
And that's dangerous because capabilities is a way to access a set of C-Schools.
For example, with the C-School module, you can mod probe stuff. With C-School, you can see it through it.
In P-Trace, you can P-Trace, or you can see it through own and dangerous stuff. And we don't want that because this can lead to escaping a container pretty easily.
And so we want to drop whatever it's not needed to run our own container.
This is a list of capabilities that I very much copied from Docker.
It's very restricted set of stuff, but that lets you do whatever it's needed to enter a container or start whatever you have inside.
And we can drop them, and we have less now, so it's more secure.
We're almost there. So we can pull from our registry, we get our own TIGGs, we create our wrong root of us, we can run it, we are rooting side it.
We cannot see process outside of it. We have the right mapping, we have the right set groups, we have the right capabilities, and we don't have network.
Like we wanted at the beginning, but that's not enough to have a real secure container.
We are still missing C groups, so we can limit the resource access of a container, we are still missing second filters to limit the access to C schools that the container can do.
We are missing a Selinux or a Parmer integration, so that we can go even further beyond.
This is the repo on GitHub, it's located on LilyPod, and thanks.
How'd that any questions?
There's one.
Okay, so I'm pretty curious, you mentioned that you needed root access for this helper, you know, for a group mapping, and so do you know how projects like Podman do that?
The same thing.
So they use new idea to map them, just for that brief moment, and then they are rootless, others otherwise.
Actually, it's not true, we released.
It's rootless as much as it can.
Actually, just making a quick comment on that one because it's kind of funny, so Alex was standing there and myself, we've been working for probably a couple of years now on the fully isolated privileged username spaces.
So with that, every single user on the system will be able to get an entire, well, an entire 42 bit UID and GID map, we're not needing the privileged helper at all.
It will still be a tricky for the file system access because you need to map that, but like you'll be able to do a few, or time affairs, or whatever, and get removed that path completely.
I look forward to it.
Hey, I have a question.
Basically, do you plan to add any distrobox functionality that can be used only by LilyPod instead of Podman?
I want to keep this to box container manager agnostic.
So there won't be a special treatment for LilyPod.
It will just be the fallback because it's like a 8 megabyte binary without external dependency outside of the basic c-schools that we are making.
So this is the fallback solution for when I cannot find Docker or Podman.
Thank you.
Anyone else?