So, hello everyone, my name is Haralambos, but you can just call me Babies, and today we'll
talk a bit about how we can run these applications as Linux containers.
A bit about us, we are a small company, we are working in various research and commercial
projects, and we are actively contributing in various open-source projects, we are the
core maintainers of urnc, which is a senseiF, some of the project, and we focus
on what we call systems software, hypervisors, container untimes, lightwaters,
utilizers, and in general hardware acceleration, and also about IoT devices on the cloud.
So, I know this is a controversial topic, so I want you to be prepared, we will talk
about Linux and Kubernetes.
No, I'm kidding, we will not talk about anything.
Okay, let me try again, no, it's okay, I'll try to do it like I have here maybe this
light second.
Okay, so, we'll talk about BSD, and we know that there is, of course, we are in the BSD room,
I don't have to convince you that BSD is an operating system that works to look at, there
are specific use cases that BSD is widely used, various distributions of BSD, but the
thing is that what we want to look in this talk is how we can bring BSD in this picture.
So how can we run BSD applications and managing through Kubernetes and over Linux, because
either we like it or not, this is the stack that currently dominates the cloud and most
of the servers that are out there.
So, currently we have various ways, it's still possible to run BSD on the cloud.
Of course, we can have the option of the dedicated server that we just have one any kind
of server that's just dedicated for BSD applications, and then we have to do various networking
things to try to connect it with the rest of the cluster, for example, Kubernetes cluster.
And of course, we also have the option of a virtual machine with a cube variant, we can also
use this virtual machine through Kubernetes, and we can manage it like that, so we can
again have BSD running in Kubernetes over Linux.
But with a worker from Emile, with a worker from other people from FreeBSD that have managed
to make a net BSD and FreeBSD run as a micro VM and boot that fast, there is a new thing
that we can do.
We can run the most microservice.
So the idea here is that we still have the same stack, we have the Kubernetes, we have the
Linux, and then we can have the BSD running the application that we want us a microservice.
And Emile saw that it's so small and so fast that we can boot, so it's and we will
also that this is actually possible.
So the thing here is that what we want to do is that we, of course, have the option of
a full virtual machine, but sometimes we just have a few small applications that we just
want to execute and we want to manage as an application, like a container, what happens
for example.
So in this kind of scenario, full virtual machine doesn't really get us there, because
the full virtual machine will have to need to have a full BSD operating system running
there and as a result, we might have a lot of unnecessary services, but run on the same
time with applications, stilling resources and yeah, also increases the maintenance cost.
But what we do in UranC and the with other kernel shelter is that we have the concept
of a single application kernel.
So a single application kernel is just a kernel that is configured for a single specific
purpose.
It can, it is supposed to run only single service and this is how the root of S is built
only just for one application.
And as you can understand, this has a lower resource usage, less noise for mother services
that don't exist anymore.
And we just need to maintain only when it says everything that we actually require for this
service.
But there is also the case of unicarnals.
So run run based on it is D is the most, it's one of the first unicarnal frameworks
that existed and anti-hatt made the great work on that.
So the concept here in that is that we have a specialized kernel, a specialized library
operating system, which we can link directly against with our application and therefore
we have everything under one binary.
This is extremely small and extremely fast.
We don't have any separation between the user space and the kernel space.
So everything is just a library call, we can boot very fast and as you can understand,
it has much better improvement in regards to performance, resource consumption.
But we have to face that it's not the easiest frameworks to work with.
Unicarnals require a lot of time to port applications on them and the benefit of
specialization comes with a cost of porting applications to them.
So this idea sounds good, but the problem is how can we build these things so we can use them
as in Kubernetes and how we can deploy them later on Kubernetes.
So the concept is pretty simple, we'll just build them like containers.
And what we mean build them like containers is that we will have to pack it as an OCI
So for that purpose we have implemented bunny, bunny is a new framework for which is based
on build kit from Docker and it allows us to build any kind of kernel, any kind of
library operating system in the same way that we do with containers.
So basically we have a container like experience, we can do a Docker build and this will build
the kernel for us.
The purpose of bunny is to provide, let's say, a uniform experience for all kind of
library operating systems that existed there, but even for generic kernels and to try to
make this kind of dependency hell that we have, for example, with different frameworks,
different kind of operating systems and the tools that we need and everything it tries to make
the cost smaller there.
Now how does it work? So everything is declared as a YAMO and bunny reads this YAMO
and based on the framework it has to build, even if it is a Linux kernel, if it is a BSD kernel,
if it is a Rampron, if it is MirasOS, whatever it is, it fits the building layers,
this can be the tools that will be needed to build this kernel or it can be any other kind of
utilities that are required, it will build the kernel like any other kind of system and later
it will package everything as a NOSI image. So the packaging takes place depending on if we have
a kernel or if we have a kernel. So in the greening things it is what we have in the case of a
unicarnal, because in the unicarnal we only have the kernel, the libraries in the application
are pockets only at once, squads everything is together as single binary and then of course
we need the configuration files for our application. Now in the case of single application
kernels we have different kind of layers because we also have the dependencies of the application
and the kernel that we need to execute. So everything's all these things together are pockets
in the OCI image. So let's see how this works. So we have here for example
so I'm here and for example I can go here in small bsd. So we have here for example I have already
got the kernel from small bsd and I also have the base image that from again from small bsd.
So this is the isle image and the the block file, that's all the block and we have a two ways
that we can declare the how we can package this thing. The first way I hope it's visible now.
The first way is just using a traditional Docker file way that we just copy the stuff inside
and we just have to specify some annotations for the runtime that I will explain later.
And the other way is with the bunny file that explained before that we have here for example a
yamu file and we declare what we want to include inside the root of this. So here we declare that we
have a kernel that exists locally and it's a net bsd. Here is a path of the kernel. We declare that
we have the framework that we want to use this net bsd. It will run on top of chemo and we have this
root of phase that we will build from scratch. It will be a row which means that we will just copy
everything in the OCI mods and yeah I have already created here I have mounted the blocker that I
said before. So yeah let me try and build this for example. So what I do is just a Docker build.
And this will fetch everything that is required to create this kind of OCI image for us.
Now it will take a lot of time to okay a lot of stuff we're cast so we didn't build anything.
But we put statements here and we have this kind of image that we will deploy later.
So let's go back to we will play a bit more later with that.
So the question here is okay now we have an OCI bundle. We have in our OCI mods we have the
kernel that we need. We have the root of phase for the application. We have the application. We have
everything. How do we run this thing? Because we cannot just do it with Docker or something like that.
Well we can. We can do that with currency. So currency is the and you continue run time
focusing on Linux and it tries to be the runc for unique kernels and single application
kernels is a sensitive sample project as I said before. It's a here I compatible run time.
So we can it's written from scratch. That was a bad idea we tried to change it.
And we the goal here is to support a variety of guests and monitors. So we can have source for
base monitors. We can have a virtualized base monitors like TEMO like that they use KVM but also
software based think about it like in the divider if you're familiar like using second filters using
some more restrictive software based things. And it's very extensible so anyone can easily add
a new monitor as anyone can easily add a new guest. So the key differences of year and
second pair to other kind of deployment models that we have is that in case of runc for example
in actual container. The application is just executing inside the container. Runc just creates
this Linux container using namespaces and figures. CubeVirt is a different kind of philosophy
that it deploys a full virtual machine for us in Kubernetes inside the pod. Then we have a
case of cut-a-container which is a sandbox container on time. And the idea here is that cut-a creates
a virtual machine inside this virtual machine it will create the container. It's more for security
purposes. And then we have the case of here and see which takes a completely different approach
that we create the virtual machine inside a container. So inside the Linux container we have the
virtual machine that will be executed. And inside this virtual machine we will have the actual
application like a microwave. But only for specialized purpose. So how does this work with BSD?
So what happens in the case of BSD is that we can have for example runc run which is based on
the BSD as a unit kernel. So everything is just packed in the kernel. And we also have the case of
for example small BSD that we saw before that we have the root of FSD and the kernel that
it's packed in the OCI. So what Uranc is doing is that we construct a Linux container. We use
namespaces, we use C-groups, we use all the stuff that we will do in a normal container.
This will happen again. But inside this container we have created a very, very tiny environment
depending on the monitor. If we have far cracker, chemo, software based monitor.
These all these kind of you have different dependencies for the execution. So we prepare this kind of
environment. So we can boot the virtual machine and inside this virtual machine we will run our
application as the init process. So we have directly direct execution of the application.
There is nothing inside the VM that is part of the Uranc. No agent. Maybe an init that will help us
I will talk about later about how an init can help us make this work properly. And the idea
how it works in Kubernetes is that we create a pod and we only like all the side of the containers
will run as actual containers, but only our application will run in let's say this virtualized
environment. So this way we can also separate the actual containers, the user container which is
not let's say trusted with the rest of the containers in the pod like side of the containers
that can be part of the deployment of our deployment that is supposed to be secure.
So I showed before here that we created this little
image. So I will try now to execute it here in this. So I will just do a simple
nerd CTL is the third placement for Docker. I use nerd CTL because I can have
easier access to DevMapper. DevMapper is a snapshot that allows us to create block-based
root-of-face for our container and here on second get this block-based root-of-face and directly
attach it to the VM. So I will just do this one. I will specify that we want to execute
of your RNC. I will have the DevMapper snapshoter and I will run this one. I hope I didn't forget anything.
Yeah, I forgot anything. And let's pull it. And here you can see that we have booted the
this small BSD. Yeah, this is forget about these, there's some debug stuff that I was doing.
I will explain later why. And we can see that we have also, we don't have any network here,
but if I get here, you can see that this one is looking like a container. I can inspect it.
And oops. Yeah. So if I go here and
I will be able to do this one. Yeah, this one should be what I said before that this is part of what
the init should have done for us, but right now we don't have any init. And if I add here
okay, so I can even ping anything hopefully. Yeah. So this is just running here. So let me go
back to the presentation. The same thing I can do it with a free BSD. We have support for free BSD.
We added it to works in the same way. Maybe I can try and yeah, if we have time I will also
it later. So the use case for this kind of thing is that we can, as I said before, we can deploy
this kind of small BSD microservices in Kubernetes. And we can manage them with any other kind of
any other kind of container. We can run into with K native, so we can have a serverless
stack that built the spawns of BSD application. We can also use it for BSD development in Linux.
So another thing is that we can, for example, run some of the stuff that we want to create
for BSD in Linux, so we can create this kind of VM as a dev environment that we can have access
and we can build whatever we want there. And we can also have it, I don't know why this is, it shouldn't
be twice. And yeah, we also did a small, very early evaluation. This is, this evaluation has been
for the past two weeks and our results need to work a bit more and we try to retake them
because we had some very weird behavior with both free BSD and the small BSD case.
And we have an included, for example, here the case of small BSD. So I will explain why.
So this is the start of time that we go to from service. So to do this kind of Ben Spark,
we created a small server that keeps the timestamp of two things. So when we create the container,
the server gets a timestamp and when the container boots and the application directly connects
with a server and we get another timestamp. So we have the duration that, from the moment we
had the first timestamp, we invoked the container and when the container was able to respond,
they use the actual application. And as you can see, run C is a typical container,
it can boot in around 400 milliseconds, the spec is just a small nuke machine, so just a very small
device. The catacam we also can see, it exploded, it's a lot of overhead. And then you can see that
you can see either booting Linux or a chemo. It's okay, but even if we go further and use
run prem for example, it can boot even faster and also the case with free BSD. And I didn't
include small BSD because we had some issues and we, for some reason, the user land was taking
much more time to boot compared to the kernel. And most probably we have screwed up the configuration
there, we have done something very wrong and we would be happy if we could work with, for example,
with a mill later and we can discuss about it and I'm pretty sure that these numbers for a
small BSD can easily be down there to what we see for a rampant for example.
And we also did some kind of networking. So here is the packets per second that we sent.
We have the hyper server and the hyper client. Of course, the battery is opposite, it's not
a bit like that. Higher is the better and we measure the packets per second. This is not a good
evaluation setup, we know it, we will try to do it better, we just wanted to get a glimpse of
what happens. And as you can see, all the BSD versions are running much better than all the links ones.
However, yeah, we had some issues with the TCP stack and stuff, but hopefully we will
with the help of you, we can easily resolve that. So regarding the future things that we want to do,
we want to actually this, the first bullet has already been done, I guess, from the people from
small BSD of what we saw today, things can be way much faster. We want to have integration with
OCI images from FreeBSD that exists and what we're going to do here is just get a FreeBSD OCI
image and use it as the root of S that we're going to run over a FreeBSD kernel.
And we want to find some ways to pass some specific information to the inside the VM,
SWG would be an option there. We need the E need to set up the environment like the user,
for example, the UID, GID, the IP, all this kind of stuff. We can also use Docker build Docker to
actually build, like as I said, as I said, so before with Bunny, we also want to explore integration
with volumes, so we can easily attach stuff in this kind of containers, in the VMs, and of course,
it will be very happy, we have a done it in the past, we will be very happy to update the
run brand with newer net BSD versions. I would like to note that this work is partially funded
from some European projects. And to summarize, we know that BSD has some significant benefits for
certain workloads, recent work in the BSD community has allowed us to run BSD to boot BSD in
micro VMs in extremely fast way, and we can take advantage of that and actually package BSD applications
and deploy them, like any other kind of container that exists, I mean, any other kind of linear
component. And we can do that with Uran C and build it with Bunny. Everything is of course open source,
as I said, this is a CNCF sandbox project. You can find everything in GID hub, you can join the
Slack channel that we have. We have the documentation and the parts for BSD. We will hopefully
with help with our BSD people will finally merge them later. And yeah, so that's all from my side.
We have one and a half minutes, so if you have any questions, I'll be happy to answer to them.
Okay, there is one question there.
Well, not really because Uran C doesn't target right now the BSD. Oh, sorry, I had to repeat the
question. So the question is that currently we can use a podman to run BSD container as a Linux container, right?
On free BSD, yes. And yes, with Uran C you can do the thing that you can run a free BSD container as a Linux
container. yes, it would not recommend use for that. Currently we don't have support for that.
But yes, that would be a nice concert for us. And we would like to explore it using, for example,
like a perspective, some books like Mac and it's not existing, BSD, B5 and stuff like that.
They run Kubernetes, will they, theoretically, also run the, for example, in an existing
point one or Docker set up?
Yes.
If the question is, if these containers can run in Kubernetes, Docker,
podmon, yeah, the demo that I showed before, it's NerdStale.
NerdStale is from the container, the community, that it's just
tried to replicate everything that Docker does.
So yeah, it's going to run both in Docker.
We don't have, we have some issues with podmon, but we'll resolve them in this
release.
There is not, I mean, there is not an app dependency on the door, I see.
There is a dependency on the urans, like this kind of images, they have to be
around on top of uransy.
Oh, so for example, in Docker, I'll show the work, it will go, it's just already, yeah, we have
been in the other time, but I can, we can, you can just ask this, yeah, this question.
Okay, thank you.
Thank you.