Okay. Thank you. Can you all clean me? Yeah, it's a little bit loud sometimes. So, thank
you all for coming. I'm going to present today a journey on how to add a risk
five support to my favorite operating system. So, a little bit about myself, my name is Adrián
Vladó, I work for a small company in Romania, cloud-based solutions. And for the past 15 years,
I've been doing quite a lot of software work in the cloud world. I don't know if anyone
here, it's familiar with OpenStack. Anyone? OpenStack, guys. Then of course, OpenStack
could change to Kubernetes on OpenStack containers and here we are. I have also a few
years of experience in making short Linux actually works on hardware or software virtualization
like Hyper-V, also did a quite a lot of work on R. So, that's my background. I'm a flat-carbon
container, flat-up container Linux maintainer, maintainer. So, this is the favorite operating
system that I'm really going to present about. You can find me on GitHub and you can shoot me
any mail if you want. So, let's establish some outlines. Why it is five? Why flat-carbon
container Linux? And also the bulk of the presentation will be about how I managed to add
another architecture support to flat-carbon. And if the networking works, hopefully I will present
you a demo with flat-carbon running Kubernetes on it on R.5. Hopefully that works. So, a little bit
of history actually found out about R.5 two years ago here at Fosden and decided, oh nice, let's
let's see how it works. And then last year it was a big year in R.5 because a lot of chip hardware
appeared on the market. And I thought, okay, maybe it's time to buy one, get one and see how it goes.
I decided to get one of those boards. And of course, as a normal thinker would do, I actually
end up buying three boards, not one of those. My background about R.64. So, I've been doing this
inkling with Raspberry Pi from the Raspberry Pi version one, two, three, four. I have a lot of
Raspberry Pi's at home and a lot of other boards. So, I thought, okay, it might not be so difficult,
after all, to also work with these five. So, first board I got, it was in February last year.
And the best board that you can buy from the market was the L.C. Pi 4A. Anyone here has such a
board. So, this was the entry point for me as a thinker because it had quad core, had an availability
of 16GB of RAM. Unfortunately, it had only as the cards in EMMC. So, the NVMe part was lacking.
So, what did I do? Well, I bought one with NVMe support right. This one is the high-five unmatched.
It's a precious board. It's an amazing board. Although, I found out after I bought it that the NVMe
microcontroller was not that fast, but still, I could use it with an NVMe. And then I found out
more about the how old let's say environment works. That canonical supported this board for like
three years already, which was amazing. And I could basically run a bunto on it without an issue.
Then I found out that the bunto image came with a downclock of the CPU.
Very interesting. Then I learned how to overclock it, of course. And it's working very well.
Then in October, 2024, came the banana pie of three, the new CPU, the spadesmith,
spadesmith X60 with the M1 or K1 variant. This one had the K1 variant. And this board
did all the boxes, had NVMe, had enough from to play with, had vector extensions,
almost all of the boxes, because it doesn't have a H extension, the virtualization extension.
So, I was ready. I had the boards. Okay, let's see how I can go further.
It's being said that, and I strongly believe that the Linux,
the Linux equivalent of the CPU in structured set of textures. And I really like that.
The recent availability of hardware was the tipping point for me to actually involve time
into this file. So, because you have to test on something. And the only way to test it is with
KEM formulation. And I will show a little bit later on the demo how slow it is. You can not
use the most of the time on waiting, which is wasted time here.
You can get it for a personal ownership. And there's also a provider that
scale weight that started offering from last year away to cloud.
Can I maybe I should remove my microphone or to the point of the device to be called it?
Okay, thank you. So, scale weight offers this, basically to paper per the time used of that
bar metal, which is amazing. I could remove it because I could remove it, because I could remove it.
Maybe this way? Yeah. Okay, perfect. Then I decided, okay, and also for this five,
two, because it's fun. I like tinkering with stuff. And I see a quite a bright future ahead.
I'm not going to enter into a geopolitical things, but things are looking great from
a number of perspectives for this five. Okay, why a flat car container links? I'm a flat
car container links maintainer. And I thought that, okay, we sport and this 64, 164, why not try to
at support for this five? Flat car container links is an image-based operating system. It's a very
minimal operating system. It's secure and reliable by design in the sense that it's very
compact and it's also immutable. So, and the only purpose for flat car is to run containers.
Basically to run Kubernetes in today's world, in the cloud native world. Of course, you can
run also virtual environment, of course with Kubernetes, but that's the main idea for flat
car container links. Okay, so starting for this five support. I had to look first at the flat
car ancestry. Flat car was a drop in replacement for Coroés. Anyone here remembers Coroés?
Still use Coroés? Fedora Coroés? Okay, flat car. Perfect. So, it's a drop in replacement for Coroés.
It's a direct fork and it had evolved over the last four or five years into a, let's say,
to follow the legacy of Coroés. Important note that flat car via Coroés is the downstream
fork of Gen 2. Gen 2 is the basically where flat car draws its packaging system.
And it's good to note here that it draws the way to build the packages. It doesn't draw the
packages itself. So, it's a source based operator system in the same sense as Gen 2. Gen 2 is just the
way of building those packages or provides the way of building those packages.
About this tomorrow there will be a presentation about how flat car communities and Gen 2 communities
work together and how they help each other, you know, going back and forth with the packages,
and the things. If you'd like to then my colleague James will be presenting that tomorrow at
90th. Important part too is also about the necessity of the ability. So, being immutable,
there is no way other than using containers. Because if you cannot change the OS,
there's just the container way. There's no other way. Upgrades are also done
so being immutable, upgrades have to be done via the AB partition in scale. Like Stimois
another operating system. Because if you cannot change it, you have to have another partition.
And then at the reboot time the B partition becomes the A partition, the active partition.
So, this is how upgrades work. This is all important for the risk support.
So, Gen 2 has already support for this five. It's currently in the unstable architecture,
but most of it is already working. Last year after the first day, I got my leechy pie for A.
I went to Gen 2 and okay, let's see how far I can go. So, I could actually go really far
the root file system from Gen 2 was perfect. It was working on this five. The only problem
is with all the boards for this five. He's that they always, at this moment, require a very,
let's say, different kernel so-street. They're own so-street. There's no way to use the vanilla kernel.
Also, the way of doing the DTPs and the other stuff, the firmware, it's again.
So, what I ended up, I actually took from, took the kernel and in it are the, from the
already working operating system. The one that came with leechy pie for A, put the root file system
of Gen 2 and it put it magically, everything worked. I was lucky in this because the,
there was the same kernel major version. Otherwise, it wouldn't have been possible, but it worked.
It was a miracle. Okay, I booted it. Then I tried to, okay, do I have Docker? Yes, I have Docker,
if I build Docker, right? And after one day, I could finish building Docker because Docker is not
Docker. It has continuity, it has tons of packages. So, on the leechy pie, it took me around 12, 14,
20 hours to actually build Docker. So, that doesn't work. That's no way working.
So, flatcard is solving this already because flatcard comes as a, as an image, as an image of
based operating system. And afterwards, you have Docker and then there's a lot of great support
from the alpine community. Anyone here using the alpine list five Docker images?
There, I mean, they have support for anything and everything. You can, you just get a GCC
and all the building libraries and you just build stuff on top of using the alpine. You don't need anything,
any compiler on flatcard, for example, to do your work afterwards. So, it's, it's quite amazing.
Does it mean that everybody who inherits alpine image also supports this file?
Sorry. Can you, does it, does it mean if anybody use alpine image to build a Docker image?
Will it also support this file?
No, so, so how it works? The flatcard operating system already has to have a kernel for this
file. So, you can actually use with some shenanigans, we clear KEMO on other architectures.
But by default, you should have a flatcard running risk-5 and then using the Docker image for risk-5.
The Docker images are separate 100% separate for each architecture. So, it's not just one Docker
image for alpine latest, for example, there are three or, well, how many for how many architectures?
So, yeah, it won't work directly unless you use KEM and you will end up in the same problems
of a slow environment. So, yeah.
I have a question. Yes. Have it tried, I have a question. Have it tried using DCC and cross compilers?
Do you leverage something like, I'll be, I'll be going exactly to that topic.
So, we have a bit of materials for the flatcard to build flatcard. What does it mean to build the flatcard?
Flatcard heavily relies on an SDK. That SDK is a Docker image, it's a very big Docker image
amounting to something like 10GB. Which, and there are all the libraries necessarily, all the
cross libraries to build flatcard. There's also the flatcard packages which you put into the flatcard
OS. So, we have to disobey here that in order to build flatcard, you need to have more packages than the
packages that make flatcard up. So, that's why, how wide the Docker image you exist for the SDK.
Of course, in the middle of materials, flatcard heavily relies on two things, system D and ignition.
Anyone here used the system DCC6? Do know how, what they are. So, system D C6
being a new table comes with a problem. You cannot augment the immutable file system.
But, what if you have an overlay which is immutable and then you have system D C6, put it on top
of the already unignutable stuff. Practically, it becomes mutable, but it's not mutable.
It still is immutable, but it's composable. So, this is how we solve the problem of
composability for an unignutable operating system. There are more steps, but it works.
And then you have the root file system, you have the C6, you have the flatcard packages,
then you need to put that root file system to have a put loader and you need RD
to work with the utilization and also to work with the DM variety. It's a security feature for flatcard.
So, the first, the flatcard is the K Docker image based on the Gen2 bootstrap workflow.
Currently, supports NM D64 and NM 64. And everything is cross-compiled.
The RM 64 is cross-compiled on NM 64 machines. So, the Docker image for flatcard is the K,
it's only NM 64. Everything, including NM 64, is using the cross-compiled
for NM 64, but everything is done in the cross-compiling way of thinking.
Okay, let's use the same workflow for RIS5. So, important, I had to create a new profile.
This is Gen2 Quartz based on the system DLP 64D, which is the implementation for floats in the CPU.
And a few flags. And now, nowadays, to build even, to build Linux kernel,
you need more than GCC, you need also REST. So, again, REST comes with its own
cross-compiling stuff. So, add that more to the Docker image. And then, always, in order to cross-compile,
you will find some packages that, in their Mac files, they do cross-compile stuff,
you know, you have a binary. And then, SNAP, I want to run that binary.
To create config files, to create something. And that is solvable via the QM static process,
or binaries. So, if you install QM, you will have always the QM static, REST before. Sorry,
it is QV 64, or REST 64. And with bin and bin format, it is a small service that when you are
trying to run binary for another architecture, it basically hooks up into the QM static.
And it works. Magicly works afterwards. Slow, but it works. Perfect.
Okay. So, I have the SDK boilerplate definition, but some of the packages need some tricks.
Let's see. Here, Flatkar has two sources of packages, or two package categories.
Once they are, when inherited from CoreOS, and were special to CoreOS,
some that are only flatcard packages in the sense of, they work for some cloud providers,
some virtualization platforms, and so on. And those are updated manually, most of the time,
and infrequently. Here, it is also the package to be a Linux kernel system, because we have a lot of
packages, grub, and then more. Basically, the most important packages are not only the source
thing and a part of how Gen2 does it are coming to Fatkar, but they are very specific.
And the most part, the biggest part of the packages are, they are just mirrored once a week
of Gen2 tested and made sure everything worked. And those ones, were already had the risk five
support, so there was no problem. The most part was spent on adding those CoreOS, the legacy
once to make them work with Tories Tv. Basically, to add a risk of the keyword, and then just
pray that they build on risk five. There are some Gen2 packages that need some fixes,
and some of the work also got upstreamed already to Gen2, basically, because of Fatkar, sometimes
uses very niche packages, very niche packages, and Gen2 is a very big ecosystem. So there's no
possibility for any package to support any architecture. Okay, so boilerplate to work done,
do we have a Flatkar is the K, not really? So there are a lot of packages that are not really
supported by risk five, because they're not supported by Linux in itself. For example, the K exact Tools.
I think they work only for AMD 64. Some cloud provider packages, like Azure VM, Amazon agents,
secure boot packages, like Shim and ShimSign, they had to be confined only to basically remove
from the risk five profiles. And then Flatkar also supports from a security perspective,
supports via SystemD, locks to, with stores like TPM, and that had to be fixed, also,
Sim that the Gen2 is not actually supporting it for a risk five. And of course,
everything is five, like with ARM 64, it's an FE implementation. So everything
FE related had to work. There are some issues with LibFFI, the foreign function interface,
that's a pain at any time, from the Python perspective. And some goline, let's
ask the architecture or close compile flags that have to be taken care of.
So we are getting here, getting very close. What did next on the menu? Linux kernel.
So I had to remove the same hypervith support from the risk five config, because it's not supported
or well supported on on on on this five, um, important part. Initially, I wanted to test Flatkar
with the hardware devices, the the actual hardware boards, but then realized that's going to take
quite a while, and I have to use a very specific kernel for each. So I just took a step back
and decided to test on the KEM simulation first, because that's easier. Well, it's easier,
but it required extra work because the emulation, the KEM emulation, relies on some
paravitualized drivers that I had to implement, it's called the HCI platform. And in order to have a
KEM VNC console working, I also had to enable some PCI whole generic and a lot of DRM framework
buffer stuff. Important, those two kernel parameters, FED bug and the early cone SBI,
risk, you know, KEM emulation for this five heavily depends on the OpenSBI project,
which is very close to the risk five. It's actually made by the risk five community.
And as Flatkar supported or had kernel Linux 6.6, I had to support for KEM firmware configuration module,
which was basically, I need to patch it. So after all, and after I think one week of work,
I'm building the kernel and just trying again and again, takes some time. After one week of work,
it finally got built. For Docker, I had it complained a lot about the linkers, so this was a
gentle issue. I have to upstream it, but also Docker work. Docker container D Kubernetes,
which were all transformed into system D66. We will see soon how that looks like.
So we have the root file system, everything is working, success? No. So we go here to the, to the
hard part, bootloader. Flatkar relies on the red, red hat packages for grub 2.
Yeah, there's a lot going on there. The packages are thousands of lines of code. It's really,
really hard, but it works from this five, because also Fedora supports this five, so everything
worked there, but Flatkar from the Coroest legacy inherited a lot of grub 2 special workflows
and functions in order for the AD partitioning tool work. In the sense that the grub had to know
about which partition is active in order to set the Linux kernel command line parameters
for the root. And that's fully a Flatkar grub package, so that's, that those had to be patched and
made sure that everything worked. And also I had to find an interesting thing. For DM variety,
there is a hash somewhere, and we have to store it somewhere, and we store it in the Flatkar workflow,
we store it in the inner defile, but I had to find an offset where to put that hash,
which should be free. Still working progress to know if anyone from the Linux community is here
to tell me if I did a good job finding the offset, please let me know. Okay, then I tried visualization
to on my litchie Pi4A and my banana Pi3 it didn't work. It seems that you only
currently available board that somehow supports virtualization extension is the
high-five premier Pi515. Okay, so next, demo time. Let's see how this works.
Can you see the screen? Yeah, that's going to be a tough challenge.
Let me see. Okay, a little bit better. Okay, so here I have the Flatkar operating system.
And as you can see here, it's running or is 564. This is the Linux kernel 6.6. And if I do
see all this release, you'll see here the Flatkar container Linux version 4187. So, I have it running.
But what I like to show, okay, the important part is for Flatkar to be able to run containers.
So that, okay, can we check if containers work? If Docker works? Oh, perfect. We have
some yellow worlds running which run at some moment. Let's run it again. And now you can see
the range of virtualization. To run a yellow world, it takes quite a while. So, this is why I'm
actually waiting patiently for the H extension harder to actually be able to test stuff.
But you see everything worked from the yellow world. Let me show you the system, the
C6. So, what do we have here? So, we have the overlays which are themselves in
usable applied on the big file system, which is imutable. So, we have a container deflatkar,
a Docker flatcar, an OMQM, and a K3S. Why K3S? Because I wanted to run Kubernetes. Okay,
do we have Kubernetes in the house? I think we should. It's working. It's working. Yeah. And I also
when it was verified heavily with deployments, everything works, networking works, it's amazing.
So, we have three, I call it a freelancer and we can also hit it. Right. And if anything,
we just run, if we don't want Kubernetes, we just do Docker run, help find stuff and do APK installs
or APK ads. And we have at our disposal everything from risk-life community there.
Which is amazing. Okay. Let's, I wanted to actually show you something.
Can you see my screen? A little bit bigger.
Okay. So, this is the console log of the, so this is the console log, the SSH1,
and what you see here is the VNC. I actually work quite hard to make VNC work with KM emulation.
You will have here really good explanation on how it works, what needed to be done,
what were the boot loaders used, because I have tried first with, so I have tried first KM emulation
with EDK2, and then also with OpenSBI, both work, which is actually amazing. And you,
at this link here, you will have download files for flatcard to test, and also the OpenSBI
firmware binaries, and also the EDK2 firmware binaries in order to test. So, actually you can use
any operating system with those binaries. Interesting enough, EDK2, it's actually based also
on OpenSBI, so basically it adds OpenSBI. So, what I wanted to show you,
is do a Sudori boot. Let's see what happens here.
Okay, my, you see, my cry containers are going, okay, let me show, okay,
just making sure that I'm in the home. So, we should see now flatcard booting,
and I would like to show you something very interesting, how the, how all works from the boot loader
perspective, what I use, okay. Hopefully, we have some time.
Oh, five minutes? Yeah, so you can see, you can see right now the risk 5 EDK2 firmware.
And flatcard booting. Flatcard heavily relies on the in-it-arty workflow.
It uses something very similar to Cloud init is called Ignition, but most of the work that Ignition
has to bootstrap the machine, the flatcard machine is done in the in-it-arty not after. Cloud init
mostly has the user data and everything running in the normal after the in-it-arty pivot pivoting.
So, flatcard is very special in this sense because you can control the behavior before, not after,
or well, after it doesn't make sense, right? Yeah, now I have to fix this part,
it will take a while, display output is not active to enable the frame buffers, so it will take a while
until it boot. So, yeah, this I will, these are the slides if nothing worked in the demo.
So, take away great, it looks great on risk 5. Most of the issues that I had to solve were
cross-compiler issues, different things with packaging. Most of the work with the kernel,
the bootloader and the DMVRity. And the biggest takeaway is that you can actually
right now run Kubernetes, which is amazing because it opens up a door of possibilities.
Testing is low, can wait for the new new board versions. And it would be glad to get some feedback
if people want flatcard to have a visual support for this 5. We'll have to do a little bit more
polishing and see where that goes. Of course, there's a backlog of patches that we have to
backport to Gen2 to make sure that we pay our debt. And yeah, so here are the links.
You can try it right now. Also, it's very useful if you want to try out the opening systems,
as I said with the EDK to firmware and so on. Questions?
We have one minute. I will take question of questions afterwards. We have very active in the
flatcard community just, yeah. We can do one.
This is extra for you. Oh, you can ask me afterwards.