All right, we're going to start again, please take a seat.
If you want to talk about stuff, please go outside.
So next up, we've got Claire's going to be talking about comparing Fuchsia components
and then ex-containers.
Hi, everyone.
I'm just going to get started as people filter in and out.
Yes, so I'm here to talk about Fuchsia components and learnings containers.
I first about me, my name is Claire.
I am a software engineer, and I am the tech lead of the component framework team on Fuchsia at Google.
And at that out of the way, let's talk about what I'm talking about.
So Fuchsia is a new operating system.
We've been working on a Google.
It's not Linux space.
We have our own kernel.
Everything's like new and custom.
And it has a software sandboxing technology.
You know, where we can distribute, like pieces of operating system
and run them in a sandbox way that has a lot of similarities to some
Linux container solutions like Docker.
And I find the fact that these two things ended up being similar, really interesting.
So the stock is going to be focusing on how they're similar and more importantly,
all the ways they're not.
Yeah, so Fuchsia.
Before I get into that, I want to tell a quick story about how I ended up working on the component framework.
So if we go back to 2015, I started working for a startup and San Francisco called CoreOS.
And the project I initially worked on there was called Rocket.
Rocket, RKT, was aiming to be an alternative to Docker.
They wanted to do about the same things, provide about the same functionality.
Just be an alternative for, you know, people to be able to choose what they're using.
And just to kind of quickly go over, you know, our objective what we were going to do.
Let's talk about what Docker does.
Just kind of some of the basic functionality.
It lets you build software packages, you know, like archives.
They hold an executable to, like, actually run software.
And all of it's dependent, like, run time dependencies.
It's dynamic libraries and configuration files, those types of assets.
And then those archives, those packages get stored on a server.
And distributed to machines that actually want to run the code using the update framework.
And these packages get stored in content, address storage.
So we break them up into pieces and, you know, put them in this content address storage.
It allows us to de-duplicate blocks across packages.
So we can save on network and storage costs if two archives have the same pieces in them.
And then, at runtime, we take those blocks out of content address storage.
We ressembled them into a directory structure.
And then we can launch name-space sandboxed processes using those executables.
With that directory from the package as the roots of the process.
Hopefully, this is familiar to a lot of you.
Now, I wouldn't be surprised if most of you hadn't heard of rockets.
This next slide's not as surprised.
Rocket didn't end up lasting long-term.
The project was officially spun down in 2020.
But development really kind of petered out around 2017.
And when, you know, I stopped looking at the rocket and turnly core lessons.
And moved to other projects.
It's fun and less interesting.
So I looked around for other cool things happening in the space.
And I learned a fuchsia at Google.
And I wanted to work on it.
So in 2018, I managed to join Google.
And more than Google, I was working.
I was put on what later become the component framework for fuchsia.
And my new manager at Google was explaining, like, what the software's deck does.
How it's supposed to work.
You know, the basic objective of it, along with, you know, some of our partner teams.
So we were going to build software packages, holding all the dependencies for an executable.
It's like run a process and all the dynamic libraries and things it needs.
We're going to distribute those using the update framework.
So machines that want to run the software.
Those, you know, packages get broken up into content and stored in content address storage.
Did you duplicate labs?
Though we reassemble those labs into a directory structure.
And then launch sandbox software out of this.
This is sound familiar.
So I like asked around.
Like, why does this look so similar to the Linux technology?
And all the people working in the project, most of them came from, like, Android and Chrome.
And, like, they knew of Docker, but they weren't, like, you know, ex-personant.
And they had somehow stumbled upon a cleaner implementation of containers,
except on an entirely new operating system.
I found this really fascinating.
But of course, the projects aren't identical.
You know, this is very, like, broad strokes definition I just gave.
And to kind of show the ways that are identical,
I'm going to walk through some of the main ways that they diverge,
the ways that, you know, are models and designs and functionality,
are different between the two sides.
And for this talk, I sat down and thought up, you know, like, where do these differences come from?
What are some of the main things driving, you know, the fact that these aren't identical projects?
And so some of those differences I came up with.
Of course, Fuchsia is not Linux.
We have a different kernel with different CIS calls, different functionality,
different APIs available to us.
And that, of course, changes what the project looks like and the functionality
that we then offer to users of the component framework.
I'll get into the second point more later.
Kind of all these later.
This is the agenda.
One versus many hosts on Fuchsia were typically,
a component typically is concerned with other components on the same machine.
You can't necessarily make that assumption in the container world,
like if you have a Kubernetes setup, for example,
when you are talking to your PC to some other container that could be on the same machine
that could be on the side of the data center that could be on the side of the planet.
You know, such things are possible.
And at the end of the day, we also accept different objectives, you know,
what people typically try to do with containers versus what we're trying to do with components.
Also, a disclaimer at this point in time,
I, you know, was working in the Linux container space,
but I haven't really looked at it in fact since 2018.
So I'm mostly going to be talking to like the Docker type of,
you know, Linux container, you know,
those are the things like system-based systems,
the end spawn, of course, and more.
So just, when I say container, just so you can guys know what I'm talking about.
Let's get into the first one.
Fuchsia is not Linux.
To kind of like show, I think one of the bigger differences between Fuchsia and Linux,
I've made this slide.
And it's kind of showing like what the kernel is responsible for,
versus everything that lives in user space.
You'll see here that the Fuchsia kernel is responsible for a lot less.
Our kernel, our kernel is names are kind.
It does not know what a user or a group is.
It does not know about file systems.
It does not run drivers.
It does not know anything about networking,
and so on and so forth.
And all of that has been lifted up into user space.
And the thing that manages user space is our component manager.
In our stack, it fits into a more similar part of the system.
This system does, you know, instead of Docker.
And with a zircon or kernel, not doing as much.
An important thing to ask, I guess, what does it do?
The easiest answer that is it's a capability-based operating system.
This is a security model and design model for how operating systems can be built.
And to make sure we're all working with the same definition,
I'm just going to quickly run this through the definition that
Wikipedia gives us for capability.
So capability is an affordable token.
So it's like a token you have that other people, other processes,
can't just easily copy.
This token references an object.
See, there's an object somewhere that using the token,
you know, it'd be like points at it.
The token has access rights that denote what you can do with that object.
The token can be used to access the object.
So if you have the token, you know, that's free and free and
supermission to go and read or interact with.
You know, whatever operations are possible on the object.
And the token can be shared with other programs.
So you could have a token, you could make a token,
and then give it to your buddy process over there to do something with.
And I know this is all really high level.
So to tie it into something, there's a type of capability that
all of you are already familiar with.
Because file descriptors fit this definition perfectly.
A file descriptor is a token, you know, like some, you know,
number that is given to you by the kernel.
It references an object in this case a file or directory of
black device.
It has access rights.
You know, you can open a file to be read only.
And then with that file descriptor, if you try to write call,
that's going to fail.
The file descriptor, yeah, can be used to access the object, of course.
And a file descriptor can be shared with other programs.
Using the send-in as Jesus call, you can send, you know,
open a file descriptor and send them between processes.
So to kind of understand what the fuchsia, what the zircon,
like system call interface looks like,
imagine if a process could only use file descriptors.
And other types of system calls are not available.
Imagine you like start a process.
It is treated into an empty directory.
So can't access anything by default that way using paths.
And during process initializations, hand it open file descriptors
to the handful of directories around the system.
And then has to use those to reference and interact with the world.
If it doesn't have a file descriptor to something,
then it can't interact with it.
It can't be named it because there's no strings or path-based operations.
At least at the start of it.
Everything is referencing an existing token that you can give in.
And so to type back to fuchsia, you see here how a lot of stuff
that the Linux kernel is responsible for has been moved into
user space being run by components here.
Because that's a user space that means you need a capability
to be able to interact with all of these types of things.
And we call capabilities on fuchsia handles.
So you need a handle to interact with these things.
And by default, a process on fuchsia, a new component,
doesn't get access to any of this.
A default component has a handle to its own process.
So it can do system calls related to itself.
A hand to its own job.
A job is a type of container for processes.
It's not that interesting for right now.
And a handle to its package directory.
So we can read dynamic libraries or configuration files,
or stuff like that that's been bundled for it.
A component, by default, with no other configuration,
does not have any access to immutable storage.
It does not have any access to the network.
It cannot launch other processes.
Other processes?
By default, it doesn't even have the ability to embed logs.
It can't even send strings off somewhere for you to see.
It can interact with any other components,
aside from a package provider.
Because remember, file systems are also components.
Yeah, and this means that most of the configuration knobs we have,
available to developers, are about these capability handles,
to noting which capability handles are needed as inputs to a component,
and which capability handles are being created by component,
and provided back to the system.
So for an example, let's compare, I think,
like kind of a basic developer flow for Docker and components.
So we have just a Docker file here.
I just ripped this off from their docs somewhere,
and then a sample of component manifest.
The component manifest, it's a JSON-5 file developers
are expected to write them to note the inputs and outputs
from their component along with some basic information,
like how to run it.
So, some of the key differences here.
The component manifest does not say how to build the component,
where the Docker file does.
The Docker file, you know, kind of as a full-term key solution
around like assembling, building, and running container.
On Fuchsia, all of our building happens in an external build system,
like Basel, for example.
And the build system, you know, will package up the JSON-5 file,
or an Ethereum, you know,
print trends, like compilation processes,
and put a binary blob based on it in the package.
Both of them do note how to run the application.
On the Fuchsia side, you know,
here's a path to an executable inside the package.
That's used to inflate a process.
And on the Docker file, we're given a command to run,
you know, in a shell-like environment,
inside of the container.
The component manifest,
notably specifies that it needs permission to connect to a log server.
If you should at logger.lux.
That's our IPC protocol for sending logs.
Containers by default can make any outbound network connections
they want.
Of course, you can configure this.
But so Docker file isn't necessarily going to specify what
network, you know, things that needs to be available to function.
And both them specify that something is going to be provided
available for connection.
On the Docker side, it's just something is on port 3000.
And the component manifest, there's a bit more details,
where saying, you know, what type of handle here,
and the IPC protocol.
Like that definition, we're excited to speak over that protocol.
Yes, that's one subject here.
Another one is one host of many.
It kind of comes down to the size of the universe for components versus containers.
As I mentioned, components usually only care about other components on the same machine.
They're all working together to make a single device function.
Whereas containers, it's more common to have deployments compared to components
where they can exist on different machines.
You know, you might have, you know, failover.
You might, you know, want to have a website where it's okay if one machine crashes.
And this lens to different pressures on how these things work.
And change is what's possible within these two sandboxing technologies.
And one of the key things about components is our IPC thing,
which you saw reference earlier.
We call it fiddle, the Fuchsia interface definition language.
It's kind of like the ERPC, you know, you write out a protocol like definition
around like what functions are available, but other return types.
But the reason why we came up with our own is that this allows us to natively
reference and work with fuchsia specific concepts and functionalities.
So as an example, here is a function in one of our fiddle calls.
This is in our loader service.
This is used during process initialization.
When a process is started, it uses this to talk to whoever started the process
and ask it for memory objects of the dynamic libraries that it means to, you know,
do its linking and get up to being a functional, you know, process.
So let's say we have process B that was started by process A.
Process B sends the string object name when making this call.
Process A returns a result code and actually a VMO, which is a virtual memory object.
And that's the key bit here because these components are all in the same machine.
They can rely on sharing machine local resources.
It's very common, very trivial on Fuchsia to have shared memory between components that
they negotiate over our IPC language, whereas in containers it's going to be more difficult
to set up shared memory between two containers because you may even know if you have
the same memory available.
Of course, you can do stuff over the network, but there's different latency and, you know,
speed expectations there compared to when you're on the same machine,
sharing the same memory.
Another big difference around the size of the universe is fault tolerance
and how the software that runs in these things is expected to behave when problems show up.
On the container side of things, if some container dies, you know, you're running along
and you're using it, oh no, trashed.
It's not that hard to just stand up a new container, update it CD or DNS or whatever
to point to that, and everything can keep going, you know, can recover in a way.
Compared to on components, one component dies.
Typically other components that have a connection to it also die.
It's very normal to be like, oh, if the connection breaks, if there's an issue with
talking to this server, I'm dependent on just trashed.
Let's just blow up the world.
And this isn't to say that the system-based components is more fragile
than the system-based on containers.
The difference is that in the component world, things like talking to your file
system are within the inter-components, like, communication.
Containers, when they're talking to something else, like, there's more an assumption
of the network, could you know, have a problem.
The other thing could crash, but I need to be okay.
Whereas on both systems, if your file system crashes, if EXT4 crashes on you,
that machine is going down.
Your container is not going to survive that.
Whereas the component land, our framework is more, you know,
we immediate those connections.
We, you know, determine who's allowed to talk to the file system.
So the, yeah, we aren't as resilient to handling one component crashes
and having that out of effect on the components, because just the definitions here
are different.
The space we're working in is a little bit different.
And this brings me to my last point, which is just that we have different objectives.
What we're trying to do with components versus how containers are often used
to, you know, build software and build solutions.
I'm just going to wrap and fire through some of the small differences here.
Commonly, containers have larger package sizes than components.
It's very normal to bring in like a base OS image.
You know, have like a shell.
It'll have a package manager.
And, you know, you run commands in there to, you know, set up your software to, you know,
grab, grab dependencies and produce an archive that you can run.
And then that base OS image, all the libraries that came with you might not use the package
manager, all gets shipped with your application.
Whereas, because our build system is external to the component framework,
it assembles really minimal packages.
We do not put a shell in a package for a component.
If you need to shell into one, the tooling can provide a shell that will drop into that same
namespace, sandbox environment.
And also deployment is different.
If you want to deploy a new container, you just knock or run it.
Yeah, if you're using Kubernetes or Docker Compose, it can be more complicated,
but things in between to run time.
You can go and change, you know, like, like, system units if you want to,
and, like, reload them.
Whereas, on Fuchsia, pretty much all of the components that are going to run on the system
are fixed at the time the operating system is built.
There are small buckets that can have things dynamically launched with them,
with really well-defined, like, limits on what they can access.
But, by and large, adding a new component to Fuchsia requires, like, a full OTA of the device.
Yeah, and this brings us to differences in deployment.
We're typically running a single consumer device, where all components are working together
for that singular purpose, whereas it's, you know, perfectly reasonable and container space
to have a much more heterogeneous workload.
You might have two different clients who want to run two different websites.
They don't need to need to be aware of each other.
We're going to co-locate them on your machine or something.
And one of their big differences is how cross sandbox orchestration works,
how you group together containers to get them to work together versus components.
Typically, in container space, you need some other solution sitting on top of Docker,
or whatever your containerization solution is.
This is why Kubernetes exists.
Docker compose can be thought of as a layer on top of the basic Docker containerization stuff,
whereas the component framework itself carries the relationships between components
and how they interact with each other.
And this is kind of a big difference.
Components are organized in a tree.
Containers are a flat name space.
You can run, you can add a container, you can remove container.
It's just in the list of containers running.
Whereas every component aside from the root has a parent component,
it potentially children components.
And let's pick on, this is an example of what Fuchsia actually looks like.
It's missing a lot, of course.
But let's pick on the network component via here.
Then we're component from the perspective of the core component.
It's just, you know, it's a node, everyone's staff.
But if you go and look at the network component,
it's actually made up of a handful of components.
And that network component, when things are made available to it,
when they're available and it's inputs,
you can then make those available to whichever and any child it has.
And likewise, anything available from one of its children,
you can make available back to the parent of the network component.
And it even makes it easy to keep things like,
let's go up within here.
If NetC of Gene needs to use something from the HCPD,
all of that can exist within the network component.
And the network component does not need to have that in its inputs or outputs,
meaning that it's parent, the core component doesn't get to see
or not interact with any of this.
And this makes it easier to organize things
and provides for better encapsulation.
It means that if you have a component that's running and doing stuff,
and you want to break it up into a handful of components,
you can do that.
And it works well within our system.
And this also brings us another key thing.
The network component exists for organization and, like,
configuration and stuff.
And you can know all of that statically in a manifest file.
This means the network component does not have an executable.
There's no running process for that component on our system.
And this is another key difference from containers.
It's trying to non-sensical to have a container with no process
that's intended to run in it, whereas that's normal on Fuchsia.
And this type of system, where the network component kind of have,
you know, a handful of children, allows us to have all of that,
you know, owned and scoped within and owned by a specific team on Fuchsia.
So the network team can own the network component and whatever
they want in there.
And all of what's going on in there, the connections between those things,
is invisible to the rest of the system.
And not really bulletin to the rest of the system.
So it makes it easy to kind of have their security boundaries as well,
and that aligns with their organizational boundaries.
Yeah, so there's a lot of differences.
I listed that at some of them here for just like a summary slide.
I find it fascinating that for such a different purpose,
from engineers who didn't really come from ladies container land,
they would build something that relies on some of the same building blocks.
And I think the ways in which the two projects are different
is also very interesting.
Like for example, containers have way more nod than the ability
to talk about network configuration than components do.
A typical component is not talked to the network,
but I wouldn't be surprised that something we need to do eventually.
So there's, you know, learnings both ways I think that are potential.
Anyway, that's my talk.
We have time for questions now.
All right, any questions?
No, so my question was related to whether I was a difference
with a component and an application.
Because as part of the first point around how future
is different from a container, you mentioned that components
are set in style mode as I always build time.
Does that mean all the applications that you want have to be set in style
when you package OS for a particular machine?
So if I want a new application, I need to effectively build the OS again.
Or is there an ability to add something later?
Yeah, so you've noticed I've been a little fast
and lose with my definitions in this slide presentation.
But so most components are fixed at built time, as I mentioned.
But you're able to define these little boxes.
You can put components in dynamically.
And the way that that works is each component, you know,
by their parent is, you know, given in the component manifest,
the set of things that are the inputs and expected outputs of that component.
Where you can say, I'm going to run the network component.
You get all these things.
I need these things from you.
And the way these boxes work is very similar.
You can set down this little box and be like, this box,
and anything that's going to go into this box will get this limited set of access to,
you know, these handles, you know, from those places.
And then at runtime, you can watch things in those boxes
that get access to those specific handles.
They're specified statically up front.
This means if you have specific types of things you want to run dynamically,
like drivers, it's, you know, easy to have the little box,
but, you know, the things that drivers might need.
And there's there are some knobs for runtime configuration,
about that are kind of limited.
But this means that you need to have a specific box that fits the shape of what you want to run.
So if you find a box that is sensible for user applications,
there might have a lot of things available to it.
You know, user applications tell you when to access the network,
multiple storage and all this other stuff,
whereas the box, again, for drivers or for file systems or, you know,
for other types of things, just like developer tools,
you're going to have different inputs available to them.
So if you do want to go and run something dynamically like that,
you need to specify the box for it up front,
and, you know, make sure that that fits whatever applications
you're going to want to run inside that box.
Does that make sense?
Anyone else?
Oh yeah.
Thank you for the talk.
I actually watched this project for a while,
and I was always wondering, like,
what is this actually targeting?
Because I haven't seen this operating system the while so far.
When the concepts look really cool,
and I would imagine there is several places where this could be used,
but where is it actually used?
Great question.
I'm not allowed to talk about future plans around the project,
but I can't talk about what has already happened.
So we do today run on the type of project from this place.
It's a Google Assistant with a screen on it.
You can ask it things and it will show up on this screen what's happening.
So all those devices now run Fuchsia.
And it's a really interesting part of the project for that migration,
but yeah, I don't know if that answers your question.
There's a consumer device out there that runs our software,
and I can't really talk about future plans around that.
On your example, I always assumed they were just running Android device.
So what's relationship with Fuchsia and Android?
Android runs on top of the Linux operating system,
and Fuchsia is separate from that.
There's no formal relationship between them.
If you have an Android phone, that's running on Linux,
that's the stack that you're familiar with.
Fuchsia is not in the same space right now.
I think there was another question over here.
You say that you resemble the files from a content address
blobs to back to directories, do you copy them?
Or do you do something else?
No, we do something else.
So we do a deduplication based on the file.
Instead of Docker does it based on layers,
really the commands in the Docker file.
So we have what we call blobFS.
Every file is stored in a content address way.
And we have the manifest file that specifies for this package.
We need this blob at this path, this blob at this path, and so on.
And we have different files system implementation.
So that speaks our IPC for the file system.
The components speak the IPC for the file system to that thing.
And that presents the actual file system view.
You would expect the right paths and names and everything.
And whenever it goes to read a file, that then gets
forward to the correct blob in blobFS,
using the cache of the expected content for that file.
So this looks like BGR of the package.
Yeah.
There's some similarities.
All right.
We are the time, but thanks a lot.
Yep.
Thanks everyone for coming.