That is...
Yeah, it should be...
Did you have a book in my book?
Yeah, okay.
I have a book in my book.
I'm going to give you a book.
It's fine.
Yeah, we can do downstairs.
Awesome.
Okay.
I'm going to just...
It's on my website, so I'll just grab it.
I'll just grab it.
It doesn't change much.
Let's try it on this one.
And the worst case is I can also just set things up.
I don't know.
Yeah.
We'll get it.
In my initial bit.
It's the easiest.
The easiest is the right book in just.
I'll put it in the first book.
No, I am the easiest.
Yeah, he's up.
We're in just need to figure out.
Okay.
I couldn't figure out.
It turns out...
It's very easy.
It looks complicated.
Yes.
What is this?
Yeah.
I was about to get to the best part of the presentation.
And I run out.
Yeah.
Because I was going to...
You know, create a namespace.
And then it was going to un-mount the root.
And everybody was going to like...
What?
Why is this a show?
Wait, you can un-mount it.
Root?
Yes.
It's going to work.
That's the best thing about it.
How?
Because since you already have your binary directory mounted,
then you can still issue commands.
It's a thing about it.
It's a different...
Yeah.
How does it help you out?
I'll do that.
Can I leave this with you?
And you can just...
Yeah.
Absolutely.
Just got to make sure...
What did you do?
It is.
All right.
Figure it out.
Yeah.
There we go.
OK.
I'll do that.
Here you go.
OK.
All right.
We'll have this on your belt.
It's possible.
It's here.
Yeah.
It's a...
Post-M2026.
Keep yourself.
OK.
I'll do that.
OK.
Yep.
OK.
Yeah.
OK.
Oh.
OK.
I need to show you how I can get to my information.
Okay.
Oh.
Yeah, you've missed him slash in the end off.
Yeah.
Fast in 2020, 60GZ.
That's it.
Just untara.
The second file in the listing.
Fast in 2020, 60GZ.
That's the, yeah.
Okay.
You can handle it?
I guess.
Let's see.
I'm going to just...
Oh.
Well, no, there is a...
Yeah.
I'm just going to do it.
It's just a bunch of text files.
It'll be fine.
Right.
Wait.
That guy's very inferior.
There's no matter what was the punch you had.
You can try it with Suga.
Okay.
Ah, God damn it.
Should have used acne with the Bart flag.
Okay.
A lib font.
I'm sorry.
I'm just going to change the font.
I don't remember how to know which font I was supposed to use.
I'm just going to change the font.
I'm going to change the font.
Oh.
Ah.
Which one is?
Ah, Latin 32 right.
Yeah, there we go.
Sorry, this is...
There we go.
Okay.
So, now we're ready to start.
I apologize for the amount of time it took.
But yeah, so I'm going to be talking about the new file system for P9 front, GES.
It... these slides are basically a variation of...
I talk a give a couple of years ago when I initially introduced it.
So it used to say the great experimental file shredder as the acronym.
It's been upgraded to the good enough file system.
Is it great?
It's good enough.
And that's kind of the ethos I went with it.
There's a lot of ways that you could optimize this file system.
But my goal wasn't to be the most optimal, the most complicated.
Squeezing out everything.
My goal was to be simple and good enough.
Okay.
So...
What?
Damn it.
Okay.
So, I'm going to give you a quick summary of the current state of file systems on 9 front initially.
The world today, there are a few file systems that we have.
We've got CWFS, the cashworm file system.
This is the file system you heard about just now,
where the can actually wrote as the initial standalone file server
that was that eventually got ported from its own kernel to user space.
It has a...
What do we do here?
Okay, we just have to do the...
Cool.
I'll do it live.
So here's the cashworm file system,
which is the old Ken Thompson file system that was initially there.
There's HJFS, which is one that was written for 9 front,
as a kind of fairly boring file system.
But it's...
It's really simple.
There is...
See, there is...
I'm wondering if we have chalk because I can just do the drawing if we...
Perfect.
Okay.
Cool.
So I'm going to...
So we're winning it because...
Well, we figure this out.
So there's CWFS.
There's HJFS.
And then there's...
Fossil, which is the classic Bell Labs file system
that you get on the Labs distribution.
The Labs...
Nine front got rid of it because it kept crashing and losing data.
And then there's CW...
Or then then...
Recently, we've added GFS, which ships by default.
And...
GFS still keeps on crashing and losing data.
At least it seems to be getting better.
And...
And actually, I've been using it for...
I'm joking mostly.
I've been using it for a year.
I haven't lost data with it yet.
Other people have, and we've found and fixed those bugs,
added a bunch of testing, which I'll talk about in a bit.
And...
Basically, made that work.
So what I'm going to be doing here is talking about the general...
structure of the file system, how it works internally,
and what the kind of path forward is.
So...
As a quick review, all file systems on plan nine are network-based file systems
that talk over nine p.
Nine p is a protocol with, I think it's 13 operations.
And most of them don't matter that much.
I mean, they matter, but most of them aren't that relevant for the interesting parts.
You've got...
The important ones are...
Read, which has the...
T message...
and they...
and then the response.
Right, and it's response...
Walk...
and it's response.
And I think these are really the important...
Oh, and open.
Which...
Basically, open is permission checks.
You can think of Walk as basically taking a...
what the protocol calls a FID, a file ID, and moving it around.
So you start off with a FID when you attach to the...
to the FES.
So attach is basically where you start the whole...
handshake process to authenticate and get...
connection to a namespace.
So attach...
and then Walk will basically start...
and attach gives back a FID for the root of your file system.
Read...
I'm sorry.
Walk...
It moves the FID around the file system.
So you start with the root for...
the FID for the root, clone it, and Walk to...
home-or-e.
And that gives you a new FID that you can use for open...
read, right, et cetera.
Okay.
So that's the protocol.
How do we implement a file system to do this?
So the idea behind GFS is...
Just...
Email.
Welcome.
We don't have a eraser.
Okay.
Oh, wait, there isn't a eraser.
Never do.
I'm learning how to do this on the fly.
Yeah.
Okay.
So...
So how do we do this?
I'm going to go through the layers one by one and...
kind of talk through it.
So GFS is basically built in three different layers.
You've got the...
the FFS layer, which sits on top of the key value layer,
which sits on top of the...
the block layer.
And all of these just kind of stack nicely.
So the file system layer is what takes things like Walk
and turns them into basically...
operations on a key value store.
The key value store takes these operations in term...
into IO on the block and cash layer.
So what we end up doing here is...
Okay.
Let's take an example of...
Thank you.
So let's take an example of file system hierarchy.
We have...
Let's just say...
All right.
Let's give it a...
About a plane 9 name.
And the expile contains...
Right?
Okay.
So we've got...
User or EX, which says...
Hello GFS.
How do we map this into a hierarchy?
So...
What we would do is we would put in a key value pair that says...
Let's give it ID...
Negative 1...
Empty name.
And that's the root of your file system hierarchy.
So at the root of the file system hierarchy...
What do you need to do to look up files?
You need to know the ID of the root.
And the...
So what I do is I have a map that...
Let's say...
One...
Or...
Yeah.
Okay.
Sorry.
Here I would have the permissions.
So ID 0...
And...
I don't know.
Owned by or...
File size is whatever...
Basically everything you get out of staff goes into the value...
Packed into a buffer.
Now this file ID here is the idea of the...
Of the directory.
And let's just say I also have...
User...
I don't know...
Rob.
Because...
You know...
I always share my computer with Rob.
Um...
Okay.
So I've got...
So what I would have in the key value story is a file...
Starting with...
There's...
Two files starting with the...
Sorry.
One file starting with zero.
And name is...
User...
Which maps to file ID 1.
File ID 1 would have...
Or...
Orian Rob.
And both of these start with file ID 1 because their parent directory is user with file ID 1.
So this is how you end up getting directories.
It's all a flat listing.
But the prefix of the directory in the flat listing gives you the directory hierarchy.
So that means that every file that starts with ID 1 goes into the directory user.
Everything with ID 0 goes into the directory flash.
So this...
This works inside of key values.
This is how you map a tree to a key value start.
And then how do you map the data for the file?
Let's just say I've got...
Sorry.
In...
I'll call this one two.
So in two I have a file called X, which has ID 3.
Right?
So...
What I end up doing is ID 3...
Offset 0.
I get a block ID here,
which points to the data on to the block on disk.
That contains the contents of the file.
So even the file blocks going to the key value store.
Which means that you have to make a pretty fast key value store.
Okay.
So how do we do that?
So that's how we map the hierarchy.
Let's talk about key value stores.
So the...
Got damn it.
It would be really nice if I could just flip the...
Flip to a new page.
Okay.
You might be able to press the person.
Ooh.
Cool.
I will let you get to...
Oh, we only have half the screen.
Ah.
Wonderful.
Let's see.
I'm just going to keep going until you've gotten...
Flip through the slides and let me know when you get to the right place.
Slide plus.
Same as it.
Okay.
So the key value store is based on what's called a B-epsylantry.
And a B-epsylantry is basically...
How many people here are familiar with B-trees?
Almost everyone.
Cool.
So I'm not going to spend much time on it.
I'm going to say that B-trees are like binary trees but fat.
So you have a B-tree which has a bunch of...
A note with a bunch of children.
And these children fan out.
All right.
And you get...
Yeah.
Other blocks that also have a bunch of children.
Right.
So that's a B-tree.
How is it B-epsylantry different?
It's actually really close.
The difference is that all the internal nodes,
the ones that have children,
get split in half.
So the first half of it is exactly like a B-tree.
Where you fan out to all the other children child nodes.
And they'll save.
These are also internal nodes.
Okay.
So you get the fan out.
And then you end up with a children here.
So what happens with...
What's the file?
What do you mean what's the file?
Just go to...
Hit slide plus until you see...
Ask Eric.
Oh, that's going to be a problem.
See each problem plus X file.
The slide plus.
And then it'll work.
See it...
Make it executable.
Okay.
So what we do here...
So what...
Okay.
So what do we do with the other half of the node?
What we do with the other half of the node is we put messages into it.
And a message in this case is basically a deferred update to the leaf.
So what we can do is, for example,
let's just say I want to insert...
I'll call it the operation is called insert.
And I'll say file X to create X, right?
And what I'll do is I'll put a little message here that says insert file X with these attributes.
This is...
Get...
Insert it into the root node of the tree.
And...
Stay isn't it?
When you read from the tree, you walk down to the leaf and say,
Do I have an insertion for file X?
Change the name of intro 1 to intro.
Does not have it.
Then next go to...
Go up the node.
Does not have anything about file X.
And here we have a message.
So we basically reconstruct the key value pair and we get the file.
Now let's just say I want to touch the file.
I don't actually need to read anything about it because all I'm doing is updating the
Modification Time blindly.
So what I do is I create a...
Oh...
WSTAT, which is...
WSTAT is right STAT.
And it will just create a new message here without actually even reading a damn thing.
It will...
And we can basically toss messages into the tree like that.
So this is...
And then what happens when this message buffer fills up?
We basically flush down to another node.
So this...
So we get the messages kind of flowing down the tree slowly towards the leaves.
Why do we want this data structure?
Well, the rights become a lot cheaper because you're not actually doing...
Okay.
The rights become a lot cheaper for multiple reasons.
The first one is that you're not actually necessarily reading any data to do the right.
The second one, which is a little bit more subtle.
For a file system that is crash safe, you really want to do copy on right.
Now, in a classic B tree, if you do a copy on right, you're doing a copy of the root node,
and a copy of the child, and a copy of the leaf.
And now you've just done three rights to update one value.
Or four rights because you have to actually update the value too.
With a Bepsal entry, what you do is you take a copy of the root node,
and you're done.
You've updated the data structure.
So now you end up with a lot less right amplification.
ZFS uses a classic B tree, which put means that they have a lot of...
The B tree itself is simpler.
You don't have to reconstruct values on the way up.
But to make it perform well, you have to do a lot of work on how do you batch transactions
and avoid a bunch of copy and copy and write amplification,
that I can just kind of go, whatever.
So this is, and you might be asking yourself,
well, if this is so great, why doesn't ZFS use it?
And the answer is very simple.
ZFS was released in 2003.
Bepsal entries were described, at least, became well-knownish in 2015.
So 12 years of time is the difference.
Okay, so this is the key value store, and what we do for...
So this gives us a lot of nice stuff.
For example, if we want to take a snapshot,
well, we need to keep a pointer to the old value,
and we keep a pointer to the new value.
And that's your snapshot.
You just don't delete the old stuff.
The way we do this is that GFS is actually a tree of trees.
So we have a, yes, I can switch to opt-out from now.
If it's working, okay.
Cool.
So this should actually let me move a little bit faster.
Good to just move this down.
Okay, good enough.
Good enough presentation.
Cool.
So this is basically what we were talking about just now,
with the file system layer, key value layer.
So right optimized variant of a B3 is what the Bepsal entry is.
I just kind of described it.
It's complex and painful to implement,
but everything else once you have it gets simpler.
So the fan out, messages, updating inserts to the message buffer.
And then when the message buffer fills, we flush to the children.
We apply the, so basically the messages turn into a leaf node,
or into key value pairs when it reaches the leaf.
And I just explained why it works really well with count.
So here's an example of what copy on right looks like with this.
The old tree and your new tree.
Look up is, again, a pin in the ass because once you walk down to the leaf,
you have to kind of walk back up and get to a full key value pair
that you can actually return to the querying layer.
So your base, so every read ends up doing a little bit of reconstruction.
This isn't too expensive, but it is a little bit of overhead over a regular B3.
And then listing a directory scanning is even more complicated,
but it's still fairly fast.
I have limited time, so I'm going to just skip over the details of scanning.
If you want to talk to me about how this works internally at any point,
just feel free to grab me after this.
But basically, listing a, the important thing to remember is just listing a directory
is a prefix scan over the keys.
You look up the idea of the directory you're listing,
and then you say, give me every file that starts with it.
And because it's a B3, all the keys are close together nicely sorted.
So you get really good locality.
In search, it's similar to the rules or similar to B3s,
except there, again, there's a little bit more complexity because
when you merge a node, well, in a B3, you only have one thing to consider
when you're asking, can I merge it?
Do I have room for the fan out nodes?
So if you've got two nodes that are combined less than half full,
you can just kind of merge them.
Here, you've also got the message buffers to consider.
And what do you do here?
Do you flush them?
Do you not?
Do you?
What do you do?
I just kind of do the lazy thing and say,
if the message buffers full, even if you could flush the tree,
just don't.
So there's room for optimization there,
but it turns out that even with the relaxed merging,
the tree nodes usually are pretty okay.
And the balance isn't bad.
The fill factor isn't bad.
Occasionally, if you're running check on the file system,
it will tell you, hey, you've got an under full pivot.
And it's not perfect for performance,
but it doesn't seem to be a problem.
Okay.
So I've talked a little bit about absurds already,
so I was talking, I was, yeah.
So what I do for the file for snapshots is basically,
this is where I got into on the whiteboard.
I have two trees or two layers of trees.
I have the snapshot tree,
which is also a Bepsalon tree that points to the roots of each snapshot.
So your snapshot tree points to your main snapshot,
which is basically the default name that you put your data into.
Your admin snapshot, which is where your user database
and a couple of other file system and many things go into.
And the reason this is separate, by the way, is because,
well, if I have different snapshots, different user lists,
who has the permissions to do what?
So the user database is stored separately from all the snapshots
and is used across all of them.
And then your dumps, which is basically your history.
Okay, so that's basically how snapshots work.
And by the way, I should have said this at the start of the talk,
but if you have questions at any point,
feel free to stop me and I'm happy to explain a little bit more.
Yeah, I know this, do you have a support for the history of coal mine
for GFS name or a support for the yesterday coal mine?
Yeah, I will do that.
Okay, so the block layer.
This is kind of the, this is where we actually put the data on disk.
This is where the rubber meets the road.
And basically what we do here is also trying to be roughly,
well, this is very ZFS inspired.
So it's trying to be, but it also simplifies a lot of things.
So what we do here is we take a disk and we split into a bunch of arenas.
So you have your disk and I think we default to eight arenas,
but the number doesn't actually matter.
And what the arenas and each arena is a pool of fixed size blocks.
So each block is the same size.
And then if we want to grow the file system,
we support that.
We can, you just add more arenas to the end of your arena list.
The slide plus.
So what, and the way we keep track of the blocks is we just have a,
in memory we have a avial tree.
And in, on disk we have just a flat log of regions.
So the regions basically say, you know,
it's, this block is allocated from here to here is free.
And, you know, when we want to allocate a block,
will you just append to the log and say, well, this block is allocated.
And this, this block has been freed and this block has been allocated.
This block has been freed.
So as you can imagine, this ends up becoming a fairly long log of,
basically, every operation.
So every now and then we basically take that log and compress it.
So we take the, like, if you have block x that you allocate,
and then you free block x later,
then it just turns into one entry that records the final state of the block.
And that's taken done by taking the avial tree,
and writing out the set of ranges to disk.
So the avial tree basically maintains, you know,
maintains the state of the whole disk in memory of these blocks in memory,
which can be, yeah, as a set of ranges,
and is just kind of updated in place.
Okay, so, yeah.
The blocks themselves, we can have,
every block pointer is a tuple of three values.
It's a address.
The hash of the contents of the block, which is how we detect corruption,
and the generation, which is used for snapshotting and,
and space reclamation.
The, ah, oops.
Yeah, one less per arena, which means that,
and the reason we keep one less, the reason for arenas,
which I forgot to mention, is because we,
when we compress the log, we basically can't do anything,
and compressing the log can take time.
So if we split the disk into arenas,
we can do log compression in the background,
while we allocate from the other arenas.
So this basically allows us to concurrently maintain the file system,
and, ah, and actually, ah, use it.
Ah, oops.
I should have done one other thing, but, ah, okay.
So I did shorten this presentation a little bit,
because, no, I have limited time.
I'm actually coming up on the end of it.
So, okay.
Last thing, how is this thing tested?
So we've got a fuzzer in the file system,
ah, which will generate random operations,
ah, as quickly as possible, and just kind of absurd,
and do consistency checks all the time.
Ah, we've got, ah,
tester, a virtual disk that,
we can use to simulate block by block IO,
so you do say a build of go,
and then we do a replay.
We start with an empty file system,
and then do the first block IO operation,
and check, is it consistent?
Then do the second one and check.
Is this consistent?
And then do the third one and check if, if this is consistent.
And then, ah, about 100,000 operations later,
we've gone through the entire go build,
and, ah,