Thank you.
So our next talk, continuing in the version control systems theme, we've got Carol, he's
going to be talking about gets and rust and get oxide, okay?
Thank you.
Hello everybody, you can hear me at the back, right?
Sounds great.
So today, sitting in the rust room today, I learned two things, falling into the pit of success.
A great way to do this is to speak about how great rust is.
And we definitely going to get there to get a bunch of laughs out of the room.
I learned with Dunkham C++ or C works every time.
I'm going to talk about taming git complexity or just doing weird stuff with git.
So let me just introduce myself real quick.
My name is Carol.
I've been coding since 2004.
I've been hacking on version control since 2020.
So I really like version control to keep doing it.
And I'm a co-founder of a company called Git Butler.
That's enough.
So let's talk about why you want to hack on git stuff.
What do I mean by hack is programmatically operate git, right?
So what that means is maybe you want to do some useful things like package managers.
I don't know anybody working on package managers, package management in the audience.
Person there actually quite a few.
Developer tools, anybody?
Oh wow, that's actually a lot of developer tools for folks.
Is anybody automating or customizing your workflow in git?
Actually that's half of the audience.
So this is going to be relevant, I think.
I think it's also real fun programming git or just working with the data structures that our version control gives us.
And just doing stuff without it's it's kind of fun.
So our agenda today we have three themes.
So I'm going to introduce you the git data model data structure and kind of tell you about how awesome it is.
And why it's awesome?
I'm going to show you how to build stuff with a library code, git oxide.
And git oxide is really cool.
So hopefully by the end of this talk you'll be pulling it from GitHub.
And then I will do a very quick demo for actually having implemented the stuff with git oxide.
If you have any questions, heck on me.
You know, just shout that's fine. I love it.
So how many of you are using git?
Literally everybody, right?
How many of you know the data model of git?
Like how should is stored on disk?
Like about half of the room.
So I think git has been partly, partly the reason for why git has been so successful is
like the simple, very hackable primitives.
And what I mean by hackable is like easy to work with.
So in git everything is content addressable.
Like you have the content is the identifier for the data.
That's cool.
You have snapshots instead of disks.
So that means that you're looking at disks but the system underneath is writing snapshots.
That's the limitation but that's also what makes it simple.
And git works on the immutability principles, right?
So it doesn't change afterwards.
It makes things simpler to work with.
So I'm going to share four concepts with you about the git data model by the end of this.
It's not going to take long but the end of this you know more about the 90% of developers out there.
So in git you have blobs.
So content is everything and this is essentially this content addressable nature I mentioned earlier.
So in git a blob is the file content and a hash fruit.
That's it, right?
Like in the example up there you know your file content you get a hash out of it.
This is abbreviated and that's your blob.
There is no file name.
There is no metadata.
It's just just a bunch of content.
And that means it's reusable.
So if you have two files with the exact same content you don't have another object for it.
It's the same thing in the object store.
So that's number one.
Blobs.
So we're going to build up from blobs into all the way to a version control system.
And then you have the concept of trees.
In git we talk our tree this tree that sounds like a complicated structure is just the file system.
It's just like a you know a directory organizing the blob into a snapshot of a directory.
So as you see the example over there I have two trees.
And you see that those trees now you have the file name and a bunch of you know like a little bit extra metadata.
But these trees are pointing to the blobs that we spoke about or other trees.
And that's kind of how how the system builds up.
Then you have.
Yeah, exactly.
So you're pointing to blobs or other trees and trees in in this structure are your main plus mode plus a hash.
There's nothing else that's all.
Building on top of that you have commits.
Now commits is the term that you work with every day.
And it's a snapshot in time of your work.
And a commit is essentially a pointer to a tree plus a little bit of metadata, right?
Like you have the author this little commit message bit stuff, right?
I'm sure nobody writes commit messages like that.
And as you know commits form a graph, right?
You have a commit one of those metadata pieces that you work with is the parent or one on multiple, right?
Like you have a parent that precedes it and that's kind of forms your history.
You can traverse it and that's what happens when you log your history.
And if you happen to have more than one parent, that's called a merge commit and those are wonderful.
All right.
So finally the last concept that I'm sure you already understand but references.
And this is just branches.
Like let's focus on branches for now.
It's simply names for commits.
That's all.
And updating a branch is just moving the pointer.
So if you create a new commit, the pointer moves, that's all.
It also happens to be the case that there is this special branch called head, which is what I have checked out right now.
This is more or less the interesting parts.
With this set up, you have now enough knowledge to be dangerous.
You can go ahead and start editing the, no, you should not edit the get directory directly.
Things will get messy very quickly.
But this sets you up for using the get plumbing.
And what that usually means in the context of get is like the plumbing commands, right?
Get hash object, right?
Like you take, take a blob, you get the hash out of it or cat file to get information about the object database.
You don't want to do that, right?
Like one of the reasons, like there's many reasons why you don't want to talk exactly these commands.
But the number one reason is it's kind of rigid.
You can't get all the things that you want to do with those commands.
They are built for the get kind of workflow.
The way the way it's been always known.
So you want something more flexible.
So what would you imagine?
You may want to say, how?
Well, get this open source.
I'm going to use it as a library.
So going to the code base, there is this libget.a.
And you think you may want to link this into your application and use that.
Sounds like a great idea.
But you can't.
This does not work.
And the reason is this code base is not set up for that.
It's not appropriate.
It does funny little things.
Like inside functions, it thinks it owns the process.
So it does this wonderful, wonderful, die function.
Exit your process.
You don't want that randomly sprinkled.
So you can't use libget.a.
So this was the part where I was dunking on a secode base.
Now, this also means you need the library.
Or like you obviously need the library if you want to access
programmatically the get stuff.
And I'm going to give you a very unfair list.
Why is it unfair?
I'm not going to cover everything.
There is legit too, which if you have, if you are a rust user,
you may have seen the crust get too.
Anybody use the create get to in the audience.
A few people.
So this is rough bindings for libget to.
And libget to is great.
It's been around for a while.
It's mature, et cetera.
So it does a lot.
There is other libraries.
That's why it's unfair.
I kind of bucket it into other.
You have jagged, go get et cetera.
And then you have get oxide.
The rest of the talk is about get oxide.
So get oxide is rough, pure rust implementation of these structures.
This is just a very quick map of what's the ecosystem like.
Every pixel in that image is a line of code.
And that's a lot of pixels crammed in there.
So in the background, this is the C get code base.
That's the one with the die and shared state, et cetera.
That's the background.
And then you have the jagged.
It's a large library.
It's been around for a long time.
There's go get libget to.
And then in the top right corner, that's get oxide.
That's at least as a 24, the size of the project.
Right. So let me introduce to you get oxide.
Anybody here who has used get oxide?
All right, quite a few people.
So it's a library by Byron Sebastian.
It's actually a really cool project.
So what's really cool about it?
Our tradition is it's get compatible.
So it does what get would do.
Right, it doesn't do anything weird.
Objects that it writes are going to be compatible.
It reads it the same way and so on.
It is safe.
It is performance.
It's performed because, you know, like a memory saved,
you can paralyze and so on.
But the part that's interesting is it's also comes with scalpel
is included.
And that just simply means it's very flexible.
You can do a lot of weird things,
or you can manipulate stuff in a flexible way.
So that means the library gives you a lot of knobs and buttons,
things to tweak things to adjust.
This is like a maybe too much of an example,
but this is, you know,
imagine that we're going to clone using get oxide.
So you can configure your remote.
You can do a shallow cloning.
You can pick a specific branch.
You can override configs.
You can customize you fetch,
like skip the tags.
In this example, you can track the progress.
If you want, you know,
you kind of display to the user in any way,
etc.
Or handle the interrupts.
But the thing that's actually is really cool is also it allows you
to do kind of more sophisticated operations.
You can be doing your updates automatically.
You can do stuff in memory.
Or in this case, you can prevent race conditions by saying,
OK, I want to mute my references in a particular way.
And I'm going to open a transaction, prepare it,
and try to obtain a lock.
And if it can't, it will fail.
And then do everything in the very end automatically,
which is kind of neat.
If you want to do stuff with the own,
or if you're doing something specific that you want to make edits like that.
So I'm going to run you through a quick real, real-world example.
So pretend, let's pretend that we want to be a good get-ism,
right? Like what that might mean is you want to make you commit
super, in a super focused way.
You have, you want to do one thing and not another,
you know, just like a small focus commits.
And sometimes you may want to move stuff between commits, right?
Like, in this case, let's pretend that this file that went into one commit
is supposed to go somewhere else, right?
Or somebody in code review told you, hey, split your commit.
So how would you do that?
Does anybody change their commits after they've written them?
Okay, that's the whole room.
So let's pretend that we do this get gymnastics together, right?
You have to do an interactive rebase,
and that's the happy case scenario.
So what do you do? You do an interactive rebase.
You mark your source and destination as edit.
You then reset to a particular point,
and then you stop a commit and reset there.
So let's say you're extracting this utils file,
and then it's out, and then you continue your rebase,
and then you stop at the destination commit.
You add, you know, it's a dance.
And if you have the commit's deal,
we're around, you have to do the, what I like calling,
the double interactive rebase sandwich.
You don't want to do that.
Or it's not very pleasant.
So let's implement that using GitHub site.
This is doing weird stuff with Git after all.
So let's see.
So you've now imported the library,
and you have gigs open.
So you open the repository in memory.
So that's what we're doing here.
It's open with object memory, okay.
So here's something that we've set up.
It's not actually gigs, but we set up something with Git gigs
primitives.
And then you look up the commits,
like you have the hash as you can look them up.
What GitHub site does, does it,
it loads these lazily, right?
So you're not parsing the commit message and everything,
until you actually access it, which is pretty neat.
And in this case, we don't need it until, or we only need the tree.
So, next, we need to subtract changes from the first commit.
Let's say, so in this case, you need to get the,
the div spec of what you're subtracting, okay?
Well, let's say, we have some function doing that.
And then you have to do like a removal.
You want to extract things out of a tree.
And you build this with, you know,
GitHub site primitives that later you can, you can say,
okay, I'm now going to swap out the tree that had my change
with the tree that does not.
Next, you want to inject that change into the other commit, right?
Like to do it automatically.
And here, we're doing a three way merge,
sounds interesting.
You do need a three way merge of, you know,
where the base is the, the source without the changes,
then the one side is the, you know, the part with the tree changes.
The other one is the destination where you want to actually put the,
put the changes.
And then you do the same swap operation.
And then do everything in memory and at the very end you write to disk, right?
So, time for a demo.
So, I showed you in theory how, how that could look like.
So, I'm going to demonstrate that it actually works.
You can do that.
So, it's a very quick one.
Just one very small example.
So, over here, is this big enough by the way?
No.
Okay.
People are the front of saying the size is perfect.
So, I'm going to go with that.
Okay.
So, this is just a commit history.
And let's, for a second pretend that this commit,
we don't, we don't care about the message.
We only care about the fact that this read me.
It does not belong there.
It belongs in this commit five, three something.
So, what you, um, a pair of them, two times.
Okay.
So, what do you, I will make this bigger.
So, what I can say is I'm moving this read me from,
from that F7 commit into commit 53.
Run this.
Done.
Let's actually verify that it worked.
If I, um,
if I run the log again with this,
right there, right?
Single atomic operation.
So, if it failed, it would fail in memory.
It will let me know.
And then kind of not proceed.
Or not reach of partial success case.
And let's actually, since we have more time,
let's, let's move it somewhere else.
Let's, let's move it in this commit 24.
So, let's say, um,
uh,
BD, uh,
24.
Right?
So, we've done it again.
All right.
It's the same, um, kind of move operation.
Now it's, now it's in the,
that's very fine.
Now, you have, right?
It's now this commit.
So, you know,
you can do stuff like that with, uh,
library access to, to, to, to get, uh,
get structures.
Okay.
That's, uh,
that's it.
I'm gonna leave you guys with just two final takes.
One is kind of silly.
The other one is a little bit more,
philosophical.
Or it, it will,
it will tend to be philosophical.
Now the first one is that,
get this awesome,
and get excited is pretty cool.
If, if you haven't,
you should try it out.
After this talk,
get the code.
Check it out.
And, um,
get them on is the philosophical one.
So, when facing a problem,
uh,
we always have a choice.
And the choice is to either spend a few hours
and do the boring work,
or spend a few weeks,
uh,
building a tool.
And since this is about hacking,
uh, hacking with get,
I would encourage you to always choose,
uh, making the tool.
After all, we are on a Sunday,
talking about the rust,
uh, like listening about rust.
So, um, yeah,
I think that's, that's the right choice.
So,
thank you very much.
Uh, we have some time for Q&A.
Thank you.
So, we have some time.
Some questions.
This way.
Yeah.
Um,
uh,
questions.
Uh,
uh, one question and one,
we're much.
Can you, can you show back the,
uh,
when you did the,
when you did the demo,
we can show it here.
When you,
when you did the demo,
uh,
it doesn't tell you that you created a commit without any content,
because when you move the file between the commits,
the photo original one did not have any content left.
Yeah.
Doesn't tell you at all that you,
that you know of an empty commit in your story that doesn't do anything in the,
right.
So, I think the question is why was there no content after extracted the file?
No.
I get why it doesn't content,
but why it doesn't tell you that there's no content left in the,
you have an empty commit in your story now.
So, so this is,
well, this was demo, right?
Like it was just like an example of of what you can't,
I can do.
So, I think what you're saying is it makes sense to inform the user that,
oh, you're going to make an empty commit.
That's completely fair.
Um,
probably should.
Ah.
Yeah.
Okay.
And just the remark.
I think there was an earlier presentation yesterday,
in a 254 of Git,
there would be comments to modify the history of different commits.
And for instance, you can now,
you will be able to split commits.
We want them, and we write their content,
after they've been landed and such.
So, I think that would be interesting for,
yeah, for this.
Yeah.
I guess what you're saying is more and more of this functionality is kind of,
you're going to mainline Git.
Yeah.
I mean, yes.
You could.
At some point we will have perhaps some of these in here.
Any questions?
Hi.
Often, I like to change the metadata.
So, for example, in this case, you move the content.
Does it retain the metadata in this case?
Does it overwrite with the current metadata?
Well, so the question is,
should you change the metadata as you made this kind of operations?
Like, it's up to you, right?
Like, it's about using the library and what you could do with programmatic access.
So, if your use case means that,
oh, the metadata needs to change,
like, the timestamp or something, like, absolutely.
Do that.
Is that handled to do it?
Pardon?
I have the capability to do it.
Yes.
Right.
Like, because you have all the bits and bobs to tweak.
Great. Great.
Then I thank you.
Git or, and I've used some internal Git libraries in the past.
And re-impermitations of Git in different languages have or suffer from performance.
Like, packing objects and compressing and all the weird internals of Git.
I imagine you guys may have done benchmarks against base Git and Git Oxide,
but is Git Oxide fairly close to, to roll Git in terms of its pack and clone type performance?
So, the maintainer and author of Git Oxide is very performance-conscious.
And so, he cares very deeply about making a performance library.
And that means that a lot of the aspects of the library are fast.
And I know that cloning in particular isn't, isn't, isn't nice and fast.
Because you can do things, you know, it's able to parallelize some of the workload.
So, some of the operations are faster than mainline Git.
And yeah.
What algorithm does it use for?
Sorry.
What algorithm does Git Oxide use for pack for doubt to a compression?
The question, I do not know.
I just don't know the answer to this.
Sorry.
Yeah.
I think that's, that's all the time we have.
Great.
You can catch GitHub.
Yeah, you can hold outside if you guys want.
Yeah.
Thank you so much.
Thank you.