I want to send one disclaimer and say that this is nothing to do with what I do for my work.
At the McSmanke Institute for Informatics, so I also do free software and that's what this
is about.
Okay, and I actually wanted to start a bit with my motivation here.
So a bit of a very short story, one of conviction and passion because this is really a passion
project for me.
I'm Lawrence and I contribute to radical and you might be wondering, okay, I mean, you
will find out what it is, right, but why maybe you have read the intro, why do we do
this and so on?
And Phosem is developer focused and we learn about lots of interesting technologies here.
We have learned a lot about peer-to-peer technologies and radically is a project that
really wants to bring this back to software development in one particular way.
So it's about the future home for open source.
I like Git.
You could substitute this with other version control systems, of course.
I think Git is pretty fine.
But what I really like is the decentralized part of Git, right?
The way it was designed to be used, the fact that you do not rely on a central party.
And I don't like decentralized Git, which unfortunately is what large parts of the software
ecosystem in general have gravitated to.
In the free software world, maybe not as much, but in the commercial part and also the overlap
between commercial software, large companies and free software, there's big centralization
effect.
There's this big pool to GitHub and a few other forges.
Of course, there are people that run their own GitHub instances for J.H.O. instances
and so on, but if you really look at the size of GitHub and decentralization going on
there with one company controlling this, this is just frightening, right?
So radical goes back and asks, okay, what about decentralize Git?
How can we make this decentralize Git work?
What is actually needed?
So the idea behind radical is not to come up with some completely different source controls
software.
It's just okay, we take Git, which works, which is widely known.
Then we figure out how can we make it work so that this scales and is one approach to
use Git in a peer-to-peer decentralized way again.
So turns out that I saw radical and what it does with Git and hard work, and that's why
that's why I'm here today, that's why I started working on this.
So yes, it's about this future home of open source, and I mean, yes, I will present radical
today, but anything you can do to work on decentralization of your software development stack
will be beneficial.
We need way more ideas in this space.
There are other ideas, just to mention, I have already said like for J.H.O. before, they're
looking into Federation.
There are also alternatives like Tangled, Building on top of 80 protocol.
There are people that just run their own Git service.
These are all valid in their own sense, right?
Just because I'm presenting radical today, I'm not trying to invalidate any of these other
ideas.
I'm just presenting one in this whole toolbox, and to my knowledge, radical is the most advanced
peer-to-peer forge.
I know, if you know, I'm any better than let's talk, but so far, I think we're pioneering
the peer-to-peer part of this.
Maybe to drive home-wide the strongest, I think, or how it became the strongest.
It's a project that has been going for a while, and has tried out different ways of doing
this.
There were previous iterations that were based on IPFS, influence by secure scuttled
but and things like this.
These were the previous iterations.
Now it just works on top of Git, uses Git to synchronize, so really when you synchronize
on the radical network, you just do the Git synchronization protocol and not some other
file system file sharing protocol.
Radical is now developed on Radical since roughly three years now, and we have released
version 1.0, quite a while ago, and we're now at 1.6.1, and pushing onwards.
Our release cadence is roughly one release every one and a half months, currently, and
we are trying to get to one release per month.
So we actually have luckily, we have two talks about Radical at first them.
You are now in the talk that is more focused on the peer-to-peer stuff.
Also, very much influenced by the local first idea, which obviously ties in well together
with peer-to-peer, but I will be not talking so much about the local first aspect of it.
If you are interested in that, phinto-haps will be talking about that, basically right after
my talk in a different building, in a different developer room, just so you know, okay,
so you can also check this out.
And this is in particular for our social artefacts, because if you think about patches,
for code review, think about pull requests, issue tracking, and so on, if you just
may even put that into git check in a two-do.txt file or something, then you will get
like merge conflicts on your two-do.txt, which is, yeah, I'm not really nice to work with,
so we use CRDT's to work around that. But as I said, this is not this talk,
it's the other one. So this talk will be about more the peer-to-peer aspects.
So there are no servers, there are gossiping between peers. There is no central user database
like with GitHub, GitLab, and the likes also not with the federated setups where at every
federation point you have a user database, so we don't do that. And the ideas, of course,
that users of the radical protocol kind of share their storage and their availability.
So you should be able to contribute to the network in that sense.
Maybe one detail here is you probably know that on GitHub you can start a repository,
and oftentimes we get the question, okay, how do we do that on radical?
And our answer usually is, well, forget about just starring and incrementing a counter
in a database, we suggest you just replicate that very repo, which means you download a copy
of that repo to your computer and you make it available to be shared by others, which we think
is much more sensible than just putting a star on some website. But okay, and the user is always
in control, this decision of which repo do you want on your machine is yours, and there's
no way that we will ever like interfere with that, of course. Just so you get a better idea of what
is like looks like, the look and feel as a radical user, I still want to go through the three main
interfaces that we have before we get started. So there, of course, is a CLI interface.
You can see here, this is just the list of repositories that I have and the very left,
you have the name, the repository ID, which is some sort of hash, the visibility of the repository.
So you can see that most of the repositories I have on my machine are public, but there's also
the possibility to have private repositories ahead of the repository and the description.
Then we have a way to obviously inspect which other peers, your node is connected to,
so this is just basically the list of connections you currently have in the network.
By default, your node, if you don't change any setting, will attempt to hold eight connections,
so it will attempt to mesh out to eight other nodes, which then makes it usually very quickly
to converge on the latest status of a repository. Two onion nodes are shown here. I didn't
configure the connection properly, but onion connections are supported out of the box.
I think I disabled it on when I did that screenshot. Then you have things like patches,
so this is the history of a patch. Think about it as a merge request or pull request,
so you can see the updates, you can see the revisions that were posted. These revisions are
immutable, which makes a lot of sense for reviewing, so when someone posts a new revision,
you have this clearly identified thing, what is a revision not something floating around on
a website that changes and so on. Then there's a list of patches where you can get the status.
We also have a desktop application, which essentially shows the same information, but it's not
a CLI, it's a desktop app. You can also search through your repository listing on a different
machine, on a desktop app. You can do the code review features are a bit more advanced on the
desktop app. You can do like line comments and so on, and then we also have a web application,
which, yeah, to refer, looks a lot like the others, so the code website you already know,
just makes it a little bit easier to find out more about the radical-mated data,
like who owns the repository, what is the repository ID and so on. Right. And now,
a bit about the high-level design, as I mentioned in my introduction, we build on top of
Git, okay, so we try to use as much of Git, and it's nice features as possible. So by using Git,
we immediately get like continuity, immutability of data, and efficient sync protocol, and relatively
wide acceptance. So by putting, for example, all the social artifacts, like issues and patches,
into Git, we also get efficient syncing of these. Right. We also get these nice immutability effects
that a revision of a patch is in some sense. It's not exposed to US that, but it's in some
sense a commit, so it has also a commit ID and it's immutable. And yeah, this makes it
it makes it relatively easy to also replicate and so on. But Git, I mean, if Git would have
been the perfect peer-to-peer system to do development, then people wouldn't have jumped
onto Git up, right. So there is something, something that is missing, that's the reason,
why this even is, or part of the reason, why this ecosystem even developed in the way it did.
And we think there are a few important aspects. So we have to add something on. So
globally, unique repository addresses. Well, now if you go around here and watch different talks,
in many of these talks, you will see github.com slash something. So obviously, I know that technically
it's not the one de facto global unique address space, but it's certainly the address space
that is controlled by Gitup.com and these addresses have become very prevalent. So if someone reads
off their slide and wants to mention the repository, they will usually even omit the prefix Gitup.com
and they will just say, you know, this slag component of the URL. This is something that is,
of course, very, very valuable. What we are doing is we have a way to globally and uniquely
address repositories. In our case, this is a hash and you can look up repositories via the hash
name and then from there, you can look at different metadata of the repository, which also contains
a human readable name and so on. But we think that this slag of global addressing is a bit of,
I mean, a bit of a problem where we want to improve Git, because what doesn't seem to work
is that you have many different git servers and you have to configure remotes all by yourself.
If you want to have lots of connections to different remotes, because you are in a peer-to-peer network,
with just playing it, you would have to set up each and every remote. So for every
collaborator you want to work with, you would probably have a remote. And for each collaborator,
you would have to think about addressing that collaborator, like what is maybe their IP address,
or which is the onion service that they are running and so on. And we want to step away from that
and address all repositories the same as kind of a view onto the network.
Then of course you need something for metadata. As I mentioned before, this metadata of
a human readable name is important, but it's slightly more than that. It's like who governs
the repository, who gets decide to push to which branch of the repository,
what are like extension points for the repository. Maybe you want to also note where you can find
mirrors of the repository and so on. So we also have a schema for that and we do access control
over this and you can actually, we have a mechanism to share the ownership of this repository
metadata and identity document, which then doesn't live at one person's name space, but at the
name space of the repository. And if, for example, you want to migrate a repository that's
like a regular Git repository onto a radical, then basically the only thing you do is you create
a new address that is based on the initial metadata and work from there. And if you ever need
to take up the work of someone else, like the project went dormant and you want to revive it
and you want to bring it to radical, you can still do that. If you are in the case that you
have to fork a repository on radical, then what you would do is you create a new address for it
and add the respective metadata. And something I've also watched the talk this morning about
the PGP and the nicety of a completely self-sovereign identity. And yeah, that's also something
that radical supports. So in in in in radical every every user is really self-sovereign. So it's just
an AdWords 250519 key and from there on this is how we bootstrap everything. We lack a few
features. I have to be honest in terms of like migrating your key, exporting your key, multi-device
and so on, but this is all on our roadmap. And this idea that the user is in full control over
their identity is still fully respected. Yes, and the discovery of replicas, of course, if you do
like if you think about just using Git alone and you want to discover who else might have an
interesting copy because they have written a patch or so without any server, would get really
tricky to set up the remotes and everything. So radical also, of course, the nose in the peer
to peer network, they will discover each other and this is how you how you can step like you
can solve the problem of how you discover someone else's work basically. Okay, so I'm going to start
with what happens after a push. So suppose you're working on a radical repository, you have
implemented something nice, you made a commit, now you're pushing. Git push in this case is local
first, right? So what happens when you push code on radical is not that it connects to some server
and pushes the code there, but the operation for like in the first step is actually local, right?
So this is implemented using a mechanism that's standard called Git remote helpers. So we have
implemented a Git remote helper which just moves the objects that you're pushing so the commit
and the tree inside of it and recursively the trees and the blobs and everything. So basically
your changes moves them across only on the file system. So moves them to a specific
other Git repository that you have in your file system that maps one to one to your working copy
and from and this is this is very fast usually because it's just on the same file system and then
there is a long running process which we call the radical node which will pick up these changes
and the nouns that these changes and the changes to your references to the local network. Okay,
so you will never actually push to some other node. You just we just still kept this
nice mental model of pushing because this gives a clear separation of what is your working copy
like your checkout the one that you work on locally where you have maybe intermediate commits that
you're not willing to share with anyone that you're still cleaning up, you're revasing,
maybe you use JJ so you will do all kinds of crazy things and then you have a Git push step
which then maps to okay now I'm pushing my changes to the network but what's really happening
is you're just pushing it locally and then you get an announcement.
By default yes but you can disable that. So the question sorry the question was does
pushing wait for the changes to propagate over the network? By default yes there are some
settings the default settings are you can nominate preferred seats for which you want to receive
a confirmation that they have received your changes and you have a replication factor which by default
is 3 so you wait for three other nodes to confirm that they have received your changes and then
the push will be completed but you can configure it to not wait so the like all of this announcement
just goes on in the background which then means it's really fast yeah follow up question.
The question was what happens if the radical node is not running I will answer that on the next slide
okay so the radical node these clients which I briefly showed before we have CLI desktop we have a
tool that I did not show we have an HTTP demon HTTP demon which gives you basically an HTTP
bridge which is optional which I also didn't show but they interact with with radical via two
pathways so first is radical storage which is this other location where we keep a copy of the
HTTP repository with other with more namespaces and there is the radical node process right
and so the push that I actually mentioned before interacts with them both the push first pushes to
the radical storage so it updates radical storage and then it notifies the radical node that there
should be an announcement sent to the network and so the question was what happens when the radical
node is offline well then only the first part will happen so you will push to storage and then
the remote helper will realize that the node is offline and it will actually tell you you just
didn't offline push the reason is that your node is offline when the node comes online it will be
announced to the network and the way we can detect this is that we have a different connection
to the node as well so this is just a unique socket which is where you just exchange JSON messages
I mean you you don't do that yourself of course but the CLI will or and the
key to remote helper will do that and they will see okay there's no no radical node running on the
other side and then they will tell you you can start it using red node start and if you do that then
you you will get the announcement the question was whether radical node is a system white
process or user process you can do both the recommendation is for like an end use a machine like
you as a developer you would run it for yourself but we also have the setup of people that
just want to use a server or some other machine the Raspberry Pi that you have lying in your
cell or whatever to replicate over the network and for these people they would run it as a system
service the question is what is the mechanism for activating the service so we distribute
system deconfig files and we also have the possibility to just use red node start which will just
fork off the process and keep it in in the background but I think I can answer more detailed
questions after the talk if that's fine yes the system service can be socket activated so the
yeah and then this this appears to other nodes so the initial question is of course how do I find
how do I find an initial node to peer with currently we do bootstrapping via two nodes that we
run these nodes are also reachable via tour using onion services and we have like conceptually
we have figured out how to use DNSSD in the future but it's not yet implemented and we might
also combine this with a DHT in the future so for now things are working quite well but we understand
that yeah bootstrapping via some specific node might not be might not be as resilient and of course
I can also mention this later if there's a question but of course you can basically connect to
your own network okay so you don't have to use these bootstraps service you can you can use
yeah any any particular configuration okay so now I want to talk about the peer-to-peer connections
so what is actually this connection between two radical nodes it's a noise connection over TCP
and inside this TCP connection inside the like the noise wrapped TCP connection
we have a simple multiplexer the design is inspired by quick and they're basically two sub
protocols in there one is the git protocol which is just for syncing so it's really the git push and
fetch you know and then we have the gossip protocol which is to announce metadata which is the
things that I mentioned which git itself doesn't natively support and the thing can also be configured
to run over toy using socks out of the box so it's relatively easy to do that and to be honest like
this protocol is relatively simple I will now go into the details of the messages that are being
exchanged but this is not rocket science okay so I will just mention messages of two different kinds
I will omit informational messages they are not really important and keep alive messages so you're
left with three different kinds of announcements that your node will send and one subscription message
that your node will send so what can you announce you can first of all announce the node okay so the
node can say hello this is my this is my public key these are the addresses that I'm willing to
share with the network of course you don't have to share any addresses for connecting to your node
one of these addresses kind of course be onion service so this is all like making your node
connectable by others is like all opt in and can be done via Thor and the second message is
the you can announce the inventory of your node which basically is the list of repositories that you
have this must also somehow be known and finally per repository you can announce that a reference
has or the list of references has changed so when you push you would change I don't know say
ref's heads main in your name space and this is basically the content of the announcement note that
all the refs are signed using your own identity so this can be also relayed and when you actually
receive like when your when your node decides to fetch these changes from another node it can be
cryptographically verified that these are the commits that were originally signed by the user
who owns the specific reference namespace and yeah it's I mean this is not this is not very
complicated it's just three different layers of granularity of things that you want to announce
through the network and then on the other side you have subscription messages which are
currently just bloom filters over the repository ideas which also include a start and endpoints you
can you can basically say the validity of a subscription and also it's usually pretty simple
you would usually subscribe to the repositories that you have on your machine because you're interested
in getting new information about them okay before I close off I think I have around
two minutes left the network today I want to just show you okay I talk to you about this nice
ideas and all of these comes concepts how big is this thing we have one node which just measures
activity on the network and I just want to show you some numbers so the number of repositories
so here this is okay you cannot see the my pointer but just here in the middle middle of the screen
there's this number 25 this is the change of the year so to the left we have 2024 and then to the
right we have 2025 and just recently we switched over to 2026 of course and the number of repositories
is growing let's say roughly somewhat linear around 5,000 repositories with a bit of pickup
the number of nodes has been increasing a bit recently so the number of weekly active nodes
let's say is around maybe a bit more than 400 maybe it's 450 more maybe 500 let's see how many of
those stay active but it's not just 5 nodes okay so people are using this people are using this we also
have a permissive seed nodes which just will pick up any new repository and we'll start replicating it
the number of pushes is also picking up a bit so it's quite nice to see this metric that
people are just pushing more code the number of patches is also picking up a bit we have more than
30,000 patches on the network currently I will skip over this and in yeah the road ahead we know that
we still have like room to grow we can improve our peer-to-peer technology but the point is that
radical is in a place where it is all possible okay so we have not maneuvered ourselves in a
world where we now have to go back to servers or whatever we will make it much much better I cannot
go into to detail one thing is multi device support that we're working on and yeah feel free to
talk to me anytime I will be at the open local first after party today check us out online
and thanks very much for listening