Hey everyone, I'm Benui and I'm really excited to share this project with you all as to see
such a tech room.
The inspiration for the project is tools like ether pads or hedge doc or crypt pads, which
I really love to use for taking notes during a meeting or to write a draft of a blog post
with another person together, but one thing that back me about this is that you cannot write
code with it together, right?
It's only for text and also if you need an internet connection, so once you're offline
you cannot access this information anymore.
There are tools to record one like that, for example several text editors have plugins.
This is a plugin called Life Share for VS Code, which allow you to collaboratively edit
code together, like files that are on your local machine, but this is a proprietary plugin,
and like the protocol is not open, so your friends who use new them as something
cannot really join, which sucks.
So obviously we need something to add to them, and that's why I started the ether sync project,
which is an editor-economic, open source tool and protocol for editing local text files together.
I'm not doing this alone, but I did it with a friend, so on and on, we like most of the
stuff we will show you last year together, with funding from an internet, and yeah, I only have
10 minutes, but I will try to show you as much as possible about it, and here's what we have
and how it works.
The idea is based around like connecting their directory is project directory, so to make
a connection with another person, you would like go in your directory of code or something,
and I say ether sync demon, which starts a demon and then shows to the connecting command,
others can use to connect with you, so the other person would want ether sync demon and provide
a peer, like in their project directory, and after that these two directories are now
magically synchronized and like files that appear in one directory also appear in the other one,
and if you start opening a text file, you can start real-time collaboration in it.
I have a little video that you see at this work, so you can see on the left side there is
a new instance running and editing a text file, and on the right you have a VS code, and yeah,
you can edit this content together over the network, and also you can see other people's
characters in this text, right, and this works like both with VS codes, multi-curse feature,
for example, and also with newvins, block-curse, this will be transmitted to the other connected people.
Right now we have plugins for newvins and VS code, this is what the VS code plugin looks like in
this extension store, for example, and more plugins are coming up, I will show you later.
The demon synchronizes entire project, so it's not only for a single file, but it's for like
the entire directory and it's up the directory, which makes a lot of sense if you're editing code.
You can not only use like text editor first, but you can also use external tools.
Sometimes you have cases where like it's part of your workflow to, I don't know, use the
cargo command to change your cargo terminal file or something, and then it would be inconvenient
if you had to do this in your editor without this external tool, so here's a demo of someone
having a file open in newvim at the left side, and at the right, and you have like this terminal session
where the person starts like editing this file.
Let's see.
Yeah, so here we are appending something from the command line to the file, and over the network,
this content changes also picked up and edited into enter-near them.
Also, if you saw the content for some reason, yeah, it's considered over, which is like
very convenient for for ourselves developing this, and the tool now I dropped this keyword is
local first, so you always have full access to the content even if you go offline, right?
This is like important if you drive through a terminal and a train or something, and then you want
to keep coding on it, and then we synchronize later once you can reconnect to your peer.
So now you might be wondering are there using CRDTs? Yes, we are, of course, this is like
if you have never heard of it, I don't have time to like go into how it works.
It's like an algorithm called conflict-free replicated data types, which is very convenient for like
making like having a situation where both parties can edit, and then once they connect it,
it kind of automatically fits together again, but check out other talks if you are curious
how that works exactly. And yeah, we're doing peer-to-peer connections, so you don't have to trust
the sentable server. How does it work? Let's look a little bit into that.
We wrote the demon in Rust, and the plugins in their respective scriptmen languages,
like Lua and Tab script, and the components fit together like this. So each peer has a running
demon, which are connected by the internet, and then like make this synchronization of the content,
and you are editors connect to the demon on your local machine using a plugin.
Let's look at what those arrows mean in more detail. So demon to demon,
we're using a library called LipPear to peer right now, which is like custom build for a
situated like this, very flexible, and we're using their private network feature to encrypt
the content that's going through it. And inside of this connection, we are speaking there
are automatic, automatic sync protocol, automatic as the CDT library we're using.
And demon to editor, we have this design goal to make it as easy as we can to write new editing
plugins. So we kept the protocol really small. Basically, the editors have to spawn an
interesting client subcommend, and then talk Jason RPC to it. And yeah, we have a small syncing protocol
that's kind of inspired by LSP, the language server protocol, because we thought editors
support that most of the time anyway. So offering our protocol in Jason RPC format might be
like practical to implement. And for this protocol, because the editor is a separate process,
we also have to do something clever, and we're using operational transform for that, which is
like another approach to make real-time collaboration. And we're doing it in a one-sided way,
where the editor doesn't really have to do much, it can just reject the changes
is getting, which it doesn't apply to its own content right now, and the demon will do most of
the work and send the content data. Our demon editor protocol is so tiny that we want to
fit it into a two-t, so here it is, and like, well, not 250 characters or something. And
this is like the core of how the content synchronization works, the experiment, how we can
express that. But we also have documentation, which goes into more detail. It's not really a
replacement for Git, I would say, it's more like a tool that works together with it.
It's not really a system for which text editing right now, it's most independent. And also,
it's not production, where the software I would say it has some rough edges, for example,
one of our open questions is, which makes it a bit inconvenient to use, how to make connections
between two local networks. Right now, what we do is, one person opens a part in their router,
which is not very convenient. But yeah, we would like the better solution for that. And if you
have suggestions on how to do it, let us know. Lipit repair has this private network feature for
encryption, and Lipit repair also has relays, but I don't think the two fit together in a good way,
it's not really intended for that. And yeah, our green would be, like, to have a magic rumhol
style connection making, where one person gets three English words, and then tells that to the other
party, and then the other person can connect. And three ways to get involved. You could write your own
editor plugin. And I certainly have two official plugins for new them as VS Code. And some other
contributors are working on once for IntelliJ that one is quite far and also for E-mix. I think the
PRs here, but you could write other ones for a clip, for example, if you're using that.
Some kind of web-based editor would be really nice to have. And if you were like, really having
want a good time, you could, I don't know, fork nanos, something, and then like heck,
it's a thing supported into it if you wanted. And second thing is to help us improve the syncing protocol.
Maybe if someone has like a lot of experience with LSP for example, you might have suggestions on
how to make it more general to be able to use it with other synchronization tools, maybe.
And yeah, what we would really like to do is to have maybe a small group of people who work on
refining and sanitizing it. And yeah, in the end, it would be nice to have this protocol be
able to like use it with other software as well. For example, we heard about the API for crypto pad
and being able to use it with crypto pad would be quite nice. And yeah, the third thing is just to
use it in crypto feedback. If you find something that's annoying or broken, let us know anything
you're missing. And yeah, as the very last thing, what I would really like you to take away from
this talk is not that you necessarily have to use our tool or something, but I want you to be
unsatisfied with like the status quo. And I, like the next time you have to pair program using a
screen share or something, maybe you are a bit annoyed and have like desire for a solution like this.
And that's all I got for you. Here are some links for all of the documentation.
Thank you very much. And fortunately, we don't have time for questions. So don't hesitate to
talk about it in the whole way. And we'll prepare for the next talk.