Okay, we're ready to start the next talk, Robert, you have the floor.
Thank you.
I'd like to talk about the level of energy,
and we'll see what's in this acting.
What's it?
No?
I can't speak up, let's listen.
So, yeah.
I'm going to talk about,
I just think, we've seen a couple of examples they already
in the room.
So, I'm going to talk about the media weekend,
why am I?
Thank you so much.
Okay, I think that would be better.
Great.
But first of all, I want to introduce myself,
and the software that I'm going to talk about.
So, my name is Robert Fogel.
I'm a software developer at the Halibald company,
since 15 years now,
and we develop a media week heat distribution.
So, that is, I think,
very different to many other open source projects
that we have seen today.
So, we're not developing the whole set of code
in this application,
but basically, we're distributors of an existing software,
and the media week is software,
and we extend this software,
we add a different functionality,
we adapt it to use cases,
especially from the business world.
And this is what I'm going to talk about today,
that is the media weeky portion,
basically.
So, as I said, we're just building up on that.
We're bundling it, and distributing it.
And media weeky is a very famous project,
I believe most of you probably heard of it.
It empowers the Wikipedia project,
and is also widely used in third-party,
yeah, in wyamands, in organizational,
organizations of different types.
And obviously, the most important part of
a weeky is the wiki content,
and the wiki page editing.
And as many other wiki,
media wiki also uses special,
special word version of content,
or special type of content that is so called wiki text.
Wiki text is a plain text mark language,
just like mark down, for example,
or other wiki text, yeah,
variance that you might know.
And now there's many people on that much
into using plain text, wiki text anymore.
So, the demand for wiki editing arose,
many years ago already,
and in that context,
the wiki media community,
the wiki media foundation,
decided to implement visual editor.
And unlike many things that we've seen today,
they did not decide to use
an existing solution,
like CK editor or whereas others,
that are around,
but to implement their own solution.
And this solution is called visual editor.
Now, visual editor comes,
it's a little bit,
it's ambitious.
So, there are two types of visual editor,
basically, if you read about it,
in a context of wiki media or media wiki.
First of all,
visual editor is just a regular HTML editor,
as you know it from other implementations.
Runs completely in JavaScript in the browser,
just as all the other ones.
But then, on the other hand,
there is the media wiki visual editor extension,
which is nowadays a core extension,
so it's delivered together with the media wiki core.
And it basically builds up on the standalone version of wiki editor,
and as a little editor standalone,
it definitely can only edit HTML.
There needs to be some code that transfers wiki text
from plain text wiki text into HTML,
and vice versa.
And the other way around,
so we can properly store it in wiki page again.
And for that,
there's a service that is called parsoid,
which I will not go into detail here,
because it's not really important for the use case of,
yeah, of simultaneously editing,
but that's the basics here.
So, media wiki itself needs wiki text for storing,
for comparing, for working with,
but visual editor needs HTML for,
yeah, for modifications for the modifications to that content.
And when they started with the visual editor implementation,
many years ago,
they very quickly,
the media foundation very quickly realized that there's a chance
of implementing a real-time collaboration editing functionality,
and that is due to the way visual editor handles,
the handles changes,
basically handles the document and handles changes,
I will go into more detail here.
That was back in 2011 as you can see.
It was also the time where Google Docs already was a thing,
and Etherpad came up, so, yeah.
They put in some effort into that,
and they actually developed a back-end component,
the server component, and a front-end integration,
but it was never went into production,
on any wiki media project on any wiki,
it was just for testing purposes, basically.
The main reasons for that were that,
they were concerns from the community,
from the wiki PD community,
especially regarding contributor attribution.
So, basically, if they are multiple people,
editing a wiki page at once, and you save that,
the wiki can only store one revision of the change,
and two should then be attributed as the author of that change, for example,
and that was subject to discussions in the community,
and in the end,
there was a second approach, as you can see here around 2017,
to restart this, but in the end,
they basically abandoned this code,
and it set there for some time.
And yeah, then we came around that,
and we did some analog, some tests with that code,
and we decided that it was really worth looking into that,
and adopting that code,
and we did.
So, what are the things that are required for setting up
such a kind of collaborative editing in the wiki context?
This is the set back and part,
it's called rebaser,
and this context.
Basically, it consists of two parts, a server application,
which was originally written in Node.js,
so that JavaScript application,
and the client integration,
which still lives in the wiki editor code,
unchanged, since basically 2019.
And the second part that is really important for that,
is this so-called linear model or linear data model,
and it turns out to transaction.
Because this is what actually enables this kind of functionality,
the way the visual editor represents the data,
the HTML data, and the way it interacts with it using transactions.
And this linear data model,
I want to spend a little bit of time on it,
so what does this mean?
What does linear model mean in this context?
So, this is a regular HTML,
everybody recognizes that it's very simple.
Actually, it is a wiki text,
it's, I have mixed this example,
it's a regular HTML, the span element,
but you can also see those similar quotes
surrounding the words hello and what,
which is, in this case,
wiki text.
And now let's see how the visual editor itself represents
this kind of content.
I have drunkered a lot of things,
because otherwise it would be not very readable,
but what I want to show here is that it basically splits up
the whole content of a page in a linear way,
basically character by character,
or section by section,
it will add,
it will split this up,
and it will annotate those things,
which we can see very easily here,
though, so the second pair of brackets here,
that is the annotation,
which basically tells what kind of formatting
is applied to this particular part of the content.
Here on the other side,
I've shown the various things like bold span
and the telek, this is what it wants,
that we saw in the previous slide.
So,
and the next part of this,
of this process is now the transaction,
so every time somebody changes something,
somebody selects something,
a transaction is created,
and the change is first applied to the linear model,
and then the linear model will be synchronized
with the view in the HTML,
basically what the editor shows to the user, to the end user.
Transactions,
obviously, like, basically, many,
most transactions are atomic,
and they can be undone, basically.
Here you can see this is an example for deleting,
sorry, for inserting some character in this case,
I believe.
Next to the actual transaction,
there's a lot of other things that go with the change,
that is,
there are stores and selection information,
which I did not put in here,
because it's not that important for this example.
It's just so you know that,
for sharing, basically,
which part of the document are selected,
or what the cursor is,
you also need to have this inside of a change object.
Also, there is the stores,
the stores are very important for anything that is,
let's say,
not part of the actual document,
but some external resource.
For example,
the,
for example, templates,
templates the way,
and in a way,
to include content from other pages,
these are indoor stores,
and they need to also be transferred,
at least the information about it needs to be transferred,
in a proper way.
So, as I said,
there is this rebase backend,
there's this rebase client,
and the rebase client runs inside of the wheel editor,
extension,
and to,
yeah, make things work,
basically, on every key,
the first of all,
we established a web-socket connection
to the backend,
and then on every keystroke,
the change,
including the transactions,
is transferred to the server,
and here,
some magic needs to happen,
because we cannot just send out the same change
to all the other documents,
but you need to rebase it,
basically,
because in the meantime,
due to concurrency,
things may have changed.
So,
it's not just a distribution mechanism,
where one change arrives at the server,
and then it's being sent out to the client,
but there's also some rebasing being done on the server side,
and so that the change arrives
in a proper,
in a proper place,
basically,
of the document.
Also, the server stores a whole history of the document,
so whenever a new client joins the session,
this entire document can be loaded
for directly from the rebase of server.
Okay,
and yeah,
that's time,
not some for a little demo,
it's not super exciting,
I believe,
but it should still want to show it to you.
So,
let's see,
I've hit two browsers open with two different users,
logged in,
and currently we have,
as well,
yeah,
as this feature is pretty new to a blue spot,
about half a year,
maybe in production.
We still have a dedicated button for edit together,
we call it that way,
it's a nice sort of term that collaborative editing,
a real-time editing,
add it connected together,
which basically starts this change,
and I can now,
in white people,
like, for example,
this other user here,
and yeah,
in this case,
we get a notification about entering the,
entering the,
a collaborative,
collaborative editing mode,
and here we go.
So,
we had to do,
with a couple of design decisions here,
which were different from the original implementation.
The original implementation,
for example,
was not even integrated into a regular wiki page,
as we see here,
as a mode of editing,
but it was a standalone page on the wiki,
where people could just enter very similar to what you know from,
either path for example,
then people could,
could type something,
could collaborate on that,
and then they could export it and save it to some,
some page.
We have integrated it as a,
as a full,
flat,
and well it,
edit mode,
can save that.
If I'm saving a page,
I can't,
I'm not,
not necessarily leaving the page.
There will be a revision created in the background,
but it doesn't mean that I need to leave the,
to leave the edit mode.
I can even leave,
collaborative editing sessions,
so I don't,
there's no need to save it.
Obviously,
it's only going to be persisted into the wiki's revision,
store once people's safe,
but you can use that as a draft mechanism as well.
So let's see if I just,
if I just leave that now,
okay,
the session is ended,
and now we should see a,
in the history,
which you see both the users,
as contributors here.
Again,
this is,
this is the way we implemented it in blue spies,
in the order of the media wiki implementation.
This was very different,
and let's go back to,
the slides.
Uh,
I've heard,
but before we come to the,
to the future plans that we have,
quick,
quick information for you.
If you're interested in that,
so we have published this as a,
as a standalone media wiki extension.
It's not in any way bound to blue spies as a distribution,
but it is included in blue spies.
And,
there,
we are,
doing regular releases on that.
Using regular pet level releases,
we are bound to the long term support,
uh,
version of media wiki,
that's a,
specialty of media wiki,
actually the way they release.
Um,
and also to,
to ease the setup of this whole thing.
Um,
we provide a,
a localized,
a container image,
basically,
for,
um,
that makes it easy to,
to start up,
um,
and operate the rebase of back end.
So yeah,
setup is,
fairly easy,
actually.
Um,
let's move on to,
uh,
what you plan in the future.
Um,
I somehow,
set this already currently,
it is,
um,
an optional way of editing wiki pages,
that you need to,
uh,
pick from a list of other options.
Um,
we want to make this the,
um,
the main way of editing stuff,
uh,
editing pages,
and,
um,
obviously there are still some,
some things that need to be optimized,
um,
even after,
um,
a lot of things that,
that can be added for managing,
for example,
um,
about orphan sessions,
um,
and especially,
um,
the backup restore,
um,
are things that,
at the moment,
back and be done,
um,
but it's,
um,
more difficult than it should be.
And,
especially,
this is especially important because,
as I said,
as you can leave the session at any time,
and you can use this mechanism for,
uh,
drafting,
um,
wiki pages.
Um,
those session,
uh,
just those collaborative session data.
Um,
like,
properly,
and we need to make sure that they,
they're not, um,
getting lost in any way.
Yeah,
and I think that's it.
Um,
um,
I have,
on this slide,
on the last page,
I have also some resources that you can look up.
Um,
I will share this slide,
later on,
on the website.
And, yeah,
thank you very much.
Thank you.
Okay.
So the question was,
um,
whether it's possible to see,
precisely which changes a user,
uh,
applied during the session,
after saving the page.
No, that is not possible and it is also not desirable because the WikiTex is a plain text
markup language that if you marked up everything that the user typed which is theoretically
possible, you would pretty much mess up this WikiTex which makes comparing the WikiTex much
difficult in the long run so we don't do that. We store who was part of that session
but we do not store who actually contributed with the part because to be honest it doesn't
really tell much so people might just fix typos or something like that and this is really not
so useful information I believe.
So the question was how do we deal with selecting changes basically?
Yeah.
Okay so how do we deal with the position of the character in the document during the edit?
To be honest I cannot really answer this question at the moment I would need to look it up.
I think it is part of the selections field but I'm not entirely sure how this is handled
in the rebase logic kind of tell.
By the way this is a very nice example of why open source is so important so this was a longer
abandoned code since years it was not looked after anymore not maintained but as it was
it was open source it was easy for us to pick up to learn from that and to bring it to life again which is great thing.
Thank you very much.