This is coming to an end, this is the problem, and then just another one, so some other
announcements for me. In case you don't have enough, you're going to get me a time-powered
and a bit more than a month's time, it will be a pilot, that's the reason why we're
keeping the matrix between the pilot, it's the first eight years, and I just got a check
a few days ago, there will be a matrix instead, so if you want to purchase a big
there and just get to touch with me or just give me, we will be there and try to open
something more matrix for related, and now the stage goes for Kevin, you will talk us through
Rob Briggs, is it Rob Briggs? Yeah, Rob Briggs. Yeah, give him a warm welcome and let's learn
about Rob Briggs as a future of last time, thank you all. Thanks very much, so yeah, I'll introduce
myself first since I'm relatively new to the community, I started using matrix only about a year
ago, and we've been working on the parent project of this for just about a year, so it's
everything to do with rust-apped across multiple platforms, so my background is in academia, so
I'm relatively new, in fact this is my first fossil, so I'm happy to be here. But yeah, thanks
thanks for sticking around to the afternoon, so what is Rob Briggs? Rob Briggs is a new matrix
we hope to make it a lot more than just matrix, but that's kind of the beginning of the story.
So these are some old screenshots I'm purposefully keeping the old ones here, so I'll reveal the new ones
coming up soon, but basically we have a single code base that runs pretty seamlessly across all the
major platforms, that's Lenin's Windows, Mac OS, Android, iOS, and Web, although it doesn't yet run
on Web because I didn't even know that the matrix has to be a supported Web, I thought it was still
broken, until I saw T-Most talk this afternoon, so I'm very excited about that, but yeah, that's kind
of the elevator pitch, so I do need to give some background information about what Rob Briggs is
and what the project overall is, so about a year, maybe a year and a quarter ago, we started
project Robius, which is kind of a large community endeavor to make multi-platform, active,
experience fully and rust happen, and to make it happen easily. So the idea is that developers
can do things like leverage the features of rust that everyone's familiar with and that we all like,
we don't have to do with multiple languages or different environments, it's just all rust all the
time, and you never have to write a single line of platform specific code, so yeah, come on.
And then the secondary goal here was to make mobile systems, iOS and Android namely kind of a
first class concern and a first class citizen within the Rust community, because historically,
it hasn't really been that targeted. So that's kind of a little bit of the parent project here.
Like matrix, Robius is fully open source, decentralized and community driven, we have a bunch of
different independent collaborators, some of who are funded by future ways, some of who are
funded independently, and some of who are just open source contributors at large across the whole
globe, and everything is written in Rust. So you may be asking if you're familiar with some of
the Rust community's long-standing GUI efforts, like why do we need Robius, right? There's already
a lot of great Rust UI toolkits, things like dioxys was already mentioned, there's things like
Igui and Iist, Laptos, Torian, and Makedhead, which we're using, and then the Lyme Banner crew has their own
framework called Zylam, and a bunch of others, packs is not on here, but the thing is there's a lot
more development and application, as many of you will know than just drawing a GUI interface, right?
So traditionally, a lot of Rust apps, like signal or something, have been more about the other
code, and less about the Rust code, where you have a lot of different platform specific or language
specific wrappers around a single core of Rust. So this is definitely not that, this is the
inverse, and it's almost also the inverse of what T-mo showed, which I'll get to later in
his talk about with assembly. So then why robrics? Why do we need another matrix client?
Well, I don't think that we've done this yesterday, do we need another matrix client, but we've
done it anyway. The original motivation behind this was that we needed some kind of key or killer
app to drive the development of all these different robius components, right? So all the platform
integration, build tooling, all that stuff that the GUI's don't typically handle in the Rust space.
So I was scratching my brains again, you know, what kind of thing is highly demanding,
specifically for UI and for platform abstractions, and we settled on a modern chat client, right?
So as some of the other presenters have said today, there's a lot of additional features that
chat has come to encompass over the last 10, 20 years. So it is really quite demanding of the platform.
So these are just, you know, a small list of some of the things that you need to support.
All right, and then you may be also wondering why do we pick matrix? Well, I think as I already
alluded to it, shares our views, shares our values very nicely, open source decentralized community
driven, and as a bonus, it seems like a lot of the matrix ecosystem is moving towards Rust or at least
focusing on it in a future-looking way. So we're also doing that. I'm very excited to see all the
Rust to use here because I've been a Rust contributor for many years now. So then, oh, sorry.
So then, once we have that, once we settled on writing a matrix client with thought, you know,
okay, I've written this client, it's pretty decent. There's still a long way to go, as you'll see,
we're not by no means complete. But looking forward, you know, what else could we do? I think we can go further,
right? Could we make a more unique client for power users or productivity sake? And by that,
I mean, something that I would like to see. Can we do more things with the application,
rather than just a matrix client, perhaps make it a central hub for a bunch of different
federated services? With focus on the needs for the open source developer community, like myself,
again, this is mostly about what I want. And then, you know, combine a bunch of different
other federated services and things that open source communities use and sort of a unique way.
So I'll talk about that later. The idea here is if we are able to create a very nice app,
we can make a stronger case for Rust app dev and then also a strong case for matrix being the core of all of this.
Okay, so just a little bit of boilerplate still. Let's talk about the goals of project rubius.
So our main goal is to create a reference design, sort of a full system stack here for all the
different components that you would need to create apps across multiple platforms purely and Rust.
Right? So the UI, just the orange part, that's kind of everything that we don't deal with.
I do, although I do contribute significantly to UI frameworks, like makepad and diaxis.
I would be, you know, everything else you see here, primarily in purple, but also in red.
So I don't need to get into this at all have time, but this is kind of what the current
architecture of the rubrics app looks like. It's the same sort of diagram just a bunch of
different details filled in. And basically, this is just a show that through developing rubrics,
we have allowed that to guide the development of project rubius components and also of the UI toolkits,
like makepad. We've had quite a lot of features there. So doing this has forced us to
dog food the components and kind of an interesting and barely digestible manner in some cases.
And we have made a lot of new contributions to, I think, the Rust UI space and the Rust app dev space in general.
So again, I don't really want to bore you with the details here, but this is kind of our road map
for the project. So we're just at the end of stage one here, which was basically to publish an app
with all the basic fundamental matrix client features that you would expect. So messaging login,
app resistance, and encryption. And then kind of while doing that, flush out the rest of the
ecosystem that we need to make this possible. In stage two, we plan to add a little bit more
functionality, perhaps integrating some local AI toolkits, AI tools, with an LLM runtime that was
one of our other flagship app projects under the Rubius organization. And then have some additional
things, which I'll show off in a demo shortly. So we hope to finish that thing by around the fall
of this here. And then the future looking stage three is the more of the combination of different
services here. And I'll talk about this a little bit if I have time. I'm not sure. I think I made
my presentation too long. So in stage one, so far we've pretty much finished all the basics here.
The only thing I didn't get a chance to do before this presentation was to implement editing message
support. But we have all the the major stuff that you would expect. So you can use it. And I
have been daily driving robots for about three or so months. Doesn't support a lot of stuff that
you would still expect. Like, you can't join a room yet. So that stuff doesn't support notifications,
which is my other, I think, big blocker. But in order to do that, we have to actually implement
all the platform notifications for all the different platforms. So we have published some pre-alvivations.
I just published a second pre-alpha. And I would like to emphasize pre a lot. So if you try this out,
which I wouldn't encourage you to, and you hit some bugs, know that we are fixing a lot of things,
mostly at the platform layer on a daily basis. And so this required us to kind of develop a lot of
the build tooling for releasing code signing on macOS and Apple, notarizing and packaging up all
these things into an app bundle, which is kind of annoying if you're not using the platform
native build tool chain and language, right? So to do this in Russ is actually a pretty big hurdle to
overcome. Thanks. And then we also spent a long time kind of establishing a nice little framework
within the app, which I want to then abstract away and allow other folks to use it to, for hitting
high performance metrics and efficiency metrics, despite the inherent asynchronous and the
inherent heavy concurrency of something like a client. And the key, which I'll particularly say is
to do nothing on the main third, and I really mean that. So if I have time, I can go into that.
But basically, we don't ever make a single matrix call on the main thread, and everything is done
purely on the back side, because on the back end, because there is absolutely no asynchronous
support on the UI main thread in Macpad, which is our UI toolkit. So yeah, just just to show another
thing, we do have complete platform support from the very beginning. So that was kind of the goal.
And then we're adding, as we add more features, we ensure that we are only doing it once we have
all of the platform support. So there's a slide later on that shows more detail about the platforms,
but this is kind of just an overview. And then these are the, this is our future tracker, a screen
shot that I took. I think a couple of days ago showing that, yes indeed, we have made pretty decent
progress, and we're ready for daily usage. So I mentioned a big part of Project Robias,
which has been driven by Roeberg's The Matrix client, is to develop all these platform
abstractions. I'm not going to go into all of these, but over the last roughly a year,
this is where we're at now. So greenest things that are largely complete, orange,
things that have begun, or are partially complete, but not really ready for production use.
So you can see something like camera support. There's a reason we don't support QR-based
login yet, because I don't have the ability to view a camera on all of the different platforms
in our appative ecosystem. So yeah, that's where we are. Okay, that was a lot of kind of boring
boilerplate stuff. Let's get to a demo. All right, so I'll start off with Android, because this thing
absolutely destroys my computer. The similar, I don't know why it's so blurry. Okay, I think that's
an issue with the, okay, it's fine on my screen. Oh yeah, okay, can't tab. Oh, we also don't support
emoji working on that. That's a UI toolkit issue. Did I miss type my path? Okay, now we're good.
Yeah, we are using the native simplified sliding sync, so I'm happy with that. Okay, this one is
good, so maybe it's just an issue with Android. Okay. Anyways, yeah, you know, you get everything
that you expect. This is iOS. Nothing to particularly, we're here. I didn't verify this session,
so we haven't some kind of errors here. Yeah, I mean, you get rich text formatting, which we
spent a while implementing. Didn't have that. You get, okay, that's not wanted to load.
Oh, there we go. Definitely pre-alpha. So on desktop, it's a lot, I think it's a lot, okay, thank you.
It's a lot more stable. So here is the macOS version. You can see that opening about maybe 200
milliseconds is incredibly fast. So for that, that's basically what we've earned by, you know,
doing our due diligence, writing everything from scratch and rust, making sure we don't have
any garbage collection, we don't have any weird memory safety errors, and we don't, we don't do
anything on the main thread. So one thing that always bothered me with a lot of clients, and this is
not to, you know, shit talk anyone here. But there's a lot of, like, I think, wasted white space.
Right, so my entire life runs on discord, unfortunately, because it seems like a lot of open source
communities use that. I used it for my open source projects as well. One thing that I never
liked about discord, and it's seemingly a lot of other clients is that you can't use the whole
width of your screen. So this screen is not very wide, but if you have 49 and ultra-wide screen,
that's not the case. So we're able to, we're able to have sort of a multi-pain
dockable tab review here, where you can basically emulate kind of what you'd get from an IDE,
and the idea is as we add more widgets. Oh, thanks. I'll take it. I'll take it. I will take it.
So as you add more widgets, and more features, they would continue to be dockable tabs,
just like this. So, yeah, I mean, here it is. You know, generally, okay. I've got some white space
issues there, but that's all right. Okay, thanks. And yeah, I would also like to highlight that
it. It's really quite fast. So I've been easily able to hit 144 FPS on my other screen that supports
that I don't think here we're going to get that because it's probably kept to 60, but hopefully
it's showing off that it's quite responsive. So again, this is due to the the massive amount of
work that we put into the UI toolkit and the platform of fractions underneath. So just to give
an example, I mean, we do have generally support for locations. So here I'll say, hi there.
I'm going to share my location. And let's see if it works. Okay. I don't know what that is, but we'll
find out. So hopefully you guys can see that. And then, you know, we're able to show all these
other nice little widgets here. I don't know. So yeah, if there are any other questions or any
other things you'd like to see, I'm happy to show this off later. You can also install it and try it
out. Not everything is perfect across every platform, but that's sort of just growing pains.
But that, yeah, that's kind of the, I don't know, two second, two minute demo of what's going on
here. So yeah, we have with these nice little table UI. Okay, I think that's good enough for the demo.
Let's go back to the talk. Okay. Great. So as you saw, we have overcome quite a few implementation
challenges to get to that point just to have things basically working. There were a lot of practical
like unexciting UI features that many GUI toolkits don't really focus on, you know, like being able
to show a context menu. Just two weeks ago, I had to re-implement all of our platform of
instruction code, so we could actually differentiate between different mouse buttons, right?
Didn't have that before. So there's a lot of practical, kind of business logic oriented stuff
that we've had to do. One of the things I also mentioned was Rich Text Formatting, so we took
myself and the developer of MakePad Rickerrins took a couple of weeks to write an HTML
parser and then have it render in the MakePad native widget format. So that's what we're able
to get super fast Rich Text Formatting, like hopefully you agree that you saw. Other things
that I've most have been working on are exposing and extracting all these platforms, specific APIs
and OS services, right? So you sell the location one, there's a bunch of others, I've showed
some of those for biometric authentication. So I do plan to shortly add a future for something
like, you know, Mark in a room is private, so you can hide a bit behind a fingerprint or something.
And the only way we can do that is by implementing all of these different platform APIs and
abstracting them. So the rest community I think is pretty good. There's a good amount of stuff in
the rest ecosystem that we've been able to reuse or contribute to. One example is like keychain
and secret storage, so you can take your session key that you get from the Matrix SDK and store it
and if you want to secure storage facility provided by the underlying OS and we need to make that
work across all the five platforms. I mentioned some of the build, multi-step build issues with
dealing with packaging and combining cargo and non-cargo build procedures. All of these things,
I think I'll skip some of the details. And then of course, I think like many rest developers
are main goal is high performance in the face of concurrency. So I alluded to this previously,
do nothing on the main UI thread and I really mean that. So using the Matrix SDK, there's a lot of
things that are async. We explicitly made the choice within MakePad and within our overall
development toolkit. We do support async and offer async APIs for all those platform abstraction
feature crates, but I don't use them generally because I don't think it's necessary and I think
it actually hurts a little bit more than it helps. So if you look at something like Flutter, you can
do asynchronous workloads on the main UI thread and I think they call them like micro-tests.
I forget the exact no-one clature. But if you're not careful, you can actually cause frame
stutters and other things. So we just made the conscious decision to ban that right up at the top.
So everything is routed through the sort of concurrency management framework here, which I don't
think I had a lot of time to go into. But effectively, there's three classes of MakePad
here. It's an concurrency context. One is the UI main thread, which has to do some things
and, fortunately, like Mac OS and some other Apple platforms and I think some others require
certain things to happen on the main thread, which is really annoying. But any time we call
an SDK function because most of them are async, but even the ones that aren't, I've noticed
in digging around in the implementation a lot of them do acquire internal locks, which is
something I do not want to allow in the main UI thread. So we always call out to some sort of
async runtime async thread pool in the background and make sure that we're not ever waiting
for those things on the UI thread. There's a lot of other details. We make copious usage of things
like thread local caches to do things in a lock-free manner that the UI thread can access and hopefully
you know, never be get bogged down or never have hiccups. We're working on moving literally
everything off the main thread, including things like image decoding. That's currently not there,
but it's still fast enough, but we can always go faster. And then we have a bunch of other
techniques that I don't really have time to get into. I only have five minutes. Okay, geez.
So in terms of platform support, this is what's out there currently. I'm excited to support web,
make pad also has native support for web, and most of the platform abstraction
grace that we've implemented also do work there, or you can integrate something like wazzy.
And then we also want to support a lesser known platform called open harmony, which is
sort of like the open source version of Huawei's harmony OS, which is widely used in China,
especially now that they've dropped support for Android on their new devices. So this could potentially
bring matrix to, you know, a billion new users or so. Not sure. Oh, sorry. So I might have to
power through here. So looking forward after, so that was stage one, right? We've we've hit a lot of
the kind of main fundamental features. Some things that I frequently miss or like the ability to,
you know, when I come back from vacation, I have hundreds of discord messages and one of my
things. And I'd love to know like what the hell happened without having to skim through all of them.
So there are some things that elements can help with and I can help with in a limited and
tasteful capacity. So I want to make sure that we we straddle that line and we're not, you know,
going and doing ridiculous stuff like forcing AI down everyone's throats. So okay, I'll skip all
this stuff. So we, where did the multi-paying dock will tab? Yeah. Yeah, so you saw that? Okay,
right. So I think one of the key tenets of matrix as we all know is in encryption and we want to
preserve that. So how can we bring in AI? You obviously don't want to send off your entire encrypted
room, message history up to open AI and upload it to chat, GPT, probably your fellow
room users wouldn't be a fan of that. So how can we do that without, you know, exposing our data?
So the key there is a local large language models, right? And this is definitely easier,
something done on desktop devices, desktop platforms rather than mobile, but there are some models
that are merging that can be run mobile. So we could do some cool features like this and we can
do, you know, maybe some other longer-term goals where we're using matrix once we have this to
share different agent recipes or chatbot demonstrations and help other people configure it through
matrix, all in sort of a seamless manner. But the idea is to leverage one of our other flagship
projects called motion, which is basically a local LLM runtime, if you're familiar with LLM studio,
it's an open source version of this. This is also the same exact thing that you just saw in
Robricks. It's fully entirely enrusted entirely from scratch using makepad and project robis.
So you can see you can have customizable chats with the different LLM models that you have installed
on your device. You can select them. So I think this would be relatively quite straightforward to
integrate into Robricks itself. So I don't have too much time to talk about this.
I also worked on this, you know, packaging it and getting all the platform features there.
Okay. But there's a lot of features that we can use. So work has actually already begun to
separate out all of these different components of motion and modularize them such that we could
just do something like this, where you're able to select a model that you've installed locally.
This will run using the Wasom Edge Wasom runtime, which has a wide variety of models already supported
and is getting updated every day. So deep seek is already available.