Without me further ado, I guess. I'll get started. I'm now in Skarnet. I am Devreau at
Blue Sky Social, and I'm here to talk about the app protocol. Thanks everyone for keeping
the energy high until 430 in the afternoon almost. If you go to Blue Sky Social, this
is our home page promoting Blue Sky the social network. Blue Sky is big world social. It's
a model you're probably familiar with and probably had opinions about. Blue Sky is the
company I work for, but is Blue Sky their main product. I don't know. See that app
protocol part? We're mostly going to talk about that. You could say app protocol is the
product. Will you at the end of this? Maybe. This is the landing page for our new
app protocol doc site. It's not quite live yet. It should be live this month, but I can
show a live version right here. We've got this night globe that we can rotate. It's very
neat. CTO did that. He loves it. If we scroll down, you can also see some really neat
new landing page features. You can see an app protocol lexicon, which we'll talk about in
the second. You can highlight some of the components of this app protocol lexicon, so for
example, strong typing, hyperlanking, with checksums, some very neat features. If you keep
going, you can also just stream some public posts at records from the fire hose right
here on our homepage. Pretty neat stuff. I am a docs person by trade generally. I've been
working on these new docs since I joined the company about three and a half months ago,
and I'm really excited. They're going to launch pretty soon. The app protocol is the most
important part of what we're doing. It's what distinguishes Blue Sky from other big world
networks and also from other default small world or default community forward decentralized
networks. Over time, you'll see more of our docs presence reflect the app protocol and
less Blue Sky. This will be the first big change. You'll see in that department probably coming
in just a couple of weeks. I'm going to use this not yet live version of our docs site. As
a way to structure my talk, so you can see some of our new features, what's coming soon,
what's exciting here. So take a look here. We'll start out showing you how to perform read
and write operations. This is actually very novel for the app protocol docs. Up to now,
we would have the Blue Sky docs show you how to make it get, how to make a post. But kind of
driving at the core app protocol concepts of what's a record, how do I get and post an
app protocol record? Why is that important? How does that fit into different app
protocol applications? That's a new thing for us. So we're trying to show not just higher level
SDK abstractions, but make it easier to understand our core primitives. Among other things,
this will actually improve LLM tooling for our docs. When Cloud goes and scrapes our docs,
and generates code samples, having this laid out more clearly, more human readable,
also makes it more machine readable. Good stuff. Makes it more generic, more reusable.
For those who are new to app protocol, one of the most important concepts is our PDFs,
personal data server. PDFs are where user accounts are actually hosted. They're actually an
individual SQLite repose per user and Merkel search trees, which is very, very neat.
If you ever read the fly IO blog, they're really good at talking about SQLite tooling,
all the cool things you can do with SQLite. You can host the PDFs on your own. The PDFs is
probably the easiest part of the app protocol stack to self host. It's the part that the most
people are excited about self hosting, because when you self host your PDFs, you own your own
data. You can host your PDFs wherever you want, whatever jurisdiction you want, under whatever
terms you want, and if you don't want to host the PDFs, if you just want to sign up your account
with BlueSky, you wind up on a BlueSky managed PDFs. BlueSky manages lots and lots of PDFs
behind the scene. Many other app protocol applications also run their own PDFs. The PDFs is a
very self-contained, very durable concept. We have our reference implementation that we ship in
TypeScript, that we have a Docker installer for, but there are also PDFs written in OCaml, Rust,
PHP, lots of PDFs is out there. PDFs can proxies some app protocol requests. For example,
around Login, and so a lot of the time you are making requests directly to a PDFs, we'll get into that
a little bit later, especially for private data. If you want a self host to PDFs, we've got our
deploy script right here on the PDFs repo, that is linked from our new docs, uses Docker by default.
It's got pretty low requirements. All it has to do is store your own local data, and that grows
relatively slowly as you'll see in just a bit. We also have brand new this deploy recipes repo.
I created this because when I was going through after I joined BlueSky and looking at all the
open issues and PRs on our PDFs deployment, a lot of them were like, hey, can you support
deploying this on Arch Linux? By the way, I use Arch sometimes, not this Mac, but I have used
Arch in the past. Pac-Man, great package manager. I love the artworks. Credit established.
But there's a lot of things that you can't really support upstream very effectively with a wide
PDFs distribution, and so for that, I wanted to create this repo for user-contributed PDFs
deployments and for deployments for other parts of our stack. Right away, we got a bunch of
interesting stuff merged in here. There is a script for deploying PDFs to your
synology mess, which is something that I might actually use myself. I thought that was quite cool.
There is a deployment of our relay, which I'll talk about later. That's part of the big world
indexing using Docker, and there's also a pop-man configuration, lots of cool stuff. That's
relatively new. Take a look at it. When you install a PDF, you get a health check endpoint,
you can hit your browser that just renders some text. That's really cool. That says at Prodo.
But it doesn't actually give you much of a browser-based management interface.
This is something we're looking at improving this year. We want to give you more ways of being
able to administer your own PDFs. That way, you can deep dive into it. Maybe some
profana dashboard, some additional features so you can really see. How am I scaling? If I'm offering
this service to my own community of users, if I'm offering my own local infrastructure,
for a lot of people who want their account on my server for some reason, whether it's
jurisdictional, some other reason. We want to give you more insight and more tools there. So that's
coming relatively soon. In the meantime, we've done a lot of work on our Goat command line tool.
Goat, by the way, is just Goat. I can't believe that wasn't taken at home, or ready. It's just
right. It's a good one. It's Goat. The Goat, coming soon from universal pictures. It's like a
basketball playing Goat movie. I saw it in the train here. Goat is the At Prodo command line tool.
It is very good right now for administering PDFs instances. For migrating your account across
PDFs is a first order act you can take within the At Prodo ecosystem. We're big on a credible exit.
And so if you ever have to migrate your account or interrogate what your PDFs is doing,
Goat is coming along. I highly recommend taking a look at that.
Now I'm going to move on to talking about some other aspects of At Prodo in this talk.
If you're familiar with At Prodo on Blue Sky, some of these will be a review, but we have
new work to share from all of them. So let's start with the FireHopes. FireHopes how you get data
from the network. We talk about this pretty often, but it still bears repeating since it's
unintuitive to other networks. All At Prodo call data on the FireHopes is openly available.
You do not need to log in to access it. It is just a web socket. This is really really cool
when in 2012 I worked for a company that had Twitter FireHopes access, which they
had to negotiate through very complicated data broker agreements. They had to have their own
complicated way of mirroring the data, being able to go back in time and query historical data.
I had to write Hadoop queries in the job I had at that time to be able to read the Twitter FireHopes
data. Who remembers Hadoop? Anybody? Yeah. Okay. So that used to be harder. And now it's not,
which is really nice. Anybody can just connect to a web socket. Just like on our homepage and get
data out of the At Prodo FireHopes. I showed it on the homepage just now. I can also show it in a
terminal here really quickly. So I'm just going to pop open a new tab and let's try it. Okay,
here's some syntax for getting out some FireHopes data. I'm going to pull it up here, grab it from the
docs really quickly. You can't see what I'm doing, but that's okay. I'm just going to bump
up on the bottom streaming data. All right. We use this right here and we're going to have some
data coming out just a second. Okay. Here's my terminal window. You took a second to Boots
for that. I'm sorry about that. We can hit this. And there are records. If, for example,
I wanted to build an application on top of the At Prodo FireHopes, although that would be
pretty much what I needed to do to do it. I'm just using web socat, by the way, which is a nice
command line client for getting web sockets, and you can see that same record structure coming through
there. Talk more about why all that is cool in a second. Backfilling is a concept that's
related to the FireHopes, but it's also fairly unique to At Prodo. If, for example, you are the
company I worked for in 2012 that was trying to offer historical Twitter FireHopes data,
you had to figure out some way of capturing that data at scale and then having an accessible
network and your own replicas that you could then use to query over time. That was very complicated
to do. If you were an academic doing sentiment analysis or some market research firm that wanted
access to that FireHopes, if you wanted to embed it in your Bloomberg terminal or whatever,
you needed to have some way to be listening to the FireHopes and backfilling that data all
along. You don't want to have to have real time requirements for keeping up with all the data
that was hard to do. With At Prodo, one of the core goals is anybody can always backfill a whole
copy of the network. That will always be one of the core credible exit principles. Anybody can just
get the entire corpus of At Prodo records with enough disk and time. Right now, I think if you
want the blue sky post, which are by far the most common type of record in the network,
it's probably about 30 terabytes worth of those, so that's quite a bit of a hot disk provision,
but for using any other kind of At Prodo record, any other kind of lexicon, it's way smaller.
You can do it right now. Backfilling and cutting over to streaming new data, until recently was
pretty tricky just because having to manage the cut-over between historical data and new data
coming off the web socket, that was not nice. We recently released this application called
PAP that makes that cut over much easier. PAP is now like a one-stop shop for any of that backfilling
you want to do. TAP itself is written and go. You can just clone down the repo and run it on your
own. We also have a TypeScript client and an interface to tap, so that way any time you have to
get any data from the fire hose, you can just do so this way. That's really useful for example
for creating custom feeds. Creating a custom feed of At Prodo data really just involves listening to
the jet stream listening to the fire hose, pulling down some subset of records that you want to
include in your feed, filtering and then rebroadcasting at a feed endpoint. TAP makes feed creation
much more straightforward. One of our earliest code patterns is this feed generator sample. We've
got commits here going back three years to back when At Prodo was brand brand new. I recently
rewrote this and let me see where the browser window I put as a feed generator sample. That's now like
200 lines along for the entire thing. Much, much, much, much more straightforward. We're always
trying to improve what you can do with At Prodo. Make our patterns cleaner, modernizer integrations,
and TAP is a big part of that right now. For those of you who want to be able to do a feed without,
you know, actually shipping your At Prodo feed generator, there are applications like Gray's
not social. Gray's not social's whole product is you can build no code custom feeds for a given
event, for a given media purpose, whatever. They got some really good coverage during the
New York mayoral election a few months ago because they had a custom feed. That was embedded in
media coverage. That was getting that data. Now, what if you don't want to have to backfill
records? What if you don't want to have to download all that content with TAP? That's a lot of trouble.
We don't want to have to do that all the time. Good news is a solution and it was written in
Rust. Yeah. There you go. I thought so. This is the microcosm tooling. Microcosm is a set of
open APIs that index all the backlinks in At Prodo call to enable very, very, very fast queries.
There are a lot of questions you can answer with backlinks. In fact, most of the look-up
most people want to do when they're building some kind of data analysis on top of At Prodo,
you can do that just with backlinks alone. If you go to the constellation landing page,
currently indexing 11 billion links that exist in At Prodo call records. And when I talk about
links, these are basically all the edges in that social graph can be conceived as a backlink of
one form or another. So for example, how many people like to post who are all the blue sky
followers of an identity? What are all the replies to a given submission? What are all the sources of
links to any given record? Well, when everything is a record, it makes it pretty easy to represent
all the edges as backlinks. And so you can very easily build on top of this to answer a lot of
questions. One of the best interfaces for browsing any kind of At Prodo data right now is an interface
called PDSLS. Everybody on the team uses PDSLS. We just love PDSLS. It makes visualizing everything
we do significantly easier. And the best part is we didn't build it. This is made possible entirely
by community developers building on top of the backlinks that you get from microcosm. So if I look
up my own identity right here, I can now click through on any given post I've made and I can just
see the raw protocol data. With all the different attributes, I can click through. I can even look
up the Alexa con. The schema for the post itself is also just another record here in the back
links. And this is all just being resolved instantly because it doesn't really have to do any
lookups. Very good stuff. It makes a lot of our lives easier and happier. Now we'll talk about
Alexa con's briefly. I keep mentioning Alexa con's in passing. When I talk about blue sky records
being just one type of Alexa con, Alexa con is a schema system used by the At Prodo call to define
different RPC methods and record types and provide interoperability. Applications like
loose guy need a way to describe their own behaviors and semantics. Right? At Prodo is not designed
for blue sky. Blue sky is merely one set of Alexa con's one application, one kind of records.
Blue sky uses some of its own Alexa con. So for example, at.beesky.feed.post is a lexicon.
Really cool thing about how all of our HTTP API methods work. We have what we call XRPC,
which is short for lexicon RPC, which is just HTPS. And the way that we design all of our endpoints
and as well as all of our permission sets in OAuth and all of our record formats is all the same
thing. You define it all through lexicon's. And so you can tell, for example, if you have a given
at record pipe, that's going to be the same permission you request. That's going to be the same
record structure. That's going to be the same endpoint. And so for example, you have an app that
uses app.beesky.feed.post, that's going to expect in any given app that implements it to be able
to hit an endpoint like post, you know, XRPC slash app.beesky.feed.post. And with your post contents.
A really neat thing about this as well as it means that the way that you get and post data
to and from at Prodo call applications is the exact same way that data is stored on the PDFs.
So your gets and posts just turn into gets and posts on the server. And that can be your
own server where you're hosting your own records. It's very transactional. It's very transparent.
Nothing goes into a database in a different format. Nothing is ever really serialized and
desirialized away from your own data. It's just the gets and posts to just stuck somewhere.
And we can build an entire application ecosystem on top of it. We have some new cool CLI tooling
for working with lexicon. You have brand new TypeScript SDK called Lex.
When you're bootstrapping a new project, you can use Lex install to fetch
lexicon as you want from the lexicon repository. Then use Lex build to generate TypeScript
types for them. And that is how you get first order SDK methods for any generic lexicon. So that means,
again, blue sky is not special. If you want to have nice SDK primitives that call a method on
a given lexicon, write the lexicon yourself, compile it, push it up. There you go.
On the go side of the shop, we also have goat. You can use goat to went your lexicon's
and to publish any new lexicon's you've just written to your own PDFs from which any
world can install and build them. We've also got a big long lexicon style guide for anybody who
wants lots and lots of recommendations for how to format their new lexicon's for their apps.
And other developers are building their own interfaces for browsing the ecosystem of lexicon's.
So you can see, again, all those different parameters in a nice web,
viewable interface. There are lots of non-blue sky lexicon's being used in the ecosystem.
I really like leaflet.pub, which is just a long-form blogging site that is built on a separate
app. Prodote, uh, lexicon. I like this because I think that micro blogging and regular blogging
are not inherently that far apart. I like the idea of integrating micro blogs as comments to a
longer form blog and being able to store these records side by side is I think really cool.
If you think about what I've already said about uh, backlinks and about the ecosystem of
that protocol records, you can see how we can get it something that's kind of RSSC. Maybe once again,
we're going to bring back RSS. The web's back. Yeah. Yeah. Yeah. I do a lot of regular writing about
my uh, devra work on a leaflet right now. So for anybody who wants to follow that for new developments
in that protocol, they'll be a link at the end. There's also, thanks to the power of lexicon's,
some really cool third party clients that are maybe technically not blue sky clients. They're kind
of in a secret third category where they're not entirely new app like a leaflet, but rather they're
kind of like a game that makes browsing social media a little healthier because it gives you a
stamina level and you deplete your stamina by reading too many posts, which is how a lot of
people feel about social media in real life. Uh, and and deep thing about that is if we go back
to looking at PDSLS and we look at, you know, all the different lexicon's that I have interacted with
because I've used different app photo applications that have written to my PDS. You can see,
okay, here's all the blue sky lexicon's here are the core records for my posts, my likes, my
followers, whatever, here are my leaflet publications like I mentioned, those blog posts, you know,
there's a whole blog post in code of the do-app photo record, but for Anosota that Mothi app,
that's got its whole separate set of lexicon's. You can use an existing set of lexicon's and
defines a new one to make your application for a pick and mix. You can create new applications
that build on the strength of that existing social graph. And that's going to bring me to my conclusion
of this talk. Once I fix my window management, there we go. All right. There have been some pretty
cool blog posts written about this ecosystem, talking about just exploring the app protocol.
When I think about the social graph, when I think about those 11 billion edges that currently
exist in the network, a lot of those are just building on this get follows relationship,
which again, is pretty germane to how we've always thought about social media. It's big world
social, right? You can use that existing follow or graph for anything you want, whether it's shared
interests, communities, local, big scale, whatever. Any application you build on that protocol is
already bootstrap with all those follow relationships. You can add any other application features,
any record types you want, but that's a really good starting place to expand. There's been
some really cool writing done by community members, former Blue Sky Team member Danny Abramov,
this cool blog post recently called a social file system, which at the end of the blog post,
he uses a fuse file system plugin to mount his PDFs as a local disk, and then he just has a
bunch of JSON records that he's populated on his own file system, which his local clawed CLI can go,
and then just re-all his posts and do some meta analysis from that. There's a lot you can do
when your posts are just records and the whole thing is wide open. I think it's really cool.
Finally, for the activity Pub developers who might want us here, I wanted to just briefly talk
about the cool word Bridgy Fett is doing. I think we've already talked about that in this from
this afternoon, Bridgy Fett exists, so we have some compatibility between activity Pub and
at-product tooling. If you take a look at our numbers right now of the weekly users that are
creating at-product records that are posting from non-Blue Sky PDFs, as you can see in the bottom here,
green would be self-hosted PDFs, so you're running your own app, probably, LPDS. Everything that's
not green on this chart is pretty much coming from the activity Pub ecosystem, and you can see
we've got some pretty respectable numbers here. That's touching like 20,000 users a week or creating
records from non-Blue Sky PDFs with their self-hosting art tooling or using Bridgy Fett. If you look
about as a proportion of all weekly posting users, which is about 1.5 million right now, like
give or take, it's around there. There's an internal debate that it's actually more like double this,
if you have a better window to a survey, but we'll say that it's 20,000 out of 100, 1.5 million.
That's about 1%, it's a little more than 1% of people who are not just using a Blue Sky
PDFs to post, and that's like desktop Linux numbers. I feel good about that. That's a sustainable
community. You can do a lot with a 1% user base of a very popular application model. I think that's
going to go up. I think that's getting to sustainability. Why build on app? No need to architect your
own data models, build on the existing lexicon supplement your own, and you save yourself having
to make a lot of API endpoint implementation decisions because lexicon's are used for permissioning
and for API endpoints and everything in between. Our code generation provides first-class support
for all lexicon, so you can leverage that ecosystem's social graph. All records are basically just
debts are posted exactly as they look like, and so it's really straightforward from a developer
point of view. We've got a very engaged dev community lot of people building an app right now.
You can self-host as much as little of a stack as you want. I barely in this talk talked about
self-hosting non-PDS components. There's a lot more we can talk about there. If you want a use case
that's low code, no code, or what I'm calling lots of code, like a little pizza chef, like lots of
code. You can self-hear on use case, and protocol interoperability with big world scale. That is
the pitch. Thank you very much. You can find me at alics.bysguide.team. I'm also on the Fediverse
at exfielix-digit press club, from a past life. We have a conference coming up at the end of March
in Vancouver, the 2020-6 atmosphere conference. Lots of exciting talks. Sick is still available.
2020-6 will be the atmosphere. I'm sorry.
There have gone time for a few questions. Yeah, sounds good.
So ATs, the standard, is being discussed at the ITF. There was a working group session at the
meeting in Montreal, back in November. That's a really interesting process because we've really
scoped down the aspects of AT that we believe are like ITF level and not application level. So we're
trying to make sure that it makes sense what we're giving them to turn into a standard. Really
interesting ongoing discussion. You can check it out on the mailing list or we can talk about it
afterwards. Yes. Some of these are questions to check and go. But you mentioned that
McKinney is really important. Can you tell us how it can go? Do you go down? Do you have
data changes because of things in no corruption? There's a lot that goes into that in terms of how
our relay works to help you fetch different PDSs. Our relay handles love the indexing across
PDSs. I would love to talk to you more after this session. We're going to be here 15 more minutes.
I'll be here afterwards. Thank you. Yes, you're welcome. Yes. Private data win. Yeah. My colleague
Matt too in the corner doesn't want me to call it. Yeah, give us a few months to a year. Yeah,
it'll be this year. Yes, yes. Yeah, so right now, even though things are quite open in this time,
yeah. The only one on the first point is provide a lot on the other side of the company to steal
the new one or the biggest relay if you're not