Hello, thank you all for coming, thanks to the thanks to Yaran and the team for inviting
me back here a couple years later.
My name is Eli Malin, you can find me on the at protocol at iME.ly.
This is a continuation of a talk from 2024 called LifeBear Catalyst in the conspiracy to
solve video for everybody for ever, I got a lot of good feedback on that talk, people
seemed to think it was appealing, but I got told, I'm not really sure what you're selling
here.
So, I'm back two years later, I've started a new company, we've got a new product, it
is open source, it's cross-platform, it does a lot of very interesting things, so let's just
go through here.
This is a screenshot that I just happened to take right before we came in here, which is
funny that the growth stream is what popped up on the top of the corner there.
So this is live video for blue skies at protocol, I'm not going to go too deep into the
at-product of stuff today because this is a video talk, but it is an open protocol for decentralized
social, streamplace recently was integrated into blue sky here, so when anybody goes live
to the streamplace you get a little circle around your blue sky avatar there, let's
talk about what we've built.
So we've got very low latency live streaming using H264 and Opus, we've got chats and lots
of other sort of bells and whistles that you would expect from sort of a Twitch style streaming
platform.
The short explanation of this is if blue sky invented the at-product of all Twitter on
the at-product of all reasons, the same thing to make Twitch.
We've got syndication, so you can broadcast your live stream and broadcast anybody can
run a node and re-roadcast your node and syndicate it, as if we're all running our own
little TV stations.
We've got multi-streamings, you can push out to Twitch and YouTube, our Kival, scheduled
streams, web hooks, lots more stuff like that.
And what I'm going to be talking in the most about today is how all content on the streams
is cryptographically signed and has embedded user consent and metadata, so that's what I'm
going to get into most today is talking about the screenplay segmentation format and everything
that goes into that.
So what have we used to make this?
Lots of things.
I want to first shout out the G-streamer folks and anybody here that's worked on Mason.
I think it would be very impossible to do what we're doing without the especially the cross- compilation
features of Mason.
So we have, we have a nominally a go binary but also embeds a bunch of rust libraries for
C2PA signing and I wrote which I'll get into a little bit and then of course a bunch of G-streamer
on top using the go GST bindings.
And yeah, we want all of that to be statically linked Linux, macOS and Windows so we can distribute
it all as a single file binary.
And doing all of that without Mason would have taken me the two years since the last
talk, but thanks to Mason it's been a pretty good deal.
We have two forms of ingest right now.
We have RTNP over missed server and we have the Pion WebRTC library with Whip.
Playback is mostly over Pion WebRTC web.
We also have HLS output that's not very good.
I'm probably going to just like lean on missed server for all the other output formats.
And we have an embedded app so that's live right now on app.stream.plase but that's the same
code based running on Web, iOS and Android and it's bundled in the binary so it's really
supposed to be sort of one file, batteries included, everything you could want to do as
social streaming platform.
So this is sort of I think the answer to some of the questions I was asking in my talk
a couple of years ago.
So it is a decentralized live stream right what does that even mean when we're talking
about decentralized vibe or just sort of decentralized video in general you've got something
like bit torrent right or a lot of these sort of IPFS a lot of these sort of content
addressed systems where you could say okay I'm going to download this video segment or this
video file from somebody and I can be confident that I'm looking at the right thing
because there's a content hashtag to it right.
But live streams haven't happened yet so absent time travel we can't assign a content
ID to a live stream ahead of time.
So what do we do instead is have you generate a key pair, public key and a secret key.
The public key gets posted to your app code account, your blue sky, black sky is all
hosted, I run my own server at IMELite.com.
You can see at the bottom here this is the did key that corresponds to that public key.
And then there's a private key that gets right now put into your OBS stream key and then
you can you can send it into a stream place node.
It's a slightly primitive mechanism I wish there was like a more of a local signing option
but what we have right now with OBS there's sort of like static long lived keys.
So the the simplest way for us to accommodate that list to just put the private key into
the OBS stream key there.
So what happens next you send the live stream in to everybody else I say we slice it up
into one second segments you all know when I say that that I mean we slice it up on keyframe
intervals in each video segment gets into its own MP4 file with G streamer and then
we use the C2PA libraries to give all of these files and embedded signature using the
private key that we generated a second ago.
So this is for those that haven't sort of bumped into the C2PA format this is one piece
of the C2PA spec I really like right.
How do you I've worked with a lot of different sort of decentralized video applications
and a lot of them have sort of oh there's like their video and maybe there's some like
signatures over here and if you download both of those things you can like validate them
against each other.
The cool thing that the C2PA did for this is that it's an embedded signature so it's
basically an inline hash and a signature over all of the other parts of the MP4 file that
aren't part of that hash which is not trivial to do but the cool thing is then the interoperability
is awesome so literally when I say one second and before files you can if you run stream
place on your computer you can see the folder here shout out to the maximum I started
putting them into subject directories shout out to the maximum number of files that you
can have an XT4 file system that I bumped into right away.
So these are all streamplace segments they're all of an embedded signature you could
download them or get to that in a second but you could syndicate these all over the world
download them pass them back into streamplace and you could be confident that you're
looking at the right thing you're looking at the user's actual live stream through the
magic of cryptographic signing right.
We also do embedded segment metadata here so this is a streamplace segment I've pulled
down from me fight in Trabio and Silksong and you can see in here we've got a couple
of C2BA actions here so in this case I'm asserting both I created this live stream and
I published it and we have a standards compliant place.stream.medadata label here so for
example we've got a title associated with this we've got a creator so this is an assertion
that this is associated with my did PLC which is what the app protocol uses for for user
accounts in part and yeah all encapsulated within the np4 file right that was really important
to me that it's not out of band it's so easy to just like upload a file and forget to
include the other stuff so so it's all packaged up together nicely so I've been saying
C2PA so it's just in terms of things I know video people will be interested in how
we can untie I'm actually okay great so this is the allowed signature algorithms in the
C2PA specification it's got a lot of the greats a lot of the big ones in there but what
it doesn't have is the most common signing method that the app protocol uses which is
C2PA 6K1 signing which is to say they have this one here which is 256 bit ECDSA signing over
the SEC the the P256 curve but not the K256 curve because that's this is the this is the evil
math that's used in blockchains and stuff so we wouldn't want to have that one right so I don't
love that so I'm sort of talking up some of the C2PA formats that I really like but I don't like
that piece of C2PA so we sort of like we're not spec compliant with this this is the other thing
I don't like so Adobe's in the process of lobbying governments to adopt legislation that says
like all content all especially AI generated content has to be C2PA signed and not a terrible idea
in practice obviously like deepfakes are a big problem and we need to you know the idea of if
you're proactively generating AI content that you have to label it as such it's not like a
really advocating for it but that's not like a ridiculous regulation but this is Adobe's motivation
here for this right is that they can sell a lot of these all of a sudden there's a bunch of
European governments that have this new regulation passed and oh by the way here's like the one
piece of software that's compliant and you can actually do all of this so don't love that piece oh
also they want us to pay for sorts again which is really funny to me like years and years after
let's encrypt that we're gonna all start paying $107 a month for document signing certificates to
did you search which I don't think that's probably gonna happen so yeah what I can announce here
today we are in the process of standardizing a fork of the C2PA specification and collaboration
with the IPFS foundation specifically there the the dazzle project that has standardized a bunch of
C4 profiles that are used in the at protocol a bunch of other things so yeah that is we're just
in the process of kicking out that project it's been in the last couple weeks but if you're
interested in building sort of totally free alternative to some of these other structures I would
encourage you to check out to dazzle it dazzle that in there okay I need to help on a couple things
part of the reason I came here I'm on time yeah the first thing is I appeal to you all as engineers
imagine that you've made a video product and every time people use it the video looks really bad
and you have to go and chat and everybody's say oh well we use web RTC playback so you can't have
beef frames so you need to enable advanced settings and type beef frames equals zero into x to
six four options and then you have to do this over and over and over again for dozens of different
users biggest single problem that we have right now there's a solution to this which would be
get streamplace included in the OBS default list of drop-down that you can pick OBS from there and
then we can say have some hard-coded video settings in there and make it a lot easier for people
we've been turned down twice for not being popular enough but the policy says they can make
exceptions so I appeal to you all if anybody has any poll with the OBS team that could help us get it
on there oh this is such a painful one for us the other project that we'll be taking on as part of the
S2PA simple standard for provenance and authenticity is deterministic muxing so we've got all these
one second files they've got embedded signatures but having many many thousands of them on your computer
kind of stinks so what we'd like to do is combine a bunch of them into like say one hour and before
file which would be a much better archival format but in order to do that and still have the
cryptographic signatures the intact what we need is I said deterministic we don't do need deterministic
nuxing but we also need reversible muxing right that we can take in this one hour like this one hour
file split it back into one second files validate all of the signatures in those one second files
so you can get out exactly what you put in it's spent two weeks on it haven't quite cracked it yet
I think this is probably going to take rust and wasm but we're going to probably have some like
funding for people to work on this kind of thing through the IPFS foundation and some other folks
I'm going to try and get working on it so yeah if anybody's interested yeah if anybody's
interested you want to read more about it I've got some blog posts written on how this works at
box.stream.plase talking about becoming the video primitive for the next thousand years of social
video thanks very much
we have time for questions
I have tried medium medium tx oh yes so the question was why missed server for our
kmp ingest rather than uh nx rtmp or media mpx um a big requirement would be the static linking
so linking all of nginx into this probably possible but not something I was super excited about
I tried media mpx we might have gone with them they only recently this is just like a go technical
thing but media mpx until recently had all of their rtmp stuff in an internal sub directory so it
couldn't be included as a library that has changed relatively recently like in the last year
and I have played around with using the embedded media mpx server and just like hit a couple
problems with it so and we already have like a really nice working one with miss server so that's
that's a short answer but we do use other media mpx stuff internally and I'm interested in that project
hey how far back just the authentication code from this like how close to the cameras you get
from when you start signing it great question right right now this sort of the signing starts from
yes so the question is sort of uh this is a good C2PA question right which is how far
how far back does the attestation chain go right do we go all the way back to the original camera
right now OBS doesn't have any integration like that so most of our users that are using that
that's sort of where the the source of truth begins and where the signing begins
I you know as somebody working on all of the stuff I'm very very interested in the idea of
having compatibility with C2PA cameras or having like or taking multiple streamplace streams
and combining them in a way that keeps all of the the manifest and provenance intact so
I would love to have all of that stuff that's just a matter of like instrumentation on the software
that people usually use for these things
hey
yeah it's it's just a matter of when you get into it's it's nice for this syndication step I have
back here it's nice to have this sort of like well defined chunk uh sorry didn't repeat the
question the question was why one second chunks as opposed to like a continuous stream that you're
that you're rehashing you know one uh so like one advantage here as you get sort of very bit
torrent like properties where if I want to watch a stream I could be downloading it concurrently
from four different sources and for that reason it's nice to have sort of the self-contained
and well-defined chunk um I think we will probably especially as we get into like the
concatenation stuff that I was talking later combining videos I'm looking maybe at changing the
segmentation format because you can't just squish in before files together and then use Iro
and Blake 3 and then like you're done and you've got this like really nice sort of a format for
everything um so uh yeah more more to come on that but yeah it's like the bit torrent properties
like I was describing I guess what do you see the thing the issues of the next thousand years
that you know but I don't know it's gonna happen in the next thousand years but uh the question
is uh like what are the issues that are gonna come up in the next thousand years what I'm going
for here is like a thousand years from now how are you gonna know that somebody's twitch stream
or whatever right like it's just like there was like a video on a website and we like to
picture of it or something um as opposed to uh this is I think the the nice thing about content
addressing and signed primitives and things like that is you could legitimately look at this
I don't know a thousand years but hundreds of years from now and still be oh yeah that signature
is intact even if the private key hasn't survived and and we know we're looking at authentic content
I have uh yeah I think we're I think we're at time uh we'll do one more real quick
so if you were anyone who's there can't be where I would stream you can push to that
important right that's correct that's correct the the the the question was uh can anybody
with the R&P's key is uh R&P the this like private key that I talked about earlier can anybody push
to that uh the answer is actually yes right now but we have a open pull request working on that
to make it so um to separate out these two actions uh so that the created action and the published
action are actually signed by two different things so the creator if you get captured that one
key you get the creator key but that would be only valid for a certain publisher that would be
controlled on the server or whoever else is actually publishing the live stream and only with those
two things combined uh would you be able to actually stream somewhere so you know still the static
credential it's still like not ideal but that's another layer of security if you really really
care you should run stream place locally and use like BKS11 and do local signing of your segments
before you send them out that's like the really good way but this is what we need to do for OBS compatibility
right now cool thanks everybody
.