I want to start today with a story, and we're going to hear lots of stories today, but I like
signal.
I use signal, maybe some of you, also use signal.
And you might have noticed during the latest AWS East Outage, that's your signal stop
working.
Did anybody have similar experience?
Yeah, and I live in Berlin, and I was messaging other people in Berlin, and I found
this quite peculiar, of course, later Meredith Whitaker was asked, what happened, and it
was like, well, there's no other choice, all right, there's no other choice.
Okay, at least she also said that's a problem, right, there's no other choice.
But what I'm going to ask us today repeatedly is, is it a story or is it a fact?
I'm a scientist, I come from machine learning science, data science, and what I want
us to keep asking around this sovereignty debate is, is it a story or is it a fact?
And if we think it's a fact, I want us to ask, why do we believe that fact?
I want us to assemble evidence, and I want to test it, like we would everything else
with the scientific method.
Okay, so who's with me, wants to talk about stories or facts?
Yeah?
Okay.
All right, I'm going to tell you some other stories that are going around.
This is the BSE, so if you live in Germany or familiar with the BSI, who does a lot of things
around type of our infrastructure, security, and the policies around it, and they signed
a deal with AWS because AWS is so advanced in cloud security that they're the only
ones that the German government can learn from, right, is true, right, must be true, right?
And this continues in a lot of the marketing that you look at around this, that says
that sovereign clouds might be good for your legal department, but they lag behind in things
like AI, it's just impossible.
And this tends to be the story around U.S. technology companies with this idea that somehow
there's something very special about U.S. technology companies or U.S. AI companies that means
that nobody else in the entire world can understand how to do this.
Okay?
This is a story.
Then there's another story going around, which is it's sovereign if it's controlled
by us, astric for us, astric.
And here's an example of T-Systems, and we see at the bottom it's trying to hide, right,
like it's very secret, and T-Systems controls that, and that means it's sovereign regardless
of what runs on top of it, right?
And AWS went so far as to actually require EU citizenship to work on the AWS sovereign
cloud, which means many persons in this room, including myself, who is not yet a citizen
due to the long line in Berlin, would be unable to actually work on this regardless of
our capabilities, right, because something about citizenship must bestow sovereignty.
And then there's a third story, right, which is something something military and sovereign
means, let's build military drones and police systems and all of these things.
And we can see a lot of this push in the changes to the AI act that happened, which basically
made an entire huge loophole for things like border control, military and police.
So something, something sovereign T means, let's use all the AI that we can for such systems,
right?
At least you see this with T-S.
Okay, so it's my opinion, and I hope you're with me that it's time to spread other stories.
And I think I was very inspired earlier today, and the AI plumbers room, I think there's already
some stories that I'm hearing that are echoed in this talk.
So let's spread some different stories.
First and foremost, I think very happily that I hear for many years here at Faustem is
let's try thinking about building hardware and infrastructure differently, which coming from
the machine learning side, so my background is in machine learning means we need to have
new silicon chips and we need to have new designs for silicon chips.
And how do we deploy this with any reasonable security?
Well, open Titan, which I think there might be some contributors to the project already here,
is one such way that we can start to think about these things.
And we can start to open up silicon for different types of machine learning workflows.
Alongside, there's also some pretty cool conferences growing around how do we actually
do this in more efficient ways?
And whatever we mean when we say green computing, how do we do that for machine learning?
And the eco-compute conference this year in Berlin was sold out, but I'd be very happy
to hear of other conferences that you might recommend, where people are talking about,
how do we actually increase efficiency of the types of workflows that we're running?
And use that instead of thinking what is the next biggest gigabyte center or whatever we can build?
And I want us to take very Bernie energy when somebody comes at us and they start talking
about how everybody flops they have or other metrics like this that the giga factor is
use because at the end of the day, this is usually an extremely inefficient use of the resources
that are available is also not possible with the energy density in most places.
And thirdly, I've been doing machine learning before giga factories, so I'm happily here
to tell you, and many of you probably too, that you actually don't need giga factories
to do whatever it is that we call AI today, and we don't always need the next biggest foundation
model for obvious reasons.
And how might we build things differently, both from a green compute point of view, but also
from changing the way that we think about AI communities or machine learning communities?
Well, I come from a background of decentralized machine learning, and I don't know for those
of you that are also infedited learning or decentralized machine learning, flower labs,
which is also European startup built out the first pre-trained LLM of 1.3 billion with
the same accuracy of its competitors, all via federated learning across multiple continents
with normal data center links, and these by the way were separate types of data center structures,
so they weren't all on AWS or Google Cloud or something like this, right?
So you can check out, I think there'll be some more exciting announcements this year for
other models that will be released the same way.
I also sit on the jury of the Splend composite learning challenge.
I don't know if you're familiar with Splend, but it's a part of the German government
that is looking for what they call Splendent Innovation.
So how do we take a leap with our innovation to go from where we are today to something
totally new, and the composite learning challenge is about how do we actually learn
across very disparate types of hardware, and when we're able to do that, we're maybe able
to expand our idea of what we need in terms of compute to actually do machine learning
at scale, and even to do machine learning at a variety of sizes, so not always training
something with billions of tokens or billions of parameters.
And I was here, if you're in Berlin, there was a few last year that will be a few more
this year distributed AI hacks where teams get together, compute was donated by AMD, and a
few of the teams from the composite learning provided the software, and some of the infrastructure
layer for running these.
So I encourage you to also do these types of things, and what we're moving there is
to what we call participatory learning.
So a place where you can show up with your own hardware, with your own data, and be able
to contribute to building a model, which you can also take a copy home with you, right?
And to do this, we also need to enable, kind of, how do you use that model, right?
Once you have it, or how do we reason about the different open weight models that are
already available today?
Well, over in the other room right now, unfortunately, at the same time, there's some
of the core contributors of GGML talking, which is basically, if you've ever run one of
the LLMs on your computer, there is a good chance that some part of it was using LMSCPP,
and then there's a good chance that that is also then using GGML.
And this is essentially ways that we can take something like TensorFlow computations, and
we can move them over to a variety of different hardware backends, and hopefully, eventually,
over time, this becomes kind of seamless and something that people can use.
But GGML, as far as I understand, still has like three core contributors, right?
So we really are, again, in this precarious types of situations where we have a very small
amount of libraries supporting, again, a very large or growing infrastructure around running
models locally.
We have a new act in the EU called the Data Act, it went into a effects last year, which
basically gives you the rights, if you're an EU resident, that if you buy something that
you have a rights to get the data from it, how this will actually play out.
I hope it doesn't play out like the GDPR data portability, which basically, a lot of
companies just hand-waved and give you your user form back.
But what we really are trying to get is how do we get at the data that you can then take
along with your local libraries, and you can run things locally for you and buy you
of you being an individual in this case.
Whoops.
And I ran last year at Picon Germany, Pided Germany, the feminist AI land party.
How many people are old enough to have attended a land party before?
Yes, if you want to come to Domestat, if you want to bring your computer, come to our
land party this year, we're going to host it again, it's an April, but we've basically
had a few different computers, that's my little machine, it's got 32 GB GPU in it, and we hosted
and served for inference, numerous models, we did a little bit of fine tuning experimentation,
I ran a workshop on hacking LLMs for feminism, which is a very fun exercise, and yeah,
if you want to donate computers, it's free, you don't have to have a ticket to Picon,
you can just show up, so I want to see more events like this, we open source all our kits,
so if you want to run your own, please do.
But by this, I mean, bringing your data, bringing your compute, getting together as a community,
and really having a participatory way, though it kind of also debunk some of these myths
around how machine learning and AI works.
On top of that, so most of my work is actually on deploying privacy technologies in deep learning
systems, what does that mean? That is a lot of random words. What part of it means is different
of privacy, has anybody here heard of different privacy? Use different privacy?
Yes, I want to see more of that hand, so different privacy is basically a robust, mathematically-based
probability-based method for us to try to measure what is the amount of information you're
contributing, and how can I attach that amount of information you're contributing to the level
of privacy that you might be leaking via your contribution. And by limiting this, we have
variety of ways that we're using different privacy. We're able to then give you some reasonable
sense of a guarantee. Now, of course, does this always translate? No, right? Technology is
in magic, and privacy is a social phenomenon, right? So I don't want to say all that, but I do want
to say, different privacy is available. There's lots of cool libraries. There's many amazing
researchers here in Europe working on different privacy. And I'd like to see those
logos grow beyond just Google and PyTorch and Harvard, right? I want to see the logos on
the different privacy that we have available for us grow. And if you're interested in integrating
different privacy and open source that you're working on, please feel free to come find me and
let's talk or find that person and talk with them. Sorry, that was very not private to point
at you, but yes. So let's build that. Let's make this normal, because when I think about
sovereignty from EU perspective, maybe or even from a European perspective, we have an understanding
here that privacy is a human right. And it's not necessarily every place in the world where
that is a fundamental understanding. And so I think this is one way that we could really add
to the diversity of projects and experiences out there when we think about AI and machine learning.
I come from a background of working on encrypted machine learning. I worked on the TF encrypted
library, which was the first library to do multiparty computation or encrypted computation for
deep learning training. We've TensorFlow at that time. Most of the cryptographers, the project
is kind of archive now. Unfortunately, most of the cryptographers that I worked with moved on to
Tsama, which is in France, and they work on new protocols for developing fully homomorphic
encryption, but also MPC protocols. You can prove these types of things. And they have a lot of
cool open libraries. There's open FH, she is growing and really cool. And there's plenty of
people that are starting to work on the compute problem that we have when we think about encrypted
computation. And to just summarize it, it means that we need maybe new types of chip. We need new
types of maybe even optical computing that allows us to do the expensive steps that we have
in encrypted computation and allows us to do it faster. And encrypted learning in inference is
real. It's certainly not comparable with what happens on plain text, but I would love to see
this side of this slide grow and change. And even to hear from you all, how are you building
maybe encrypted computation into parts of your projects? So, if anybody wants to dev room any of
these together next year, please let me know. Finally, open source communities. I mean, I've talked
a little bit here of geographies. You ask you these things, but open source communities, I think,
are a really good example of diplomacy overall. And I think we need diplomacy right now in a big way,
in many, many ways. And I personally think, and I don't know about you all, but I personally think
that open source can be a way that we achieve this diplomacy when other geopolitical tensions
are rising. And so, I point out, PyTorch, you know, this is one type of AI open source project.
I love PyTorch. I've been using PyTorch now for many, many years, but many of you probably know,
PyTorch is a project from meta and is mainly driven by core developers at meta, right? And so,
it really is kind of driven by that road map and that decision making. Of course, there's many,
many contributors who are not working at meta and many, many places that are improving other
libraries at touch, PyTorch that aren't meta, but this is kind of one community that, again,
is international and reaches across many different types of machine learning communities.
And then we have Pykit Learn, which is a very different type of community. So, I get learn
grew mainly through just open source contributors who for very long time were contributing
on top of their day job, right? It was not their main project. And Pykit Learn has an even bigger
amount of people using it. And currently, also finally got some funding and support from the
French government, including most of the core contributors are now working at a company together.
And yet, it still has many contributors across other places. So, I want to share this as two examples
where if you use this definition of sovereignty like citizenship, these projects could never exist,
right? Or if you use the definition that we don't ever use US tack or we only use this or that
or whatever these communities might also not exist the same way that they do today. And so,
I want to encourage again us to think through diplomacy as part of our work. And of course,
more than anything we need to think about prioritization, funding, and we need to be imaginative.
We need to think through conversations that are being have had right now around competitiveness
and scale. And what I think around these is not only we need more collaboration, which I'll get
to in a second, but we also need to decide like go the locks and the three bears. What is the
scale that makes sense for us, right? Because not everything needs to be foundation model scale.
There's many different types of machine learning beyond that and perhaps that is one way to counter
this conversation around competitiveness. And then I think we have to continue the idea of collaboration
and diplomacy maybe start building in how do we support open source as part of EU tenders and not just
the same five companies that always get tenders, right? How do we combine that with investment?
The composite learning is doing a lot of reaching across borders of how do we connect
HPCs with EU clouds, but I think there needs to be more of this as well. And how do we do this
in a composite learning or a federated learning type of setup? And obviously with ODS and supply chain,
we need to collaborate and do diplomacy to build new types of chips, new types of open silicon,
that allow us to run inference or training workloads, but also start to develop open standards
that can compete with people that have two year waiting lists for purchasing.
So I'm going to leave you with a few questions, but I want to hear yours. After this,
I'll be around today and tomorrow, I'll be hanging out in the privacy and internet devroom mainly.
But can I decide for what is sovereignty, AI, sovereignty, I mean to me as an individual?
Can I decide how I interact with a model? Can I decide not to interact with a model?
Can I choose what models I use based on how they work for me?
And if I'm doing that, that means that I need to be able to test locally. So can I run models locally?
Can I run them on my own data and on my own compute? And can I support? Can I take my portability?
And can I support projects, organizations not only with code and contributions in touch,
but also with my data, should I want to? And perhaps we can think about this from
whatever it means to have a European perspective. I think we need to talk about and think about
what does it mean to build human rights, like privacy and agency into AI ecosystems,
machine learning ecosystems and model development? How can we discuss sovereignty without
forgetting lessons learned from nationalism and from war, which I think maybe we have a unique
perspective to contribute? What becomes available if we forget about whatever we mean when we say
global scale? Your local park bench is not global, but it serves a purpose. Can we move beyond
this idea of scale? And how do we use our diplomacy and collaboration to grow and support
international open source communities? So I want you to challenge the sovereign status quo.
I want you to gather real constraints and evidence. If somebody tells you nobody does that,
I want you to reply yet. I want you to stay creative, get curious. I want to replace zero
some conversation, which means we have to do either or or or no but. I want you to replace that
with yes end and I want to nourish hope. So this is me deploying to EC2 in 2010. I nearly got fired
for it because the app that I was running fell over at one point in time and I got called into the
IT directors room and he told me you should forget about this Python thing and you should
forget about this cloud thing and then maybe you'll have a future in technology.
Thankfully I did not listen to this person. I left the company and I went on and did other things.
But I want you to have the Hutspa that's little K jam had then. I want you to have Hutspa
and courage to say no we don't have to do it the most obvious way. We can do it a different way.
And I'm pretty sure that in 15 years if you start that journey now you will look back
and you'll be proud of yourself. So thank you very much for your time today. I look forward
to talking with you later.
Thank you. We don't have time for a question.