Hello, welcome to the next speaker.
Alfonso will be talking about open way dilemma.
So the reason precise moment when the safety of artificial intelligence model for support
work, when it evaporates.
And it does not happen when the model hallucinates.
It does not happen when the user coaxes the model into harmful behavior using a gel-breaking
prompt.
It happens quietly.
It happens at a speed of a download.
It happens when the model waits, those lower parameters that constitute the cognitive
architecture of the model itself, leave the control perimeter and enter free circulation.
Now, in that moment, the acceptable user policy stops being a control mechanism
and becomes essentially a piece of paper.
In the moment, API frauduling, monitoring, and each and every enforcement mechanism
vanishes.
The model stops being essentially a service and becomes a material.
It becomes clay.
Now, if you work in open source, you already feel what this implies, what does it contain,
right?
Because that clay that model operates are just software you can expect, but they are capabilities
you can fork.
So, good afternoon for them.
We are here because we believe in that clay in one way or another, right?
We believe in the convenience provision of innovation that open weight model such as
Lama and Strahdip Sik have unleashed.
But we believe essentially that doing inference for tuning or some very limited extent training
or now of premises prevents monopoly capture and drives transparency.
But I am here to tell you that that very openness created the security paradox.
The kind of security paradox that we cannot longer ignore.
For 20 years or so, we leave by Laura's last in Maxine, code is low.
The architectural software regulated behavior.
But in the age of open weight I, code is no longer the low.
Code is the substrate.
And once the substrate is only on the user GPU, the low is whatever the user choose to be, right?
We have entered here what I call a mitigation gap.
When the one side we have regulators in Brasser, in the sea, trying to regulate AI,
I see if they were softer as a service platform, right?
On the other side, we have the technical reality of open weights.
Where control is distributed is anonymous and is irrevocable.
I am also the Gregorio.
I am a researcher at the intersection cybersecurity research in AI policy.
I spend my career attacking the funding computers.
And recently I set across the table or the European Commission where I try to help them understand
that we cannot regulate a downloadable file in the same way we regulate cloud API.
So today in the next 20 minutes or so, I want to walk it through three key ideas.
First, the threat reality, the technical evidence that the skill floor for fancy cyber operations is collapsing.
Second, the policy trap.
Our intentional regulations such as the EU AI Act,
could accidentally kill open source if you are not careful in what we are doing.
And third and lastly, the way forward, how we fix the open weight dilemma without killing open source.
So let's strip away the eye and look to the signal place.
We often hear that AI risk are fictional, that are speculative.
The reality is that they are not offensive capabilities of open weight models are real
and are here and they can be measured.
So look at nitrate occult framework for instance.
This is a framework for measuring the effective capabilities of open weight models.
We particularly respect with cyber security skills.
And in early 2025, less than one year ago, structure evaluation showed that open release models,
specifically the reason in models such as Deepseeker 1,
achieve over 90% accuracy on challenging offensive cyber knowledge tests.
Now, we are talking about expert level knowledge and proficiency in vulnerability, discovery,
privileged escalation and exploit chaining.
They can be downloaded by everyone everywhere on earth.
Now, possessing knowledge is not the same as secuting a clever arc granted,
but the gap between knowing and doing is closing.
So the unique danger of open weights is not just that they know something peculiar about these fields,
but rather the constraints the security guard lace can be easily stripped away.
When a company release a model, they first try to align it.
They use reinforcement learning from human feedback in order to teach the model not to what
or not to generate fishing emails, not to build a bomb.
So in the close-up guide, this alignment is sticky,
but in the open weight model, this alignment is superficial at best.
So my research and the research of other people in this field, possibly also somebody in the room,
has shown that safety in this space is not a permanent attribute of the model,
but it is a team veneer.
With a consumer grade GPU such as this one, a few hours of interning,
or even surgical techniques such as obliteration that suppress refusal back to or selectively,
an adversary can strip away those security guard rays with this.
This is not just theoretical.
In early 2025, we saw the emergence of tools like Santerus AI in darkenate forums.
These are not chatty bots.
These are modular autonomous attack platform running on private infrastructure,
empowered by custom tune open weight models.
They don't rely on open AI APIs, they don't have a kill switch,
they automated the Rogerius cybercrime by generating polymorphic code that can escape the detection,
or scaling spearfishing campaigns to 1000 targets across multiple languages,
or do vulnerability research faster than we can typically do in real life.
This is the so-to-speak democratization of a fence.
The floor of a fence is cyber operation is collapsing,
and this brings us to the core problem for our community working in intersection of cyber security and AI.
Cyber conflicts are typically shaped by asymmetry's of power.
They typically revolve around the idea that we can control,
who gives access to what specialized knowledge and tooling.
But open weight AI may receive this power imbalance,
they may receive this power asymmetries by giving state-level capabilities to script key
this and criminal syndicates alike.
If we open source community do not have a good answer for this,
a compelling answer for this.
I promise you the regulators will find the answer for us,
and we are not going to like it.
And this brings me to the policy trap.
I've been deeply involved with the consultation process of the EU AI Act,
and specifically about the general purpose AI code of practice.
The initial instinct of the regulators was to regulate the provider,
the entity that trains the model,
the logics in sound the first.
If Ford builds a car,
Ford is responsible in the added to the brakes' failed.
So applying the same line of reasoning,
if Mr. Lomita releases the model,
they should be responsible if the same model is used for acting a bank.
But here lies the trap, right?
Because the EU AI Act creates obligation for AI risk AI systems.
It demands risk management.
It demands post-market monitoring and record keeping.
Now ask yourself,
how can an open source developer perform post-market monitoring?
Realistically.
Once you download the weights,
the developer has zero visibility into what you do.
They cannot log your problem,
they cannot recall the model,
they cannot push security patch to your laptop of edge devices.
Now if the low-insist,
the original developer is liable for their stream issues,
the rational business decision is to do not release the open weights in the first place, right?
This was the mitigation gap I highlighted today European Commission.
We risk creating a regulatory environment
that implicitly favored close proprietary AI systems.
Now because they are necessarily better or safer,
but because they are controllable.
So we add to fight for a new legal architecture.
And we secure a critical victory in the interpretation of the Act,
center around the concept of substantial modification.
This is the legal firewall protecting open source.
So bear with me for a second, please.
The argument I successfully advocated to the European Commission is the following.
When a third party downloads a model and fine-tune it,
especially for the sake of removing the security garrays of specializing the model
for malicious use,
they are no longer a simple user.
They become a new provider.
So by performing the substantial modification for any
meaningful notion of substantial modification,
the liability shifts accordingly.
And so if I release a general part of the model and you obliterated
to create a fishing engine,
you have created a new AI risk AI system.
The four you become responsible for risk assessment
and in turn you become responsible for compliance.
This distinction is vital,
because a knowledge that the technical reality
has safety of the model is conditioned to the deployment context.
It moves the regulatory crosshair from the creation to the modification
of the weaponization of the tool itself.
However, the battle is not over,
because there is still ambiguity about the open source
extension present in the general purpose AI code of practice.
Does it apply to open weights that do not release the training data?
If we stick to the strict open source initiative definition,
then current models like Lama and Stral do not qualify.
So we need a pragmatic interpretation that protects the practice open
weights model regardless of data purity.
So to speak, provided the documentation is transparent.
So how do we navigate this my field?
We cannot close the source, we don't want that.
But we cannot ignore the possibility that the open weight model gets weaponized.
We need a new social technical part,
a new standard or responsible release,
and here it is what does a look in practice.
To begin with, because we cannot control the model after the release,
the actor releasing becomes the single most critical moment
in the lifetime of open weights model.
We must treat the documentation not just as a simple rhythmy file,
but rather as a legal and technical doorie
that is passed to the downstream actor to the downstream user.
This means the team model cards are explicit,
that list explicitly offensive capabilities
needs to be populated after proper reading.
So if your model scores let's say 80% on the court benchmark,
then let's write it down on the label,
let's write it down on the model cards.
This is not about admitting the feat,
pretty the other way around.
It's about allowing the liability shift.
By disclosing the risk,
we transferred the duty of care to the downstream actor.
Second, we need to move away from the DL Bunny models
based on parameter count and computational thresholds.
Instead, we need to focus on specific irisic capabilities.
Can the model synthesize biological weapon protocols?
Can the model discover zero-dibular abilities in C++ code bases?
Now, if the model have this such type of capabilities,
perhaps the corresponding weights need to be released with some kind of friction,
and this friction may entails tire access or get at the end of your customer protocols.
Where the general reasoning core remains open.
Now, granted, this is technically art to do is any sub-optimal,
but it's better than black and bang.
And finally, we must recognize that only defense against AI scale of fans
is AI scale defense.
Look, we are in the 2026,
and it's clear that many of the assumptions security problems
will be the pawn are fully apart.
Attackers are no longer constrained by human speed,
with AI reconnaissance, exploitation, and lateral movement
can now run continuously and empower a little.
What used to take weeks to complete now takes just minutes if we want to?
Most defensive models still assume time exists,
but the audic testing assumes attacker moves slowly,
reactive detection assumes there's time to intervene,
a manual validation assumes humans can keep up with all of this.
Those assumptions are breaking apart.
No, because teams are failing,
but because the model itself is understrain.
The real question in 2026 is not our risk cure.
The real question is, can we run security problems at scale
without adding humans becoming the bottleneck in all of this?
The same open with models that attackers use for finding vulnerabilities
can be used for patching vulnerabilities.
So, we need to prioritize AI enable defense.
We need to flood these metaphorically speaking,
this security space with defensive technologies.
Because if the skill floor for attackers is dropping,
we must raise the floor for defenders faster and accordingly.
The open source is probably the only way we can do this fast enough
for the largest number of security stakeholders around.
Because security throws security never work in the past
and it's not going to work any longer in the future.
So, let me conclude with this.
The era of code is low,
which is to say,
the architecture that itself enforce the rules of the game
is definitely over when we speak about artificial intelligence.
In the open weight era, the code is low less.
It's fluid, it's everywhere.
This terrifying freedom is also our greatest strength.
The mitigation that is real, the risk are high.
But the solution is not to retreat beyond the wall garden or proprietary APIs.
The solution is to build the governance model that is well matched
to the technical realities we have out there.
We can do so.
If we shift the ability of actors to the actors who actually control the system,
if we are radically transparent about the real work capabilities
of the open weight models we design.
And if we use the power of open source to build the new system of the internet.
Ultimately, we are the architects of this new reality.
So, let's try to build the kind of reality we can survive from.
Thank you.
We've got time for some questions.
So, is there someone with questions?
I don't see anyone.
I guess the communist model works within you.
But yeah, what do you think about, for example, the or global picture,
are the governments and organisations?
Well, thanks for asking.
Because you need over the next one year or two,
my ambition is to bridge the gap from a regulatory perspective,
also overseas.
But you know, in Europe we have a game.
You typically joke saying that the American awaits China manufacturers
and we regulate things.
Because you know, the power of EU regulation is that we are using the market lever
as a bargaining chip.
And we try to put forward the kind of regulation that are cross nationals,
because the cross beyond borders and by the time other players,
around the globe, want to do business with European entities.
They need to comply with our regulation, right?
So, by operating at a EU level, we are already having an impact overseas.
Then of course, there are different sensibilities around the globe.
And it's my ambition to work on that over the next 12, 24 months or so.
Any other questions from audience?
You said the collect response to kind of a aggressive AI is reasonable with defensive AI systems
and the open source specifically.
Are there any projects we should keep in our effort?
I don't have a taxonomy because this kind of space that is moving very very fast.
I also is a cut-and-mouth game as per usual.
But you know, I've been working in the offensive cyber operation working group,
the Royal Old University of London, which will try to brainstorm
how offensive cyber operation are going to evolve into the future.
And one element we agreed upon was that the tempo of everything
is going to be dramatically affected.
So, this is the reason why it compels me to suggest scaling the fans using AI
because humans are not going to keep up.
Any other questions?
Maybe the last question?
Any wishes for software developers who are not developing new open-weight models
but use existing models for improving security and safety in their applications
when it comes to like plunge action protection and other issues?
Well, if we don't measure things, we cannot manage those things.
First, most important thing I would like to do myself and with the help of everyone
here is to measure the factic capabilities of open-weight models.
These have been done to some limited extent by metric with the framework
but they didn't publish in open-source data set in corpora.
So, a very first step can be replicating those results and build incrementally on the top of that.
Some last question.
If not, thank you for your dark.
Thank you for the questions.
Thank you.