So I think we are going to start.
This is when an AI system, when is an AI system free slash open for a lack of a better title.
And I'm Richard Fontan, but I'm just the moderator.
And we have an awesome set of panelists, and I think we'll just start out having everyone
quickly introduce themselves.
My name is Kiran O'Reardon.
I've worked on free software policy for about 20 years now, and I kind of work for open
form Europe.
Hello, my name is Zoe Koyman, and I'm the executive director of the F.S.F.
I'm Julia Ferrioli.
I am here on my own as representing my own personal opinions, and most of the time, I work at AWS, otherwise.
Hi, I have a black 25 years using writing contributing to open software.
I also hear on my own, so he might have thought I was affiliated with something else, and not just here as myself.
Okay, maybe we should, I don't know, this is the most important question to get to, but
given that I observed that the title of this session is kind of problematic.
What do you, any of you have any thoughts on what the scope of this question ought to be?
I think we're going to, at least touch upon the OSI's open source AI definition, which has captured a lot of
attention and criticism over the past several months, and it's sort of defined.
It reuses this legislative definition of, or OECD definition of AI, so-called AI system, which is,
I think is kind of a problematic definition, and you could be argued to be too broad, too narrow.
When we talk about open or free software, AI, whatever you want to describe it,
what do we mean by AI, is basically the question, because I don't want to have thoughts on this, should this be limited to
machine learning, or a subset of machine learning, or should it be AI in the sort of academic sense, or what used to be the academic sense?
So my background is, I have a bachelor's and master's in artificial intelligence, and human computer interaction,
last I checked, and I've been working in open source for basically my entire career,
I've been doing research in, I've done research in AI using open source software,
20 years ago, I think at this point. So I think that it gets very complicated when you're talking about
a system, because no one really knows what that is, despite the definition, we can't agree
upon the interpretation of that definition, and it's not necessarily the right
level to be talking about free and open source at the system level. It's like saying,
okay, well we've got an entire stack that we're going to call free and open source, including the
run-able components, which it's unnecessarily complicated, and we could simplify that a lot
by looking at what actually is in that system definition.
I'm not sure if the question of the AI system is something that we're going to have problems with,
because I always think like I know that a certain company that is disagrees with everyone
on the stage at the moment, and they travel around European Parliament, I'm a lobbyist,
I cross paths to these people, and they have this diagram with 25 different modules and they
explain to the members of parliament, this is what an AI system is, and this is what all the
different parts do. But if you have 25 modules on this slide, you're not actually expecting
anyone to understand it. So that means you're expecting people to not understand it, and you're
trying to put them into a position where you're hoping they'll take your word. So when I look at
this set of modules, I see I look at each one and I'm like, well that's software, that's software,
that's software, you end up with a 24 modules that are software, and one module which is an AI model.
So I don't really think in terms of an AI system, I think of a software system, and 24 parts
are software, and we've already decided pretty much what to do with software, so all we really need to
focus on is the AI model. I haven't yet found anyone who found a practical problem with the
difference in machine learning and AI models, but I also haven't understood the distinction,
I know that that would be described in tomorrow's presentation, or I think it would be described
in tomorrow's presentation by FSF, but I think in general the AI system, it's not the big
stumbling block that we have to focus on, I think that'll be some goodbye over time.
I think I agree with Julia to say that the word system is a vague concept that would do well
being identified when I think it's also worth pointing out that for us, at least from a
reception perspective, the word artificial intelligence is something that
gives people the idea that the application is doing something that we don't necessarily believe
them to be able to do, and so we at the FF at least limits our wording to the
words machine learning, because that's much more appropriate to what the application actually does,
intelligence is something that is still specific to humans, and I think we should for now at least
keep it there. And I think from where I sit as the person who I think was the first one to suggest
during the definition process with the outside that it used the OECD's definition,
actually painted a friend of mine on OECD at the time to be like, hey, fix this typo,
while we were looking at it in the work sessions of sorry, but my reason was this is an emergent
system, it's not intelligence, I agree, but it is trained. It's a database, in essence,
it's all code except for the data, and then some properties emerge as that data gets compressed
and put in a format that can be used whether to pattern match or to predict, to draw insights
in ways that humans really struggle to understand, still how these things are making the outputs
from the inputs that they receive, but in their use, they are automated decision-making systems,
in their systems. Let me ask this question, and I don't know where this would best fit in,
so I'm just starting out with it. Do you think we need a, do any of you think we need a definition?
So we have the free software definition, we have the open source definition,
those of their flawed definitions, but they've kind of served as well for, for, for fuss,
and they've been applied in practice to things other than software,
maybe you think that that fact is sort of trivial, but it is interesting. Do we need a definition
as some people have been assuming or can we just build upon and interpret the existing definitions
we have for that work for free and opens our software? I think the four freedoms were well
articulated and applied, still here, it's still a computer, we can build a system in that
programming plus data and we want the recipients of it to have the same freedoms.
This is a good question, and it's not one where the simple answer. I think the question
becomes what is the definition designed for what is motivating the definition.
If we're looking at it from a, a technical level for machine learning and the outputs of
machine learning, whether those be models or, or user-facing applications,
it, I don't think we do need a new definition for that. However, that's the, that's the
kind of maybe five, ten years ago answer. If we're looking at it from a defensive
position about pushing back against use of the word, or use of the term, open source,
in conjunction with machine learning models, then maybe we do just to say, no, this is,
this clearly is not that. I don't like that approach because I want things to be motivated from,
okay, we, we need this. We're not just going to artificially introduce a new term, a new set of
requirements, a new definition, when we have perfectly good ones that already cover both software,
data, and the combination of those. So there might be need for a new term. I don't think it needs to
be called open source anything. So we thought about this as well on whether or not it should actually
get a whole new definition. As more and more people are doing their computing through machine
learning, the concept of machine learning is for people not quite the same as just software,
and we have responded to people's requests to bring clarification to what a system like that is built
like, and it's not just software, it's also, so it's software licenses, but it can also be
creative commas licenses or other things like data, and papers that might have to be published with
a complete free application, and those all don't necessarily fall in the concepts that we as the
free software foundation have always advocated for, and so we're at least looking at expanding
upon the information that we've given in the past, and we want to make sure that the two things
are not considered the same. We want to make sure that they're not conflated, because we are
very clear about what is free software. We want to stay clear on that, and machine learning is a
trend that is moving forward really fast, and there's a lot of development in it, and it could
many different directions, and we want to make sure that we're talking about two different things
in approaching in. One of the issues is that whether we want to or not or whether we have a certain
timeline, there's already, for example, meta is calling their Lama model open source, and our
deep seek have been called open source as well, and then we also have legislation like in the AI Act,
it says that if your AI system is free and open source software, then you get a certain
lighter regime, and then how do you know if your AI system is free and open source software,
they have a 10-word definition that was copied from a page on Gnudelorg, and that's fantastic
that they did that, but we also don't have an interpretation for how do you apply this definition
to the concept of machine learning. So what I think we're going to need it, people are going to
be calling software free software after a model has been added, and then we have to have our own way
of deciding, is that still free software or is it no longer free software, and we need a way to
be able to correct people if they call something that is clearly not free or open source software,
if they call it free, open source AI, then we have to have a standard that we can say to the media,
for example, this is absolutely not open source AI, so I think whether we want to do not we
need something to to fill in the gap that will be created because people are going to touch the
two terms anyway, free software and this software package which now includes an AI system.
So, and that goes back to the defensive posture that I was mentioning, because if the
motivation is to be able to correct people when people are misusing the term open source,
yes, we do need a term, I mean we can call it, I don't know, like, publicly available,
we can call it like the table machine learning for like air, but if I don't want it to have the
term open source attached to it. So, in terms of how do we know if it's open source or not,
I mean, there are well-established ways of analyzing something that's been received,
a digital good to understand if it's open source software, the difference here only seems to
be around the data, I would propose at least from what kind of work I do around security,
looking at can we know certain security essential properties about this component? If we cannot
know them, if we cannot fix them, if we cannot study them or change them, then by implication,
it cannot be open source, because if it were, we could do those things. So, if I think you anticipated
what I was going to next ask, which is also I think the, to me, the biggest question here
is, or set of questions is about source source code. So, free software definition has availability
of source code as a requirement, the OSD, applying to talks about source code uses this idea of
the preferred form for making modifications, which is taken from originally from the GPL,
where it was kind of applied more narrowly. So, this, to me, this is, I think, a very
difficult issue, I'm not sure that if all of you think it is as difficult as I think it is.
But when we think about, I mean, when we go to source code, and I think this is about model
weights, basically, for the reasons you were kind of saying, when we think about what source code
means, or the source code requirement means for model weights. It's not, and the issue is basically
whether primarily whether training data is equivalent to the source code of the model weights.
What does source code mean in this technical context? It's not source code in a strictly technical
set. So, what is, is source code the preferred form for making modifications is the concern reproducibility
is that the ability to kind of inspect, for a lot of people talk about the ability to inspect
data training data for evidence of bias in trained models, what do all you think?
All of those are good points. I'll add a few to the list of questions. Identifying
vulnerabilities or backdoors, if there's an edge case in, let's say, an image recognition model
or language generation model, and certain types of inputs result in very unexpected outputs
that would be potentially catastrophic if that model were used in the wild, say, as an entry
sensor on a door in a security facility. If you cannot detect those, or once detected, if you
lack enough information or source code of some type, to fix it yourself, then it's not open source.
Right, if you don't have that digital sovereignty to fix it, even once you know about the
vulnerability, how could you call it open source? I appreciate that you're bringing this security
point to view because I'm going to take a different approach, which is about the empowerment that
free an open source software gives each and every one of us, which is the ability to at a fundamental
level understand what is going on in the code that you're using. And while for machine learning,
part of it is heavily dependent on code, the type of algorithm you choose to train, your model,
or whatever other machine learning approach you're using, it is also incredibly heavily dependent
on the data. If you use the same pipeline but change the data source or sources, then you're
going to get a different model. Some exceptions apply. So, I want everyone, every person who
wants to use a machine learning model to have that informed consent. This is what that model is
bringing to me. This is what's going to be represented abstractly, but I want to know what risks
that I'm taking on from a variety of perspectives. And I want everybody else who's adopting models
to have that ability as well. Yeah, I actually don't have that much to add. We
want to make sure people can fully understand study to the full extent of the application. And of
course, the key word that we tend to use at the Free Software Foundation is the word control. We
want to make sure that you can control your technology. And I think when you bring it down,
it sort of boils down to all of the different elements that I've been brought forward by the people
on this stage. So, I think the question of source code is the core question. And so, it's almost
asking for a definition and none of us have the definition. I think this is something we have to
realize that we're still in the early stages of discussing how we're going to deal with this issue.
And we might be 25% in actually getting close to a solution. So, I think at the moment,
do we or do I have a definition for source code? The answer is no. And we're hitting
some certain dead ends in our attempts. So, when I'm looking for the opportunities, I do see
that in the private sector and on the individual level, a lot of people are using AI already
and they're using it through Facebook or through apps that are using non-free AI systems.
And so, I think that's a difficult area, but there's also an area that hasn't yet adopted AI
very much and that's the public sector. And that's also the area where there is a legal
obligation to be transparent and to be aware of what you're doing, which you're citizens data
and to be in control so that if you detect bias, you're able to correct that bias. And so,
being able to detect bias and being able to correct that is basically another way of saying
study and modify, which is the summary of the difficult questions of applying the free software
definition to AI. So, I think rather than trying to come up with a definition of source code right
now, or at least we should be trying, but rather than thinking that we might have it already,
maybe the first step would be that we can do now is to give the public sector a set of principles
for what they should be able to do with any AI system that they plan on using. And by doing this,
we at least get them to also think about how they would implement a policy that would give them
the ability to do the modify and it buys us some time because we're still working in this as well.
Do you do any of you think that there's an over emphasis by some people talking about this issue on
modification or like the term modification? Because like one thing I've heard from some people is
like here's a couple of things I've heard and I think are kind of related one is that
well in practice most people modify pre-trained models. So they aren't training
models from scratch. So it isn't useful for them to have
access to or knowledge of and access to the corpus of training data to exercise the right to
modify in practice. The other thing I've heard that I think is a little bit connected to this is that
it wouldn't be just wouldn't be useful for something like an LLM, the amount of training data
so vast that if you just like through this enormous amount of training data that's at a user,
they wouldn't know what to do with it anyway they wouldn't have the hardware resources to make
use of it. Is there anything to that argument? I think off the cuff it's pretty hard to say if that
were true lie on an common crawl it wouldn't be so lightly downloaded. I think this illustrates
or this shines a light on of a very common problem with the discussion which is
the discussions typically focus on LLMs and when we're talking about machine learning
there's way more there's way way more than LLMs and the rise of small language models
which I'm really kind of as much as I'm excited about and anything these days that's one of the
areas that actually does exactly as the idea of small language models. So we don't want to center
the discussions on one specific part of the technology and then subject the rest of it to the same
um maybe delusion of open source free and open source principles that are we're seeing when
when this discussion centers around LLMs. So I need to pass off that on so that someone else can pick it up
so I think the question is is fine-tuning or incremental training sufficient to be able to take control
to make this software do as you wish it would behave as you wish and so I was very hopeful about
this last year because there seems to be two ways that we could have an AI model that could be
studied modified and one would be to get the training data and training software and the process
and then the second would be to directly study and modify the model and to be the retraining or
fine-tuning and the I looked into the second option because the first option is so difficult to
training data sometimes there's copyright problems there's privacy problems sometimes the
the data doesn't exist by the time the model has been trained sometimes the training would take
data center that is just not available to almost anyone on the planet and so the training data
model approach looked like a dead end and then I focused on the directly modifying the the model
you can you can do retraining and it does give you some control but I haven't found yet an
answer to the question of is it enough does it really give you enough control does it give you
yeah wait yeah so you like because it's a because actually yeah I mean you go first
so so specifically retraining latitude training fine-tuning of the model is mathematically shown to be
insufficient to remove hidden layers and back doors there's plenty of good articles out there
some scholarly research some by large security companies I think crowd strike is one that's
done some good research that I often point people to like try to understand it they wrote a nice walk
through for other customers and posted it in ongoing learning systems systems that are built
and then continuously learn based on what the experience in the field like detecting
malicious attacks against networks if they learn something that is that reduces their usefulness
or it adds a back door you cannot remove it later you have to roll back the training to a
chat point and start over after you scrub the bad inputs from the data set it's only way I think
that tells me enough like you have to be able to modify the data set to retrain so thanks um
I'm not a machine learning expert so I'm happy to hear what you're saying because it's something
that we have been discussing as well because it has been brought to us as well that that
people indeed say like retraining is actually possible you can control your entire stack with that but
I'm glad to hear that you know our thoughts about that which are insured so we should be sure
are on the same on the good path there the other thing that I wanted to point out is that
retraining is in many cases possibly something that works for people that want to indeed modify
a machine learning application and the other objection was of course the the vastness of the amount of
training they did that can go into a system and that of course can run into availability and you can
run into privacy issues and computing power and all of those things and that makes it
difficult I think to some of the two objections up is is we genuinely believe that we should be
creating a definition or criteria here at least that apply to all systems and not most systems
people do train their own models people do recreate their own models and
the idea that that people don't it's it's similar to arguments against open source software
when most people did not have the capability to recompile their kernel
they did not have the ability to modify I don't know for Tran but very passionate and principal
folks pushed forward with the idea that the source code should be available for you to look at
and study so just because it's hard is not a good enough reason not to push for it
so the question specific about the the OSI's OS AID and maybe this should be specifically for
for Kieran because I think most of you probably I think probably most of you would what I was
going to ask is does anyone want to say anything in support of the the way the OSI has approached this
issue which I would say is basically by saying it's kind of a compromise a policy compromise where
they've said you have to provide as I recall sufficiently detailed information about the training data
for a model such that a skill a person skill in the yard I guess can maybe it's produced
an equivalent technically equivalent model I think most of you would probably reject that but
Kieran do you have a no that that's indeed the weak link in the the OSI AI OSI OS AI definition
the the OSI AI OS definition the definition it says that you you have to distribute the training
data unless that's not possible and if that's not possible then you have to give sufficient
information about the training data such that somebody else can rebuild a comparable system
or maybe study a modify I can't remember the detail but the so this part of the definition
I expect will will change possibly drastically in the next version and just over the coming years
in general the I suggested to my employer that we should endorse OSI's work and you know we added
an explanation to our endorsement and the reason is that the the definition that has been published by OSI
it's enough already to be able to look at that definition and say well you know that explains
the Gemini and Deepseek and Lama and online AI services are not open source AI and really that's
all we have right now that we don't have a definition that covers all of the cases we do have
meta running around the European Parliament saying that there's other AI things are open source
and we do have European legislation which says you get a lighter regime if your AI is free
and open source software so having something that we can use is positive and you know it was
flagged as version 1.0 there will be a process for developing a version 2.0 or 1.1 or you know
and so the second reason why I think it was worth supporting this work is that I think like we
had disunity 20 26 years ago when the open source term was created and this
took 10 15 years to settle down if it even did the situation back then was actually a lot
a lot easier because back then it was two groups of people who both cared about the software
who were disagreeing about something. Today we have a situation where there are there might
also be two or multiple groups that care about the software but there are also very large companies
outside that would love to exploit the software or would love the software disappeared and I think
by having disunity right now this year we have a much higher risk that we give entities that
would like to harm the free open source ecosystem we give them a lever and so I think that the
definition of particular part about what you do and the training data isn't available that needs
serious work it's going to take probably years I hope everyone will get involved and this is part
of that and I think it's more the unity of everyone can find the work is the important thing
rather than the text of this version one window which will change. It's very difficult for me to listen
to that because I just I don't see a reason to adapt what we believe in just because the OSI
wanted to publish a definition and I and I think also on some of the smaller arguments that you
made I mean made as lama license is known free in any interpretation that you can go with so they
can call themselves open source or free software all day one but but that's just not what it is and I think
most people also know that and from what I understand I don't run the whole ways in Brussels but
from as far as I understand most people in Brussels actually already have a concept of what open source
is supposed to mean in these laws and I think the way that that will develop it doesn't necessarily
have to be something that we need to rush towards we need to make sure that we do it right
and not fail because we see all kinds of issues with not publishing something today.
I hear you concerns I do and I understand the motivation and behind that I also agree that we
shouldn't compromise our principles for the sake of convenience um here's the thing if it's a one dot
o it's got to be workable you have to assume that's the last thing that's ever going to be
listen to or take it in that it's if it's a one dot o then as long as as long as people satisfy
the requirements as spelled out in one dot o that's what's going to be considered open source
AI for the foreseeable future the disunity is unenforced so there could have been unity much
earlier on in the process the disunity that stalled around data information and the requirements
of data um to be made available or not that disagreement was completely unforced and so we could have
been much further along if more people with the expertise in machine learning and open source had
been allowed at the table so you're opening point there are I think two camps
those who are ready to compromise the values the principles of open source in the name of consensus
right now to have a workable definition that achieves a modicum of forward progress and those who
say our principles really stand for themselves and should not be compromised that set was unfortunately
I think pretty overruled in the in the development of a definition and so the principle
that the users the recipients of something need to be able to enjoy the full
ness of the four freedoms of free software I don't think that should be compromised
I do agree that moving forward together is good but we do need to find a way and a number of
proposals for how to do that most of which revolved around using two terms for example open weights or
open science address that gap yeah that was actually a question I was thinking of asking does anyone
support this idea of having you know solving this this kind of debate or disagreement by having
two different levels of openness or two terms whether it's open weights open source or something else
I think I've heard heard many proposals but is that is this such a complex domain I guess that you
really need you can't you can't solve the problem without having two different layers or two
different types of openness or for or foriness there's there's tremendous value to a recipient to be
able to use the model as they see fit I think everyone that I've ever spoken to agrees on that
that freedom to use and the ability to do lastly or training or a little bit of customization which is the
freedom to study and modify to a degree also very very important there's a whole large
swath I would say most of the models out there today would fall in the category where they cannot
distribute the data for whatever legal reason or lack of availability or size or discussed already
sharing the data gets complicated there's value in that but there's a higher principle
in the full freedom to study and modify that's where I draw the line is there utility in having
multiple categories of transparency sure it gets very complicated though because people have
hard enough time with the idea of free and open source software already so there is utility just
like there's great utility and having freeware and open source software and I would say that
without the data you're looking even on open weights you're looking more towards kind of a freeware
model than anything that could realistically be called open source or open something as applied
to machine learning models so this is where yes having the different out specifications for
what the level of transparency that you have in the system model being discussed it's useful it's
the practice that gets hard because people like simple people like a checkbox that's yes or no
is this free and open source yes no yeah well I mean we we we we do go by this binary of free
non free and if something is non free then it's non free and not a lot more and that is something
that we do want to hold on to as much as possible but even we at the FFC that that in some cases it can be
you know or like traditionally BFF has always said if something is non free it's also non ethical
and we of course also at the FFC that there is difficulty there that you can it can actually be
that an application is being built for ethical reasons cannot release some of its training data and
that's an issue that we struggle with because we really don't want to create an extra level of
discussion there we don't want to give people out so we don't want to we want to make sure that
that free non free binary continues existing because as soon as you start eating away at it it will
eat away at your freedom so yeah I don't support the binary approach like from what I've
generally seen companies that don't share our interests they talk about spectrum and they talk
but different degrees and they have a complicated system and I'm usually just suspicious of
people who present complicated systems and then expect politicians to draw a job of a policy based
on taking the word for it but if I could quickly respond to the previous question I think the
focus is incorrect I think focusing on who endorse this document or who made what comment or
who participated in this forum I think in five years time will have forgotten all these things I think
like it's a 1.0 but it was specifically called a 1.0 because they said we're going to change
in the future this is not you know it's not going to be set in stone and I think that we
have to avoid you know explaining it to two camps I everyone that I've talked to a better AI
in the last few days at least I've been telling them NFSF has a presentation at time clock in the
cable and do you know where the cable building is it's over the top of the hill a little bit
still after it's the big one you know this is not a question of you support this or that
you know you can't measure values or you don't so I think that the dynamic of all working together
you know we're so early in this like we've done maybe 25% of the work and there's so much more to
do and I think that you know we can all fall out maybe at the end but don't fall out at the start
that's my I guess at my point as quickly respond to that and also say that my comment earlier
made it seem like I don't respect the word work that I was I did I want to make sure that it's
clear I respect all the work that went into getting them to where they are I just don't necessarily
believe that that we should feel ourselves be rushed by outside influences and that we need to be
careful and and sure about what we're ending up publishing and it's fine to be aspirational in that
and set an example that people can then live up to and then I also think that if I find a
challenging to think about a definition 1.0 that is definitely going to change as a definition
and so I just yeah that's a challenge that I have.
Yeah and I don't I don't disagree with what either of you are saying I think that moving
together is good but it's also about knowing when you're not accurately representing what's
being discussed and what's being said and we saw a lot of that in the early process so
there are a lot of one of one of my big concerns with open source is you don't know who's leaving
generally and you definitely don't know who's never showed up in the first place so
we don't know whose voices were went unrepresented and we don't know whose voices were
silenced and we saw that so it's not necessarily about the idea the ambition the concept of
figuring out a way to navigate these challenges together it's a matter of execution.
So unfortunately we're running out of time so there were so many other issues we could have
gotten to but maybe we could just give everyone a chance to say whatever they want to about this
topic we'll go around and kind of close on that if you if you want to say anything.
I guess the most important thing I've thought of recently is just the idea of the public sector
and you know maybe the next step is that we can make progress by explain the public sector
that it there there's obligation to detect and fix bias which is study and modify and that could
bring them into the discussion they could even help us get to a final decision on this.
We have a talk about this tomorrow and a different feeling so come over there as well if you
want to add to this question or ask us more questions we'll try and present at least
more complete image of what the FSTNs on this particular issue is then and I guess as a closing call
I would like to say to people if you are a machine learning expert or if you do have
feel like your opinion is not heard or you feel like it's worth having a discussion with
us about anything that we've published so far then please do contact us because we want to hear
your voices and we you know we need to learn from the people in this room or the experts but also
the believers and supporters of what we have all been doing all of these years and yeah so contact us
and we'll discuss. My parting thoughts are don't when you're thinking about machine learning
and open source try to think about all of the benefit that you've gotten throughout the course
of your lives from free and open source software and let's not rob future generations of those
same benefits and opportunities in the name of expediency nothing to add to that so I think we better
end it there because of time I don't know if there's any time for a question no no okay well thank
you to our great panelists and we are