Howdy folks, hey, so I'm Preston Dostor, I'm a board member of the Nivenley Foundation.
You only have to listen to me for like 30 more seconds.
I'm just here to introduce Quinn Tessons.
I've had the pleasure of working with her for the last four or so years on both the
Nivenley Foundation and how we're working to help the open source community grow and
hack a term, you'll have probably heard of hack a term, we're a massodon community
for queer folks and tech folks and I think right now we're like the 13th or 14th largest
instance but you know in that time we've encountered a lot of things on the social web, a lot
of good things, a lot of maybe not so good things and Quinn Tessons has had the opportunity
to do a lot of great research into new and emerging threats to our open source communities
and with that, I'll just let you take it away and talk about the data.
Hi, good afternoon, I am going to be talking about data, Preston is correct, but before we
get started I just want to do a quick check on with everybody because the event is a bit
chaotic and just remind everyone to please have your water, your snacks, your meds, and
your rest strategies on your person at all times.
This is a very large and distributed ecosystem and what an excellent time to practice what
that means in real life and with that I'll just jump right in and get started.
So before I get into the data I will also be doing a quick level set on open source
as an ecosystem as not everyone has entered open source at the same time or been here
the same number of years and I want to make sure that we understand a little bit of the
impetus for some of the things I'll be talking later early on.
And so when we talk about the launch of open source it was launched quite a while ago.
She says with some nervousness a few decades ago and the entire landscape of what technology
overall looked at at that time was very different.
So we see in the early 1990s home computer ownership was incredibly rare and our net
access was incredibly rare.
Tools were incredibly cludy and it was just kind of hard to get things done now.
This doesn't mean that technology didn't find a way to make that successful but it does mean
that the communication was difficult in different ways than it's difficult now and things
were slower and just didn't move at what we would consider to be a modern pace.
We also understand that because it was a new movement it wasn't running anything that we
would probably call critical at least not now.
It wasn't running your healthcare, it wasn't running your national infrastructure.
It was starting to run web servers and people were not only getting grounded in the
just of the movement but also the implications of what it might mean.
When we moved forward about two decades later so 1990s to the 2010s this had already started
to shift pretty dramatically so we went from having minimal home compute to some home
compute and instead of having internet access kind of nowhere most homes did start to have
it and when we talk about whether or not they were shared devices and people doing lab
access, yes there was lab access but again there was also home access and this is also
one social media and social tools as a consequence started to both emerge and start
to mature over the course of a decade.
This also means that by this time open source went from being a new movement to being
about a 20 to 30 year movement depending on your origin timestamp.
This also means that it is running critical infrastructure by the 2010s.
You're looking at things like it's running your healthcare and the other examples, government
infrastructure etc.
This is where we start to see it really be important to everyone.
There's also the tooling barriers start to disappear.
Tools became easier to use for everyone not just technologists and this means that what
is the risk benefit analysis for any type of threat also shifted because you did not have
to have a persistent education or actor to have an amplified effect and I will get
into that in a minute.
But of course from no longer the 2010s around the 2020s and now we have not only everyone
has internet access we have people with multiple devices all with internet access that's
kind of blanketing the globe it's no longer a home wire so to speak and this means
that you don't have things like the lab set up or even a single home shared computer
as people used to have you have all these chronically online devices where people can
never truly turn things off and this also has the effect of things being always online
including discussions in open source or activity or code or public lesson etc.
At this point a lot of things are still true mature ecosystem is still true the tooling
barriers are even lower and the value of open source is incredibly high so over a time
when I was looking at the supply side and demand side evaluation of open source we have
the supply side was being calculated as early as the early 2000s at a few billion dollars
and the supply staying peaked and kind of leveled out at around 4 to 5 billion estimated
over the 2010s but when we have the 2020s was when we had the first demand side evaluation
of open source which was between a and i and trillion dollars so this is a massive massive
industry that is being supported mostly by volunteer labor and in fact when we look at
the rapid scale we can see just how many code bases and not just the code bases but
their components are being used at our in open source 97% with 911 components per application
this is not something I don't think anyone would have been able to explain to people
in 1990s could be a consequence of a mature ecosystem at this pace and when we talk about
how people are contributing to open source most technologists went from contributing to open
source if they saw a project they fancied in the early odds now everybody is contributing
to open source as an entry into the tech industry as an entry into open source specifically
and this means that open source has gone from an anziant movement to valuable accessible
and critical everywhere and this is very important to understand because I feel like we all
feel like open source might be going through it right now and some of you are laughing yeah
it feels like it's going through it and so because I'm a data driven human being I thought to
myself I wonder what would happen if we try to substantiate the feeling the vibe because you
can't get a conference presentation on vibes what the data seems to look like and so I grabbed
a poll about 30 organizations and open source these are open source foundations of a variety
of sizes and the idea was to take a decent cross section of the open source ecosystem and I want
to say at the front a few important constraints and boundaries one I'm only sharing aggregate data
I will not be talking about individual projects or foundations not even if you ask me after
I also I'm going to be talking about what I call a crisis event which is scooped me in a
collapse or a near collapse event this is something that causes the death of a project or its
foundation or a permanent schism and a project or its foundation anything that is kind of a
death now to that ecosystem and then all crises that I've researched even though I won't be
talking about the individual ones are in the public domain so even if you do manage to think I
think I know what this is about you may and you can probably Google it and I still won't answer you
but because I'm not telling you who the foundations are I am going to tell you the shape of what the
foundations are so it's roughly 50 50 split of four profit and nonprofit foundations with a slight
skew into the nonprofit and that shouldn't surprise and I also balance them across t-shirt sizing
between so-called extra small and extra large the idea being that if you're a smaller foundation
with maybe one to two staff or maybe only an executive director and a board or something like that
with a project to your name you would probably be on the small side and then an extra actual
large would be something with dozens or hundreds of projects as you ramp up from large to extra
actual large level setting across the founding ages was a little harder there was a long tail
for the first 25-ish years or so of open source for reasons right and then there was a surge in
founding between roughly the years 2005 and 2015 which is in the lower two halves there of that
graph but there is still a decent spread of organizational ages in this data set as much as was possible
and now for the actual data so in order to map this out I plotted it in a histogram so this is just
the net number of collapse events in the 30 foundations that I cross sectioned for open source
and just to be really clear I didn't map crisis events and see which foundations have them
I picked the foundations first and that will be relevant to understand that that's while there
may be bias in the system and academic rigor and all of that the order of operations was
understanding foundations first in crisis second the other thing to understand is that they do seem
to have distinct types there seem to be distinct events that cause a collapse or near collapse event
you can see one of them that's easy to codify as a licensing crisis so a licensing crisis would be
one of the corporations for example choosing to change the licensing of the open source
and that may be causing some conflict with a community who felt like they gave their intellectual
property under one understanding and having the understanding change but it is not the only type
in fact the other four types are all human crises codes of conduct licensing we skipped governance
moderation and leadership so of the five primary crisis types that seem to emerge by category
four out of five of them are about people and also it's worth mentioning that basically every
organization in the data set had a crisis events at least once and many multiple so this also
means that regardless of size and age and et cetera there was a near collapse event for that
ecosystem for basically everybody there's also something that we can understand when we see a
pivot in the data like this it means that the older foundations had more time before they had to
deal with an existential crisis than the younger foundations an older foundation basically had a whole
founders worth of career to stabilize mature expand get resources et cetera before it
had to deal with an existential event whereas the younger foundations barely have their bylaws
ink dry and they are also having a crisis event and if we compare by year founded we can see that
this is true consistently basically having for the past couple of decades the other thing I wanted
to take a look at is we can see that despite they're not being many crisis events
capped at the year 2000 mostly for my ability to independently verify events we don't see many
collapse events right for a good few decades pre 2000 included until 2014
so we can see after 2014 and during 2014 we have a surge of events relative to the
prior years and then continued surges after the fact and since it's relevant to the talk I know
that people do like to talk about rust and access in particular often usually due to moderation
or steering committee or the things that people like to talk about but the reason I bring you
up at all is well they are very similar they're very large they're very interesting they are not unique
they are not unique even for the calendar year that people like to talk about in fact in combination
they are two of nine events of that particular year and we can tell that communities know that
this is happening whether they're articulating or not because they start to respond by changing
things in their ecosystem to adapt to this reality and this means we're going to look at the
20 times a little more deeply as a decade of relevance and first I want us to understand a little
bit about actor versus non-actor harms and to understand actor harms we're going to talk about
what everybody talks about for a quick minute so we have XCU tells and when we look at XCU tells
the main thing to understand about XCU tells is that it was a stained campaign over multiple years
that weaponized the contributors burnout and mental health so if we want to talk about it in terms
we can say that the human state was the exploited vulnerability this is not a code vulnerability this
is a human vulnerability and bottleneck in a way because they were able to infiltrate the project
and get to the only person that mattered to them to have their objectives meant whatever they were
and this is an interesting discussion that I will be diving into after I explain this because it
talks about we talk about trust and safety and security and human harms and so forth but this is a
very health oriented a very mental health oriented thing that we need to be aware of and it's also
relevant when we talk about non-actor harms non-actor harms are the amplified effects of what
happens when you have always online incredibly interconnected spaces so this is not an actor harm
but sometimes you can't tell the difference right you have a pair that blows up because the
community is mad it hasn't been merged yet and is are people mad or are if someone fueling that
fire to make them more mad and they do look very similar on the surface and neither requires
technical sophistication and the reason I want to bring this up and I mentioned it in the beginning
a bit long form is the tooling barriers to entries have dropped very low you do not have to have a lot
of complexity in your tools or a lot of sophisticated understanding or even significant research
and diligence to have the impact of a dedicated actor in the 2020s right anyone can spin up a
chat army even if they don't know what they are doing or what to call it because the tooling
allows them to do this and for all of these reasons I want us to take separately more deeper
analysis and academic rigor and expanding the data set because I think that we should take a look
at some of the causes and factors and amplifications that are happening there in and so to quickly
recap we have about a 40 year old ecosystem that grew from a small movement to a very large
set of ecosystems sorry we are participating in these earlier communities with the understanding
that they were small and that that smallness had an artificial gate of a type the tooling was hard
and so people really were only joining if they wanted to be there for the most part and if they
could get past the tooling right and while all of that has improved this means that everything is
easier and some things are double use because modern tools have the effect that anyone can do
kind of anything we understand how to talk to each other we understand how to connect to each other
we can commit PRs with actually not knowing how to write code as of right now with increasing relevance
right foundations are experiencing crisis events earlier and earlier and this is brought spread
across their age their size or their perceived importance right so it doesn't matter if you
feel important or not it doesn't matter if you feel big enough or not or mature enough or not
you're kind of going to be going through it by definition the volunteer teams as such don't know
kind of what they're walking into sometimes because in a lot of moderation circles you have people
who treat their ecosystems the chat the discords the zoolips and so forth as if they are more
closed than they are just because of the cultural momentum of that mindset and when they encounter
things that they don't understand they may or may not have the language or tools to tell leadership
what is happening or why it's happening or why they're taking the actions they feel they should
be taking and these communication breakdowns can and have become very public and all of this
leads to people having more difficult early detection because if you don't know how to articulate
what you're looking for you very likely also don't know how to map it out how to look for a
signal misalignment what signals even should be aligned or mislined for you to do continued research
or for you to do overtime research because again you're not usually distinguishing in the moment
if somebody is being authentic with you or not you're looking at behavioral patterns over time
which means you need to have enough data and of granular data to be able to do that
and all of this feeds into how do we keep our open source communities open
because if we understand that all these chats when you're having these amplified effects it can
start to make people feel like we need to shut it down somehow or close it down or do something
very reactive in order to protect ourselves and while it is true that we need to change our mindset
some we can protect it without being quite so drastic just by improving our understanding of the
situation and acknowledging a changed reality that we find ourselves in one of these is to
down acknowledge but also analyze and address what that means so we build over time what we think
we need to say see if we encounter an event that we did not have adequate data to do a post
more than even if it's internal only we know we need to increase the type we know we need to increase
the type of data that we are gathering and apply that future forward so that we can analyze over time
in the only way that we can do that is if we start to address things differently again over time
so that we stop engaging with potentially inauthentic conflicts as if they are authentic every time
one way that we can look at this is to look at our contributor ladders these are a concept
that are pretty familiar I believe in open source communities in general most people associate them with
code though you get your first privilege you get your first ability to commit you can merge
PRs or you can't you need to escalate where you don't but contributor ladders are also useful for
reasons we can apply organizational threat awareness to the contributor ladder as you grow a maturity
and understanding of a project and or its foundation again depending on size and scope you should
have an organizational threat model and be able to apply it more and more and more as you become more
senior effectively what I'm proposing is as you grow in the technical admin and you move up the
ladder you grow on a threat awareness and the defense of that threat you're not just aware that
something could be happening you're aware of what to do about it and every time you move up the
ladder that means that people will start coming to you level appropriate to help resolve things that
they don't understand and they don't need to know how to articulate we also need to start writing
things down better even if we don't know how to gather the signals yet and we need to write it
down as consistently as possible so that if we realize that we don't have the granularity the
data we need we can search through it and try again we understand where the gaps are that we've
missed and we try again we understand that organizational memory and turn as a risk and organizations
that don't have documentation and I don't just to mean onboarding documentation if you have a
moderator or moderator team that has a lot of churn because of burnout somehow the people who step in
need to acquire this threat awareness and the social awareness of the ecosystem they've just walked
into because organizational amnesia is its own kind of risk if people don't know what to do and
nothing's written down and they don't know how to frame it every time there's a turnover that
situation will repeat and all of this also points to the fact that anything can be weaponized
in social spaces and this is especially true if the cost to the attacker is low and the burden on
the target is high we see this with fraudulent requests like fraudulent GDPR requests and so forth
where your burden to respond as a product organization is high by definition but it is very low
energy to just spam those out right and this all matters because having these pivots also means
that we can have our moderation teams know how to distinguish and as part of that discernment
learn to help protect the mental health of their communities as a strategy for protecting those
communities this is also true for mentors when moderators and mentors and more senior people in the
ecosystem introduce responsible friction into the system they are protecting the burnout level of
their contributor pools in a healthy way and making it harder to move too quickly through the
ecosystem we can help this by normalizing conflict in our ecosystem because if we're so conflict
diverse we don't know how to have a disagreement without it blowing up that blow up is also in a
tech vector so by having a healthier and more grounded community we both have a more thriving
ecosystem and can defend better and annihilate better against that type of exploitation it's also
important that moderator teams understand that moderation is about boundaries not punishment a moderation
team can stall out if they think about moderation as punishment because if they do they need to know
if the person means it if they intended if they're a threat or not or an actor or not and you're
not going to know that you're just going to need to know if you have a boundary that's been violated
that you need to defend in some way and really quickly I would like to remind everyone to thank
your moderators and all of your teams and this isn't just for online social media though please
also thank your Fediverse and so forth moderators as well but also your contributor ecosystems
forget hug because all those spaces all those social code spaces have moderators
we'll be back into it one of the moderators ask themselves is this behavior compatible with
community health that is how you know you need to take an action of some kind and that is how you
avoid the stall out of trying to figure out if something that will only be revealed over a month or
a year is a different shape of problem than you think there's also a parallel to responsible
friction and moderation in code bases you don't want someone to take on 50 projects at once and
contribute hundreds of peers at once just because they're really excited and burn themselves out
but in moderation spaces you also need to help steward this healthy balanced reality and it's
the moderators who act more like social workers for the community and they need the equipment they're in
to be able to respond to these types of spaces it's not just fellow technologists who are tech
oriented who should be moderators it's also people who understand the socialization between people
and how to help them remain connected who would need to be stewards of community in this way
and so before I just go to the last slide here moderation teams you need to approach your
situations without judgment which is really hard especially when you're not sure what's going on
or someone's on their second third or fourth incident again depending on your code of conduct
and how that's outlined and it can be very hard to not feel like you want to judge a situation
or an individual in that situation but in order to be able to do this in order to frame
moderation as boundary protection and that punishment you do need to make sure you're proceeding
without judgment and with that all said I just wanted to make a nice recapable slide we need to
have organizational threat awareness this is true for foundations or for projects that are
sufficiently large that they have their own ecosystem around them and we need to center
the help of development of our maintainers and this is both for their benefit and for community
and occupation benefit we need to normalize conflict so that people can disagree without it
becoming amplified and it doesn't become its own exploit and we need to understand and address
the nature of protective measures that may themselves also become weaponized this doesn't mean we
can't do anything it just needs means that we need to be aware so we can proceed thoughtfully
we also have to have standardized documentation for social risk and not just code
it's not only about the code if we understand that vulnerabilities can be human for example again
XEU tells we know that the human was the exploit not the code although eventually it was used to
make a code that was an exploit right and this means that everybody needs to work together
the security specialists are the ones who understand and can do risk analysis throughout
modeling of usability testing and build the guidance that everybody else can use in act on
and since we're running a little slate on time I'm just going to go right to the point
so as I look through all the data and I have been trying to pivot and help people understand
the guidance I've spun up a working group because when you notice that there's a problem we don't
solve it out on our own we work with others and in this case we have a working group about
securing open source communities and the idea is to have some researchers in the space and
social workers in the space and then a bunch of foundations or leadership or moderators in the space
talking together about the problems and therefore the patterns and those problems so that we
can solve them together and build the guidance that does not currently exist phase one of two is
going to be the communication standards this is how we can get everybody used to talking to each other
and a way that they don't compromise information just by steering that information for collective
benefit and then we are going to use that skill to do the threat modeling and a usability
testing of codes of conduct governance models and so forth to produce guidance on what to do next.
We do have a session right after this talk we're going to be in building F somewhere that I can
find the existing group will be being there if you want to ask me questions you can also find us there
and with that I thank you all for your time feel free to come to our birds of the feather tomorrow morning.