Thank you for having me today.
This is my first time at the big stage in Humberu.
I've been here many years.
I love this conference.
It's best to have a source conference in the world.
But first time here.
So I thought given this topic is potentially contentious.
I would get a friend of mine, hopefully make me feel a little bit better.
I said, I'm the screenshot of when this got accepted.
And I said, oh, fuck.
So he could actually make me just write kind.
He thought nice things about the title.
And then, unfortunately let me down.
So hopefully I am not, in fact, fucked.
But if I am, hopefully it is entertaining for all of you to watch me
as we turn the UK down your arse.
Anyway, so, some important framing.
We're going to talk about some stuff today, which is pretty fresh.
Literally, some people who have been involved in the story are
possibly in the room.
And I've seen people at fuzz them.
So who already has an awareness of what happened with Ruby gems?
Even a vague one, right?
And who has strong opinions about me who was in the wrong and who was in the right?
Trick question.
But yeah, okay, so a few of us.
So what I asked us to bear in mind, right?
This is not a trial.
It doesn't really help us to go and point fingers, particularly
at individuals, actually, because this is mainly a story about
groups of people and communities and open source foundations
and open source maintainers and companies and all that
interesting stuff, right?
And I actually think this stuff gets
particularly when I'm studying on a stage and people
want to be in the room, it's a bit mean.
If we start pointing fingers at individuals, so I'm just not going to do that.
I'm going to try and avoid naming anyone except myself,
because I'm allowed to be self-deplicating because I'm British.
So the next thing, again, I want us to just encourage anyone who
right now is like this group with this person completely
fucked this up and this group with this person did everything right.
Like, that's a bit of a red flag, right?
You know, many of us in the room, I'm sure, are engineers, right?
And we work with software and complex systems fail in complex ways.
Right?
So if you feel absolutely certain about something,
that means either you're probably going to be missing something
or maybe have a bias.
That's okay.
No shame, no shade, whatever.
Right?
But just let's try and come into this with an open mind, right?
Again, another important frame I think for all this is
incentives of everything, right?
Like, very rarely in the world do we actually have,
particularly in things like open source and in workplaces,
and not Netflix shows or whatever.
Like, people who are just unbelievably evil
and cackle away is they deliberately do the thing
to maximally fuck over as many people as possible.
Like, people are responding to the incentives
that are putting in front of them, right?
People don't go into things generally with bad intentions.
And generally the people who we look at in these situations
who we think, maybe have bad intentions,
just have a really bad incentive such a set in front of them.
Right? So I'm asking you to look at things through that lens.
And we're going to try this being a lot of,
I guess, the word drama has come up again again
when people have been talking about this, right?
Sometimes, like, favorably sometimes used to critique
one side or another or whatever,
but I feel like we thankfully have moved past the drama stage
and we can be in the learning stage.
So let's do that.
This talk is about re-enopen source software
and specifically infrastructure projects,
which I'll talk a little bit later on why
this is a particularly important topic for me, specifically.
But these projects underpin an enormous amount of the internet, right?
Like, for most, even kind of commercial software developers
who are not here, who care nothing about open source,
who probably don't even really understand open source.
The amount of open source software that has to work
and go right and be lined up
and perfectly orient itself for that person to do their day job.
And increasingly their day job far removed from technology
is pretty breathtaking when you think about it.
Right? It's arguably open source software
is one of the best achievements we've made as mankind.
And I love conferences like this
because I think it really highlights the kind of beauty
of what we're doing here, right?
We're creating this worldwide system of software,
which is incredibly powerful and many people rely on
at empowers so many other fields, right?
But because it's invisible when it's working
and because people don't think about it,
often it just fades into the background.
And that success that comes from making things invisible
and transparent and just smoothly flowing
makes it even more dramatic or so,
even more dramatic when things fail.
When things fail, they fail loudly, they fail globally
and the blast radius on important projects is never small.
So Ruby gems is one of those systems
and that's what the story today is going to be about.
We'll talk about heroes, we'll talk about villains,
we'll just talk about systems on the stress
and what we can learn.
So who am I? Why am I talking about this?
And what can you maybe learn from me?
Firstly, I think that's legitimate to ask
particularly in something like this story
because I think a lot of the people who maybe have
strong opinions like when you understand a bit about their background
it's maybe clear why they have the view they have.
So I'm the project leader of Humbru,
a Mac package manager,
runs on Linux and stuff as well.
So I worked on that since 2009,
so that would be 17 years this year,
which is slightly terrifying.
I think what that does is it gives me a little bit of context
about what it's like to run a package manager,
what it's like to work on these open source projects
for a long period of time.
But let's see, later,
I'm not super involved with this story beyond that, right?
I was principal engineer at GitHub for a while
that told me a fair bit about open source,
it told me a fair bit about Ruby
because that's what GitHub is primarily built in
and scale security, libraries, et cetera.
I think most importantly actually for the story,
I'm a Rubyist.
I first wrote some Ruby in 2005.
I've probably written Ruby more than any other language
in about 2009.
And I deeply love it.
I like the ecosystem.
I like many of the people I have friends.
I've met because of Ruby.
I have jobs that I've built
because of my work on Ruby.
And I have dependent on it
for most of my career.
Finally, and somewhat least importantly,
like I have a day job.
Like open source is not my life.
I'm a CTO, a small training management software company
in Scotland.
But as a result of that,
I've had to do an awful lot of work
around growing as a leader
and trying to understand how systems and cultures
and incentives are set up and what happens.
I'm not a Ruby Gemma's maintainer.
I never happen.
I have simply with Bundler.
I have no affiliation with Ruby Central.
I've never been involved with them.
I've never given them any money.
I've never done any work with them or anything like that.
In the situation that I was asked to mediate.
So, some people will talk about later on.
Like the way Humbru does governance
was kind of cited relatively early on in this situation.
And it's someone who's kind of very involved with Humbru and our governance process.
I was asked to kind of come in and see,
like, well, how can they supply to Ruby Gemma not?
But, unfortunately, my mediation did not manage to stop
happening what happened.
It's interesting because pretty much this whole story
takes place between September, October 2025
we kind of have a bit of stability now.
I think some of the kind of strongest emotions
were kind of past that stage, which probably helps us
to look back a little bit.
So, we're talking about weeks for most of these events
rather than years.
Although the context set beforehand and the stuff
will learn afterwards will go back and forward.
I helped Jam.co up.
You don't need to know what that is.
If you don't already.
I help them kind of sell.
They're going in this process.
I learned stuff on the way.
So, okay, who knows what Ruby Gemma is?
Hand up.
Okay, most people saw it.
I keep this brief.
Ruby's package manager is called Ruby Gemms.
It's technically you could use another package manager,
but that's the main default that the vast majority of people use.
It was actually founded in 2004
after Ruby had been round for a fair wee while.
Like nowadays, often the software kind of comes with
version locking and package management along
with the language right out of the door, but that was not the case.
Ruby Gemms and Ruby.
There's millions of applications that will learn Ruby Gemms.
And as a result, we're not talking about some niche tool
used by a few people.
We're talking about kind of critical infrastructure on the internet.
Which brings up things like software supply chains
and all these other kind of big growing up sounding words.
It's not side project as well.
Many open source projects are essentially run by one person
in their evenings and weekends.
They never get any money for it.
They never spend any money on it.
And it's just like a fun little hobby.
Like Ruby Gemms is not in that category.
Ruby Gemms is much more critical.
So money.
Money is messy, particularly when you evolve it with open source.
Who feels a little bit uncomfortable to have not money sometimes?
Quite a lot of people.
Like open source money is even more potentially uncomfortable.
Because well, something is cost money.
So where does cost money?
Storage cost money.
Buy with the cost money.
On cool, if you want to pay someone, well, sorry, if you want someone
to be working up at three o'clock in the morning on a regular basis
when it's the breaks, then generally you need to provide a bit more
incentive to that than just peer pressure and community good will.
Instance kind of money.
If you've got years of work, you can have it undone in a few
hours when you have some critical incident.
And it's why organizations behave differently on stress.
Because depending on where the money is, that tends to focus attention
even more closely when instance is going on.
But not everything costs money.
Let's compare with the purpose of this project.
I've worked on for a while with Hombrew, right?
So this is not to say, Hombrew is better or worse.
It's just a comparison because I think we're two projects.
Both written Ruby, but we have a different approach to these things.
So Hombrew's first thing, when you download Hombrew's binary packages,
we do not pay any money for that.
Because we were originally hosted by sourceforge,
then Bintray, and then now it is its GitHub.
So the trade off for that is that Hombrew is not independent
as something that Ruby gems is.
Like Ruby gems is not relying on any single vendor.
Like they, of course, there are servers on the AWS,
but they could move relatively with big finger quotes easily
to another provider.
I'm probably certainly a lot easier than Hombrew could.
So in some ways, we're not even independent.
We're somewhat dependent on retaining in the good graces of GitHub, right?
I used to work there.
I still know a lot of people there.
So I'm not as worried about that as I might be something else.
But again, that may or may not be a decision that makes sense for your project,
or for Ruby gems.
So that's quickly scanned through the timeline of what happened
on the website.
So Ruby central, the organization that's kind of often cited with this discussion,
was found that originally 2001, by a couple of people in the Ruby community,
who want to provide a permanent nonprofit for running Ruby community events
and handling sponsorship and logistics.
A few of the same people were involved with releasing Ruby gems
in March 2004.
And then we have to go forward about 11 years before we kind of reach the next
group that's kind of relevant this way, which is Ruby together.
So they were actually announced by Ruby central in 2015.
And they described their purpose as funding on-call rotations, maintenance work
and improvements to shared Ruby infrastructure.
Usually it bundlers, Ruby gems and Ruby gems.org,
which had, in their words, historically been done by volunteers.
In 2022, these organizations merged and became a single nonprofit,
described the motivation as reducing duplicated non-profit overhead
and unifying community events under one roof.
So this brings us to basically the more contentious events.
Hands up of you are following this live on social media when it was going on in 2025.
Yeah, a few people here.
So September 2025, everything kind of goes down within a few weeks.
And again, I think there's some interesting context when we talk about
the setup for this taking almost 15 years and then within a few weeks
things radically changed.
So one of the first things I think that happened was control of the main
GitHub organization, which was used by Ruby gems.
The project was changed in a way that restricted or removed access
for some of the people who were maintaining Ruby gems.
And this came out of some folks who were involved with Ruby
central that also involved reading the GitHub organization from Ruby
gems to Ruby central.
And essentially, there was, as far as various people involved are concerned,
some people say this was very long-time coming.
And there should have been expected.
And other folks felt completely blindsided by what happened.
There wasn't certainly any public notice of this.
People had some sort of behind-the-scenes warnings that something might be happening.
But certainly a bunch of people were, as I said, essentially blindsided by what happens.
And the problem with this is when you make decisions quickly and when you deliver
them somewhat abruptly, that always tends to feel hostile, right?
It feels, it can feel malicious, it can feel difficult, it can feel targeted,
even if it wasn't necessarily intended that way.
And particularly an open source when we assume so much stuff is going to be done in the public
when there's no public communication at all, then it can make this very hard.
There was also essentially no governance process.
So I mentioned before, Humber's governance process will talk a little bit about that later.
But essentially at the time when this was all going on, there wasn't a documented process
about how people get added and removed who controls what, which non-profit
is responsible for what, what taking money means as a contractor or an employee or whatever
with this organization.
And the problem is when you don't have a written governance structure,
then whoever holds a power ends up filling the gap and getting to make the decisions.
So after the initial kind of shock, we had our attempt at recovery, right?
It looked like things were going okay to begin with, so we got to mid September.
There was multiple possible features. It looked like we could get everything back.
Everyone could get back in the same page. There might be some hurt feelings, but we would be okay.
There was a governance PR proposed on September the 14th.
You can go and see the URL or show on the screen in a few slides,
where you can see the kind of conversation that was going back and forth.
This is how I kind of ended up getting pulled in as well.
Because it was based off Humber's governance and then I was, I kind of offered to give some input there
and then got pulled into mediation as well.
And then shortly afterwards, the access restored for most of these people, right?
So if you get to the PR, you can kind of see how this went on.
And like the sort of TikTok of what was going on in the event, right?
So then by September the 18th, we get to a point where I've offered to kind of step in and be
between both sides. There's a lot of private conversations going on, a lot of private context going on.
And in public, it looks good. It looks like we're both sides of discussing governance.
And then we get to the point that I six gets removed again for a bunch of people, right?
And this is one thing that's got a little bit mess here because some folks who are there
access removed, then claimed that it was a mistake that their access had been removed
and then others took offense and essentially things get very messy very quickly, right?
And then trust just gets destroyed and never really rebuilt.
This is when we start to get security concerns invoked as a justification as well.
There was mention of supply size security, which in recent history there had been
MPM's supply size security attacks. So Ruby gems having a relatively similar trust model like that was
concerning to people being involved.
It will also concentrate power very quickly and make clear who had power to do what and who
was citing these concerns and using them to make what decisions.
So by the time we get to the late September, we have another inflection point where it becomes
public on the 28th of September, the root level infrastructure access still existed outside
of the new control structure. So the people who were maintainers and had either been removed
from the project or had quit the project in protest at the changes.
Some of them still had AWS access and AWS root access even more severely, right?
So again, this back and forth about exactly who had more access and when, why and what this meant.
But essentially, regardless, this is about situation where you have a situation where the
theoretical control of the project and the actual control of the project are diverging pretty far.
Right? And then, essentially, there was a lot of back and forth, but eventually we reached the end state where
almost a month later in October of 17th, Ruby gems and public moved to the Ruby core team.
So essentially, Ruby central who had taken ownership of these projects then gave ownership elsewhere
and then the people who had been existing maintainers of Ruby gems declared that they were happy with that
and they were going to allow things to move on.
So you can see, you can read the root access event. This has got kind of Ruby central's take of like what went down
with the AWS access and stuff like that. This was posted on the 9th October.
And then while this is all going on, it gets to the point where there's enough drama that various plots
of the tech press start reporting on it and people are getting quotes, blog posts, social media,
press coverage essentially take over. We're not having really any discussions on PRs anymore.
This is all happening out in the open.
And then the narrative stops being controlled by the people who are just involved and things become a little bit reactive.
People start raising privately a lot of people in the Ruby community. People's confidence stops like, what does this mean for Ruby gems?
It's Ruby gems still going to be running anymore.
And worst of all lawsuits begin, like we have, again, publicly on the record both sides have talked about how they've engaged in legal action with the other.
I have no insider knowledge as to what the statuses of sidewalk suits, but I think it's relatively predictable that once that happens,
people are no longer interested or able to make good and be friends with people they are in lawsuits with at that point.
So what are the consequences of what happened here? Well, talk failed pretty quickly, right?
People try to voice what was going on and how they were feeling and how they could improve things.
But then when people get kicked out of the project, then all that just becomes public discussion instead of private discussion.
So, a bunch of maintainers exit the project.
Some people will remove against their will and then some people will remove the protest.
So you get to the point where the majority of people who are working on Ruby gems in the space of two months essentially have left.
And got them done their own thing.
So, doing their own thing, what does that mean while there was a single gem co-op, which some of the maintainers went off and decided to build.
So they essentially built their own alternative to Ruby gems both centralised service and they have forks of various other projects and stuff as well.
So this was not an overnight replacement, but it was spun up relatively quickly.
And it built essentially a separate kind of site ecosystem which changed the power landscape, right?
So, forks don't need to necessarily win or get more users to matter.
And in this case, I think this kind of sent a message about how things could be done differently.
And it also made clear that that group of people were not interested in re-entering the fold again.
So, another interesting kind of consequence of this is I guess what I would call professional open source.
So, what I mean by that is we have a bunch of people in this room.
I would guess probably the majority of people in the room.
How many people are paid to do open source work primarily at their job?
Okay.
I guess if you look around the room, everyone, that is the minority, right?
And I would say in the open source community in general, it is in the minority, right?
So, when you have people whose full-time solid work is focused primarily on open source,
and when they have bosses and leaders and HR teams and their mortgage depends on them,
making this people happy, then incentives are radically different, right?
And it introduced different incentives to what exists before when you have volunteers working in their free of time.
And also, you have the blurring between the two where you have people taking contracts and then paid work on open source.
They might not be an employee, but they have agreed to some sort of transactional relationship and exchange for money.
And that makes things hard.
And this is even more tricky when you have a project like RubyGems, which is already hard,
because you are going to critical open source infrastructure.
And it requires many skills outside of white and code, right?
Which is sometimes the easiest skill to be able to get from the community in open source.
What this means is that volunteers have their limits and people are now, right?
So what does this mean for careers, right?
So a bunch of the people involved in this story had at some point been paid either full-time or part-time,
or full-time employees of the potential RubyGems together or whatever it may be.
Well, my potentially contentious take is open source is not a career, right?
And by that, I don't mean those of you who said it's your full-time job to work on open source.
I don't mean that you somehow don't have a job in you now.
He's, and don't really know where you are or what you're doing.
Like, obviously, some people do have that, but most of us don't.
Open source can be part of your career.
And even though it's folks, I imagine who are working full-time open source.
I would imagine your day-to-day looks pretty different to how it did if it was just your evening.
So weekends, right?
It's a very different way of working.
And I don't think we help our community by making out the way you worked before you received any money.
Can just be paid a Bay Area salary without any change to it.
So I think we also need to plan for our exit as individuals and as organizations and exit of funding and all these types of things.
Right?
We need to plan for the transitions that are going to happen even when they're deeply uncomfortable.
Because ultimately, one size does not fit all.
Every project is going to happen flow and change the years.
And context is everything.
So what can we learn from all this?
Well, as I said at the beginning, it's not about blame, right?
We need to not blame because it just simplifies and polarizes and it prevents learning.
If you're 100% sure about who's right and who's wrong, then that's going to feel good maybe for you,
but it's not going to explain very much.
Governance in open source is really boring until it's not.
And at that time, it's probably too late to introduce it.
Okay?
Maybe money, insurance, the equation makes things better.
It certainly makes things a lot more complicated.
I think really carefully about how and when and why you introduce money is different such project.
That's all not focused on who was right and who was wrong, not who messed up.
But instead just try and ask that questions.
Let's ask about what broke and why it broke and not who did what.
And then we could learn something about governance for money.
If your project hasn't argued about governance for money yet, it probably will one day.
Be prepared and try and do this stuff before it becomes a problem and that's the transfer of the lesson.
You've got questions, then you can speak to me outside.
You can email me or you can find how to constantly other ways on my website.
Thank you very much.