All right, can you guys hear me?
All right, so let's get started.
Please, we have only a couple of minutes
because we are going to present here with Kevin
on contribution processes for MariaDB
and for Postgres.
So I'm going to try to be really quick
and then give the floor to Kevin here.
Okay, so let's get going.
I want to walk you through the MariaDB
server contribution process here
and I want to show you a life example.
So first of all, why?
Basically contributing to database code
basis is hard stuff.
I remember my first contribution to the code base
of the MariaDB server took three months to review.
So it's not trivial.
And that's why I want to show you some support
and I want to show you how it's done basically
so that it's not as intimidating as it might be.
Right, so first of all, let's get the facts.
It's a proper, the MariaDB server code
basis is a proper GitHub repository.
It has some extensions.
It has a, like, contributors license agreement
but which checks your pull request
and then there is a build boat
which also tests for regressions, whatever you submit.
And you also need a geera for larger submissions.
This is basically to describe what you are trying to do
in a better way than just a commit message.
Because, well, as I said, it's complex stuff
so it needs better explanation sometimes.
So how to pick a task to work on.
Basically the best advice that I can give you
is pick something that is important to you.
When people work on scratching their own
each is so to say.
It's, they are most productive
and they feel more accomplished when they achieve something.
If you are wondering what to do,
there is a list of beginner-friendly tasks
in our contributing document.
So take a look at those.
They can get to start.
The interesting things there.
Another way of finding contributions
is to basically engage online with other community members
or check geera for open issues that are appealing.
And then, last but not least,
the MariaDB project participates in the Google
summer of cold program.
So that's huge incentive for people working
on open source contributions.
So, right.
This is the contribution process in our nutshell.
So basically you clone the MISK
or MariaDB server repository.
You work on your feature.
And then you submit the pull request from your branch.
And then you make sure that the built pass is okay.
All the regression tests are good.
Then you get a review.
So basically you get a preliminary review by your
truly here.
I try to do that as fast as possible
when the PR appears.
And then we get a final review by an actual developer
owning the cold base.
And then make sure to have the final review
or also push your change because sometimes
they just approve it and they don't merge the PR,
which is a problem, right.
Okay.
And for the case study that I promised,
I probably it's not very visible,
but it is ten lines of actual cold.
Not a huge contribution, really.
But it is something tangible.
It was, as I said, based on a gire,
which describes what was the problem and what
was the fix for it.
Then it also got great test coverage.
All the regression tests were fine and everything.
And then the CLA was signed.
We got the proper treatment on our end properly
There was an active conversation.
I don't know if you see that,
but there is 32 messages going back and forth
for the stands and lines of cold.
So I'm really impressed by the engagement
that the developer showed in that particular case.
There were also several iterations,
several versions of these things was
published and presented by the contributor.
And finally, finally, it got merged
by a very senior developer in the MariaDB community, apparently.
So yeah, it's not hard.
You just need to follow the steps
and you get your name into the contributes list, basically.
So take away a few free to ask me
and communicate thoroughly, communicate often,
and be responsive to request from reviewers
and be nice and don't panic.
So those are my contact data
if you want to talk to me please do.
And with that, I guess I will give the floor to Karen.
And my audible.
Thank you.
Okay, cool.
So thanks, Georgie, for the OE for the MariaDB code base.
So now I'm going to give a similar insight
but into the postgres code like the contribution process.
I'm going to get from the perspective of someone
who's done it just a few months ago for the first time.
I'm relatively early in my career.
And I just want to talk about the postgres contribution process,
what my personal experience was,
and some takeaways for how it works overall,
and maybe you guys can also do it in the future.
My name is Kevin.
So to begin with, I can talk about in how my interest
in postgres began because it was not like a typical way.
So my first job out of college in 2023
was a company called PRDB.
PRDB is an open source ETL tool,
as far as CDC tool,
that moves data from postgres to multiple destinations,
including clickhouse.
PRDB uses logical replication,
which is the postgres feature to read changes
as they happen on postgres.
And I think Rohit and like the plan scale
folks talked about logical application detail earlier.
It is a pretty complex and a pretty intricate feature,
and it's mostly undocumented.
So you end up needing to read postgres code a lot
to make the tool extable and work for all cases.
PRDB was acquired by clickhouse,
which is why I end up here.
And I continue doing the same work as part of clickpipes.
So now I focus on going data from postgres
into clickhouse as reliably as possible.
So let's talk about the postgres code base.
So postgres is a 30 plus year old C code base,
and because of C and C's fairly feature light,
you need to build a lot on top of it
to have a clean and structured code base.
So postgres has its own custom memory subsystem,
has a lot of dynamic dispatch,
has a lot of macro use just to make the code base make sense.
A reading postgres code is not trivial.
It can be very overwhelming.
There's a ton of files each file as you know,
4,000, 5000 lines of code.
So what I did was to isolate an area that I wanted to focus on
as logical application, and you know,
focus on those files, those functions,
you know everything else,
and figure out the area that I wanted to focus on.
One great thing about postgres is postgres
has great commit messages.
So every line of code in postgres has a good commit message
where you can see why that change was made,
and it links back to the mailing list
so the mailing list will get into later.
But like you can read the commit message
and then you can read the links in the commit message
to even get even more context out of it.
So that's how the postgres code base can be very readable
just by the get history.
So until now, like for the past couple of years,
my sort of relationship with the postgres code base
is more of a passive one.
I was the guy who just read the code,
you know figured out customer issues,
like helped things out at the mailing list,
but I wasn't really focused on contributing the postgres myself.
And the reason that change was kind of random.
So what happened was I was reading through postgres docs
and I found a setting called Scram iterations.
This setting, all it does is it just controls a number of times
the password for a user is hashed when creating or authenticating.
It had a maximum value, that's really high maximum value.
So the sort of intrusive thought that entered my head was
what happens when he said this to the maximum value.
I just want to virtual machine
and it stayed running for a month.
Like I left it running and then I realized it just running forever.
So I realized it maybe there's a bug.
So this is the code that postgres had
to actually do this hashing thing.
And maybe a few of you can spot the issue immediately here
where that was causing an infinite loop.
The problem was that like the,
basically if you say to the maximum value of I and then increment I,
it becomes a negative value because of integer overflow.
So because of that, the lesson that equal to sign in the full loop,
it will hit the maximum value, it will still compare through.
Then it'll increment it to a negative value and then loop would never exit.
So the fix for this was actually fairly simple.
You just change the loop to not do that.
So this was a story of how I accidentally found a bug in postgres.
And the fix was like relatively simple.
It is a bug that almost nobody would hit.
But you know, it was something that I had found.
But again, that was half the battle.
Like I made the fix but I actually contribute the fix upstream.
And this is where the postgres, you know,
developer experience comes in a picture.
So unlike RADB, it is not a standard GitHub repository.
Postgres has its own Git repository that is hosted by postgres itself.
And you know, it's the UI is quite different from GitHub.
And the way they are no PRs.
So there's no issue tracker.
There's no PRs.
This pretty much nothing for postgres.
The way you actually commit code is you create a branch locally.
You make your change locally.
And then you use the git format patch command to make a patch.
And that patch is very attached as an email.
So you actually send an email to the mailing list in postgres.
And that email contains, you know, your entire patch.
Even if it's like a thousand lines of code, it will be a single file.
As your attachment to the email you send to the mailing list.
Basically, like an email address that contains like hundreds of people that are in the postgres community.
So yeah, like the commands I run in this translate into the patch file that you know talks about what files my patch changed.
And you know what the changes are.
And this is what I submit as part of my email to fix a change.
So postgres is called commit fest as well.
So this is the postus equivalent of like a code spirit or a review cycle where a bunch of people submit the changes to commit first.
You know, they have reviewers.
They have a version of CI.
You know, people reject PRs.
They approve it.
It gets merged.
Sometimes it gets pushed to the next commit first.
So I think each postgres has like five or six commit first.
And they happen every couple of months.
And this is where most of the postgres review happens because
Postgres doesn't have a single company behind it.
It's all a bunch of people working on like this.
Pat time to review code changes.
So it's kind of decentralized and that's where it's commit first process.
So for me again, like the fix that I had made.
The fix itself was like a few minutes of work.
But it took me a lot more time to figure out how to draft that very first email.
And how to actually attach a patch to Postgres.
So you know, I, that was a mail I sent to Postgres after like some thinking.
And after some like comments and you know,
They're deciding whether it needs tests or not.
In the end, the change was merged.
It was not merged by me.
So I have no commit access to the Postgres report.
Someone merged it on my behalf and I was attributed as author.
So this is kind of how the Postgres,
you know, contribution process works.
Yeah.
So again, this was like a first book I had found.
And around the same time I had made this first commit.
There was a much bigger issue that some customers of PDB were facing.
Where they were running into an issue where the replication slot creation.
So replication slot is basically the Postgres.
The thing that PDB connects to the Postgres to read the changes from Postgres.
The command to create the slot would hang.
And it would hang in a way where you couldn't stop it.
Like even if you tried to control C or send a terminate command to the square E.
It would just get stuck perpetually.
You couldn't really stop it.
And what some customers had to do was to entirely restart the database,
which if it's Postgres and you're asking a customer to restart their Postgres,
it's not a really good look.
So initial sort of thought because we couldn't figure out what the issue was.
Was there an issue with their managed service?
Was there an issue with RDS or GCP or so and so forth?
And what was a smoking gun for me was, you know,
a customer actually sent either S trace output.
So in this is calls from the incident that is having the issue.
And that, you know, helped me track it down.
So this is like a pretty intricate issue,
which I did write a blog post on if you're more interested in details.
But what was actually the issue was,
so when creating a replication slot,
there is a step that requires waiting for older transactions
to complete.
This code doesn't function properly on read replica.
So on the primary Postgres instance,
it works just fine.
But on the read replica or hot standby,
a bit of this code doesn't function properly.
So it doesn't wait on a transaction,
but it still thinks a transaction is running.
So it just, you know,
continues the loop indefinitely because it's not waiting.
It's just checking and then it keeps failing.
The problem with this loop is,
if this loop has you never checks for interrupts.
Like, even if you signal this back end to,
you know, terminate or cancel or whatever,
because in read replica,
this code doesn't function the expected way.
It would never receive the signal and so until this loop
exits due to a transaction completing,
the process would become unkillable.
So this solution was not like simple,
like isolated fix.
So I submitted a patch for this,
the immediate fix, which is to just let a customer
stop the query, not let it run forever,
which was to just allow this lot creation process
to be interrupted by signals,
even on read replica,
in a respective of, you know, what was happening.
This was committed to the Postgres code base,
so it is in now,
most Postgres minor versions.
There was a follow-up patch by a different member of the community,
so I'm not mean to highlight this way,
so the highlight the fact that we are stuck in this state,
and this patch,
unfortunately, has not been merged yet,
so even though it like a pretty small change
which is highlight the thing,
it was not merged right now.
A long term fix is to just not need this loop at all
and to have read replicas,
you know, just wait for transaction the more efficient manner.
This, and for setting is a lot more thought,
a lot more expertise from the folks who know Postgres,
so it needs to be figured out and then committed.
This is definitely not something that I can do right now.
So the conclusion for me,
like, like this time last year,
I had no idea how to contribute to Postgres,
and you know, I figured it out over time,
but the weird part for me was like the hardest part of all this
was not necessarily the code itself.
It was just convincing myself that, you know,
I had something worth saying,
I had not, like it is not something I was hallucinating,
it's actually something that I found out was a bug
and sending that very first email to the mailing list saying,
I found this issue.
This is true of open source,
in general, even if you find an issue,
it may actually not be an issue,
or even if it is sometimes a contribution
to make a rejected or ignored.
But despite all that, like regardless of that,
like if you have something to contribute to any open source
whatever, it's Postgres or MariaDB or Clickhouse or whatever,
even if it's a bug reported,
Docs update, like just send it,
and I feel that open source projects need more eyes
on them and not feel it.
Thank you.