Hello, welcome everybody in the distribution day room for people who were in the preface talk,
you know what I was repeat for the newcomers. Here we go. First let's start with some
safety instruction in case of fire or whatever would happen. Nothing will happen. We have
the exit door at the back, first in one exit in the front or on the wings. I don't know that's
for the planes, sorry. For the rest, if you want to leave the room just with quietly,
if possible, because both the doors and the seat are producing a lot of noise and it can
disturb the audience. For the human section at the end, raise your hand, wait for the microphone
to show up so that everybody can listen and hear your question because it's also recorded in
the screen. So that will be correctly captured. And now I will let the stage two,
a couple of sceneries that I just told you before that I can call you Carlos, because we are all friends
here. And so the topic will be fixing CV and DBM almost everything you should know about it.
Cool, so hi guys, thanks. I feel the slamer is here. The first one, the stock is
mostly targeted for end users and people want to know a bit more about DBM and how we manage
vulnerabilities and if people are safe using DBM stable, yeah, like they are, but I'm going to go through
it today. If you guys are after something more specific about how we do that in DBM and like
the DBM specific things, I would recommend you to watch some while talk, unless you're a
debconf, which he goes in a lot of more details on our specific process. And the second disclaimer
I have to do, I'm not actually from the security team, so security team is a team inside DBM,
but I've worked with DBM, I've fixed CV is on DBM, so actually my last year thesis,
last year to graduate, based on a case study on how we do CV management on the DBM, that's why
I have this stock here today. So again, this slides are available, maybe everyone can see,
I'm not sure, on people.debn.org is slash tilde Charles, which is my nickname. Usually people
call me Charles, so feel free to. Today we are going to like go through some things. The first
it's going to be a bit of introduction. I want to present myself and I also want to know a bit
about you guys. Then we'll talk a bit about the DBM project, then CV is and how we deal with CV is in
DBM and then how we fix those CVs in DBM and we'll have some diagrams that hopefully will help you
to understand everything. And hopefully I will have five or ten minutes for questions.
So let's get into it. I'm Charles, I'm a computer engineer from Brazil and that explains
why the long lives. I don't like this code. Back in university, those two topics,
infos that I can free software were like my passion and see it. I did a lot of work with other
students who we created this in Brazil we call extension groups. Like it's made of
of students and we do a lot of lots of stuff. The free software group, this photo is from
said other really is party but I had to like put my identity shirt and go there talking about
DBM because yeah that's what I do. And also I'm a DBM contributor, mostly packaging, so
I help to maintain curl, I do maintain neomot and some other packages and some other team
maintain packages. But I also try to help on localization so getting DBM into Brazilian
Portuguese because if we want to be the universal operating system, we need to be available on
every language in the world. So I'm also a DBM developer and the first time I attended a
DBM conference which is our annual conference was 2022 in Kosovo and I'm there.
And today I work in embedded systems, companies which is called turrets and I do embedded stuff
there related to DBM but not exactly. So now it's time for questions for you. So if the answer is yes
you please raise your hand. Is this your first time hearing about DBM? Hopefully no one. Oh okay,
one. That's nice. I get to introduce DBM to another person so it's nice. Have you ever used
that in? Perfect. Have you ever installed that in so I would say so. Yeah. Okay. Do you know what
CVs are? Okay. Have you ever seen a CV? Okay so I will leave in. You know everything. So
okay so do you know how DBM fixes CVs and the process? That was the last question. If you all
guys knew it, so let's talk a bit about the DBM project first so we can like get a sense of what it is.
DBM project is a free software. It's like the operating system that we developed. It's a Gino Linux
based distribution basically and it's a free software in the sense that we have a social contract
that every contributor abide by when they are contributing to DBM and that specifically social
contract says DBM will remain 100% free so that's on our core and also we have some our
DBM free software guidelines that dictates what we consider free software and what can be distributed
and maintained by DBM in our mirrors. Usually we call it DFSG. It's a collaborative project so
we have developers from all around the world. I think for sure more than 1,000 people.
And as with 1,000 people from all over the world we need some sort of constitutions so things don't
get messy. But it's also a volunteer based and this constitution was developed in the 90s through
some discussions in my list actually lots of months of discussions on mailing list but it
worked out quite well. And also on our social contract we say that we don't hide our problems.
So almost everything that is transparent to the users you can like go to our mailing list where
discussions is happening. You can see what people are discussing on IRC channels and also our
infrastructure is like the rules to create them. So the puppeteer I will for the name of the other
our restrictions are public so you can see everything that's happening in that bin.
Our goal is to release the next table version always and take care of the current one but
our end goal is to always release this new stable version and currently it's book
Army. It was releasing 2023 which means that this year we will have a new stable release.
How do you get there? We freeze our development. Basically we have a development release which is called
a stable or seed. Our new patches get uploaded there. Oh I didn't tell you but I'm going to
lie a bit for you today because I don't want to make it very complicated. So don't trust me 100% but
our new patches go to a stable which is our development release and after they
pass some automated tests that we call auto-pkg tests and other stuff and after a few days they
migrate to our testing release. This testing release is going to be the one frozen and release
does the new stable release. Things happen quite a lot on those two. These are like the
stable and the testing release but after the release is done and everything is fine we can release
an inversion. We have the release of bookworm and previous we had the old stable release which was
blue eye and blister and so on and so forth. Well let's talk a bit about the stable because
it's frozen time in the sense that it's shipped with the upstream versions that were available
at the time of the freeze. So for bookworm that means upstream software from 2023 and we don't
want to introduce any big changes on the stable version because people rely on the behavior
in there so we don't want to break an API or something like that or change the behavior of
the curl software because people are writing scripts with it so we don't want to change a lot
of stuff there but we need to change eventually because it might have a severe bug that we
discovered only after the release. So we have this proposed updates mechanism where we are going to
queue up proposed updates and eventually they are going to be released as the new point release
of that in. Usually we use the proposed updates to fix bugs but there's also another need
which is the security stuff. For a severe bug a severe vulnerability sorry we don't have the
security feed. The security feed is enabled by default on all automated installation and user-based
installation. So everyone has access to it and as soon as the package hits it it's available to everyone
as long as you do an APT update and APT upgrade or you have the automatic updates enabled.
So everyone has access to the security feed by default and as quickly as possible but there are
some of them that I not that severe for that we use the proposed mechanism update that I've talked
before. And basically every two months we release a new point release so everything that is
queued on the proposed update to become the new point release. So let's briefly talk about the
CVE and what they are. Basically it's a bit of overloaded word but the CVE ideas like this global
identity fire that's unique for each vulnerability and it follows this format CVE ear and a unique
number for that year. The CVE itself contains more metadata so it has like the title which
is an issue the CVE published in the description and it is basically the main data source word
wide for a CVE management and security stuff. I think so to consider it's a crowd source effort
so a lot of people working on it they are mutable so you have to to consider that things can
change during the last time of that CVE so be careful to that and also sometimes it can contain
misleading information like we are not perfect made someone missed a variation that introduced
the vulnerability or something so it needs to be updated. How can I get a CVE because that's an
important question. You get a CVE contacting a CVE number of authority or CNA. I said look I found this
thing I think it's a vulnerability can you issue a CVE defer it. They work in a higher right cool system
so we have the top level route that is in my trip and we have some sub-SUNIA CNAs attached to it.
I have the route CNAs and then I have some more sub-SUNIAs attached to these ones. I have to say that
Debion is one sub-SUNIA so we have the power to assign CVE IDs but we only do it for Debion
and for specific things for Debion so if you find a vulnerability on I don't know if I
thought you shouldn't report to us you should report to probably there's an open source
CNA that's more general we should report to. But there are a few things that every CNA has a
poor CVE IDs that they can issue and they can grant this IDs as they see fit but they can be
disputed by other people so also be careful about that. Well we know about CVE is we know a bit of
Debion now let's see how Debion deals with CVE's. The main people that are going to take care of
this stuff is the security team currently they are the security teams composed by 10 people those are
the members they are delegated by our project leader and these are the members per se of the
the security team. But the security team has some contributors that gather around and like
try their stuff and help and they also have a assistance that can do this work for them. They have
many many tasks so let's break down a few of them. They reduce our security advisories which are
basically an advisory saying oh look in Debion stable we have this vulnerability it has this potential
impact this this version that we've really fixed it in Debion stable and you should update it
so much possible and there's a lot of data there like the CVE ID the DSA ID and more stuff that you can
follow links and discover things so the security team does that part of writing it. They also manage
embargoed vulnerabilities. When vulnerability is to spread or to dangerous for everyone it's kept
private until multiple parties can agree on a date to turn it public and so during this private
time we call it embargo time so only people that really need to know about it get to know about it
and people try to fix it as so as possible and release the fix before the date that it's going to be
made public. We also have the proposed updates that we might use to fix CVE minor CVE is so the
security team has to manage that too so which version fix what when it's going to be released and
all that. They also handle some special packages a few examples are those big patches that we
can like stay on a fixed version Linux is one of those we update to every stable release that
Greg makes. Firefox trauma where also one of those there are a few more but I'm not going to go
through all of them. They also have to handle the income males because there is a male that you can send
that only gets to the security team and they have to try it because they have received information
of vulnerabilities in there and they also have a private key so it can send encrypted mail to them
and only the members of the security team have access to the private key so only they can
read those emails and they have to manage this key and they also manage our poor ID of CVE
that we can issue and they manage the security tracker that is the most important thing I think for us.
I still found it quite crazy. Why the security tracker is important because it's the thing that
manages that do our vulnerability management. It's basically a gift repo of TXT files following some
default formats for us. They contain information about everything the CVE is on the
Adion, all the CVE is even the ones that don't affect the Adion and it's constantly updated through
the CVE list that Mitre makes available and so we get around ADCVS per day which we need to
try and see oh is this does it affect that end does it not should it fix should it not
and also they receive information for Mitre, these are that open all things and you can directly
mail them through this email address. We also generate a website with that information that you can see here
it's I think this is the CVE for a curl I'm not sure but you can have access at security-tracker.dev.org
and every vulnerability needs to be evaluated and we need to have a call to action on what to do with them
and if we need to to release a DSA obviously anyone can contribute to so you can try
exchange for for the security team they will review your work it can fix the vulnerabilities in
that and it can submit a marriage request for the package maintainer through the BTS or our
GitLab instance which is also and the security team has to make a decision if they are going to
release a DSA or not in the end. So first of these we have basically three options we can
apply the apply the fix and release the DSA this is usually the things that happen for our
severe vulnerabilities so the XE or everything that remote code execution we are going to manage it
and fix it before it's public and then on the day that it's going to make public we also make it
available on our security feed and send the DSA to inform everyone.
For the CVEs of vulnerabilities that are not that important like oh it's a local
a local vulnerability that has to have user interaction to become a problem we don't have time to deal
with everything so we document it we contact the maintainer opening a bug in the bug and
system say oh here this package has a vulnerability you should fix it.
Or sometimes it's just too complicated or involves a lot of stuff we are going to break a lot of
reverse dependencies we've introduced this fix and there is a mitigation available so
the system and the link and like mitigated we just document it and we don't do anything.
We do fix this vulnerability via backparting those changes because
the upstream they we see this information oh you have a vulnerability it's on this part of the code
they just they make a new commit fix it and release on the new version of their software.
For us it's a bit different because we have some other versions that are kind of frozen in time
when the stable release was released so we don't have the flexibility to just grab the new
code and inject there most of the time so we need to backpart those patches so see the difference
see how it fixes the cv and then make the change and release a new them release we are not changing
the upstream release we are just changing the them release so usually that shows you that you have
an updated stable package. The process is pretty straightforward like we we discover the cv
somehow someone infirm does we get from the cv list and then it needs to confirm in fact
in the five effects apply the patches if they don't apply correctly because it's the older version
it's not the same as the new version so the hooks don't apply you have to modify the patch
document this changes because you are changing you want to make sure that everyone knows
what to change and why you change that. Then someone ideally another debent developer or someone
from the security team or someone else will review this backparting changes you have to test if you
actually did fix the vulnerability because it's not just importing the patch to fix it and call it
a day you have to make sure that everything is fixed and then you can submit the fixed package
to them to make it as they won't own them also make sure that you watch for regulations
we try to do that as best as we can because you know we are not perfect sometimes it happens
and the process here for a severe vulnerability the security team is first informed
then it's going to evaluate the that cv so if it's a very severe cv they will fix it before
the embargo is is lift and they might contact the the maintainer via encrypted mail because we
we all have our gpg so we can do encrypted mail it's not always the case but they can contact the
maintainer too okay we are going to change that is it okay for you what do you think about those changes
do you want to help us and then they can have those those encrypted conversations and the security team
will upload to our FTP query to to be for the package to be built with the fix then the package is
built it's going to the security feed when it's time to to lift the embargo and they craft the
BSA and send the mail when the patches are available on the security feed. One thing to note
that all happens on security teams infrastructure it's not the same as our regular build these
so everything that happens in there only the security team has access so we can comply with the embargo
with powder problem for our severe vulnerability. For the the minor vulnerability the security team
just received those informations they evaluate the cv they say okay it's fine the fix is there but it's
it's a minus vulnerability we are not going to fix it ourselves so they don't do contact the
maintainer via the bug tracking system in the area so the maintainer can know that there is
vulnerability and they can fix it when the maintainer wants to fix it here we go through the
vulnerability see the fix try to apply on the package test everything okay it's fine
he's going to upload to our proposed updates kill it's going to get built and it's going to be available
our proposed updates feeds so those those who are like the main ways of doing this this stuff via
the proposed updates which is a regular process that we do to also fix another bug or via the
security infrastructure the SA everything else that that's available to everyone as someone it's
released so I think that's the the main idea I wanted to give you it's a bit difficult because 30 minutes
to talk all that it's way too short I will be here at falls day into 6 7 pm so if you guys want to
talk to me go ahead and do we have questions or comments a lot
you know since the state that the security team is 10 people and you sort of they're managing
the inbox and they're like they're watching the list of them with like the 18 new CVEs per day
so I mean this four ten people even with the two that maintain the security of WNC but this
house a full-time job of four ten people so it's this that we know that they are somehow
working like full-time for the deviant project or it's this that you know voluntary basically so
that is yeah I think it might be a mixed bag of things but I would say in general it's
probably volunteer based I I'm not sure I know I don't know they answer for that but yes I agree
that it should be a full-time job and someone should pay these people to do this because a lot of
companies rely on deviant a lot of servers were in deviant so it would be really nice to have that
support do you have complete separate built environment for all the security
build for all so you have for each platform support in deviant you have your own build machine
yeah I I'm almost certainly about that I'm not exactly sure if it's really really like that
for like X390 or like power PC I'm not exactly sure maybe they share something but I wouldn't think so
the question is can regular user speed CVs or maybe other companies that are used the base
deviant base to customize distribution what what is the first part of the question is
if any regular user of deviant can create the CV it's not that simple but if you think you find
found vulnerability you can mail the security things that look I think this is a vulnerability
can you help me probably they will forward you to the right place because probably it's not
the deviant CNA that has to issue the CV but yeah they they can help you
another question some vulnerabilities were not this ignored are you transparent why they
have been stuck like this yeah I mean like if the the vulnerability is not on the embargo you can
like check the deviant that the deviant that security or c-channel you can mail them you can like
take a look on the security tracker because they also do git commits and in the commit message
the their explanations to why the thing was treated or not or evaluated as a viewer or not
yeah it's it's public for embargoed stuff usually it goes in via encrypted mail these
shows that open wall so it might not be that transparent to us do you have more questions
hi so heavy from the almalinux build system team and I've been working on erata stuff
recently and I'd like to talk with you later if you don't mind cool cool yeah okay we think I'll
later so I think oh okay another one last question isn't it sometimes more risky to backport
and to just actually put the major version of some software project let's say stay as a project
like true yeah as I said there are some some software that does not completely follow this
in deviant so like Firefox the kernel from you a few libraries also pick up new versions like
I have my open system might be one of those but we do have a lot of automated tests going on
this so ideally we can try to catch up these regressions yeah I mean like it's not a perfect
sometimes we will make them steak but yeah I mean like it's a topic for another whole top about this
time is up so thank you everyone and I'll be here