OK, let's get started.
Hello, everyone.
I'm Eric Bravet, and this is Martin Eddini,
and we are working in a redhead.
And we are working on something which is called
complex.
Have you ever heard about complex raise-hand?
Perfect.
Another one.
Anyone?
OK, I will tell you what it is.
Complex is an open source, cloud native software
factory, or a built-factor if you want, and is
focused mainly on software subjects and security.
What we are doing in redhead, we are building software
in a complex, actually, a lot of software, for example,
operators, maybe an artifact, home charts,
RPM packages, but in this presentation,
we will be mostly interested in a container images.
Complex is really big, too big, to be handled by the single team,
and our team is specialized for a bone domain.
We are generating as bones, we are augmenting them,
and we are also delivering those as bone solutions
to the other teams that are committed to the security
and compliance, for example, for a vulnerability management.
In this presentation, we would like to show you that in redhead,
we are not treating as bones as some kind of mandatory
checklist, but we are trying to, let's say,
unlock their potential, we are looking for a way
how to make it better, how to make it more precise,
and we would like to introduce a new concept,
which is called contextual, as bone,
and tell you what it brings for vulnerability management.
So, let's start with a really simple,
real-board, a world scenario.
You handle a large build system as a complex,
and you build a simple image as it is in a slide.
You have a simple container file.
So, basically, you build it, you probably
deploy it to the production, you generate as bones
for that, you may be forget about as bone,
and three months later, vulnerability pops up
in a HTTP BD package.
And in your CI-CD pipeline,
your image can turn red,
and you get acknowledged that your image is affected,
and so on, we know that, right?
Question is, what's to do now?
Do you update your HTTP BD package in your container file,
or you will wait for a base image update?
You don't know at this point.
I mean, in this point, you know,
you can see you are installing just an empire,
and logically the HTTP BD package will be source
probably from the base image, right?
It's a really simple here.
The real problem is in a large scale build system
with a complex container files, multiple base images,
hundreds of dependencies, basically,
here is a real problem to get acknowledged
like from where is the package came from.
And most vulnerabilities cameras tell you
this image is vulnerable,
but they have a limited capability to answer
question from where exactly the vulnerability came from.
You might remember that you produced as bone
after the time, and you would like to check it, right?
Here are the bad news, most of the as bones
or traditional analyzed as bones has the same problem.
They provide flat list of packages,
where it's everything attributed to a final image,
but you cannot extract the information like from which base image
where it was exactly in a supply chain,
this package has been installed.
And this is the problem from the perspective
of the vulnerability management.
So what is our solution?
At Red Hat, we are solving it with the context,
with so-called contextual, as bone pattern,
the idea is really simple.
We use SPD-X relationships to say from where
which package exactly came from.
SPD-X specification is really flexible.
We can even create packages, other packages,
that are basically represent our images
that are part of the build process,
our base images, our builder images,
our intermediate images.
And we can bound them with a relationship,
and we are doing it by expressing,
okay, this is my final image,
and this is basically descendant of this base image.
And we are also using contents relationship
that are already present in a generated as bone,
and we are modifying them to point those packages
back to this original base images.
And on the right side, now we know,
now it's everything clear.
HTTBD package has been installed in a Python 3.12,
and Python 3.12 is an ancestor,
or our image is a descendant of Python 3.12 base image.
What is the actual application?
We use this contextual as bone,
in a vulnerability management,
and thanks to contextual,
as bone we will immediately see
that vulnerability emerged from your base image.
So with the contextual as bone,
the remediation is straightforward.
You will would like to probably coordinate
with a redhead UBI team to get updated UBI Python 3.12 image,
and then you will just rebuild your component
to inherit the fix.
It's very important, also in our build system,
you can track all the images that are inheriting
from this base image, and you will immediately rebuild them,
because you know that they are affected, right?
Because they are using the same base image.
On the other hand, when you can see
that vulnerability came from your installments,
so we are saying basically,
my component contains NumPy,
you need to, you are in charge to fix it.
Okay, how it works?
This is most complex slide,
with some kind of animation of that.
Let's take a look step-by-step.
Okay.
In the three brackets,
we have three different container files,
and basically the lowest one is our component.
The container file above is basically its base image.
It's a Python 3.12.
This is actually really simplified real world,
a real world scenario,
where it's installed, obviously,
Python 3.12,
and our affected HTTP package.
And also, we are at the above,
it's its base image,
and this is source-to-image base image,
and we are installing guides there.
Okay.
So first step is very simple.
We all probably doing that.
We generally play in traditional analyzed as bold.
Okay.
The second step is new.
We either download or generate base image as bold,
and this is like a little bit crucial part,
why we are downloading the base image as bold first.
And this is especially important in a systems,
like a complex, where you are building not only your products,
but also base images for these products,
and base images for those base images, and so on.
And you can end up with some prompt scratch image.
So we are downloading this image,
because this image can be already contextualized,
and could potentially contain information about its ancestors.
Luckily, this image,
this as bold of the base image is already contextualized,
so we can see that, okay.
This image actually contains HTTP-D and Python,
and it's base image source-to-image base contains git.
Okay.
Let's return back to our traditional,
my component as bold,
and let's start modifying it.
So as I said, we can add various packages,
so we are adding representations of our images
that were part of our build process or previous build processes.
So we are adding UBI Python 3.12.
This information we can extract from our container file, actually,
and the second base image in a chain is
this UBI source-to-image base,
and this information we are actually extracting
from a base image as bone.
And we are also bounding them with a descendant of relationships,
so we are creating some hierarchical chain of these images.
This is the most important part we are taking those two
as bones for a base image, and our generated as bone,
and we are matching the packages between them.
And the rules are basically simple.
Everything what is matched between those two documents
is basically sourced from the base image,
and everything what is left not matched
is actually our own installments in our container file.
And we can divide this in basically two categories,
those matched packages, those packages that came from
our UBI Python 3.12.
So for these packages in our traditional component,
as bone we just modified those relationships,
and point those packages to this representation of UBI Python 3.12,
and those packages that already have the relationship
in the base image, we just inherit this relationship
if it makes sense.
Okay, and this is the basically final product,
where is everything on the place.
So we are saying that my component is a descendant
of UBI Python 3.12, and this contains HDPD and Python
and Python 3.12 with the descendant of source
to image base, and this contains Git,
and NumPy was installed in our container file.
So this is that simple is just this between two documents.
No, it is not.
How we would match packages between two S-bones.
Ideally we need some kind of unique identifier, right?
Yes.
But the problem is that SPDX specification doesn't
enforce any, but we have some like hooks that we can catch,
and those are three basically unique identifier.
Perals, package URLs, under some circumstances
can be used for matching those packages.
The another option is package verification code,
which is basically SPDX guaranteed unique identifier for package.
And check sums could be also useful,
because if you use the same algorithm for the
code calculating for a base image as bone,
and a child image as bone, there are basically comparable.
But some packages like custom binaries and files
have no unique identifier at all.
This is the limitation of scanning your software.
So we cannot reliably match these packages,
and we are at the beginning with such packages.
We know that they are present in our image,
and we are unable to tell from where exactly came from.
There is also like other point of view that contextual
S-bones is actually exposing this gap,
making those packages more visible,
because you can mark them with some annotations.
And also allows us to ask two very important questions.
Can we improve the S-bones generation process
to introduce some identifiers?
It can be comparable across different documents,
or it should be a little bit more careful about these packages
in our trusted ecosystem.
Okay, before discussing those questions,
we have another result, one myotechnical project,
and this is actually third source of content
in our S-bones, and those are multi-stage builds,
because this is something what is left in our
component content, and it looks like it goes installed
in our final layer, but it is not.
It was actually built and taken from the
other stages of multi-stage build.
And right now I am calling my colleague Martin Eugenie
because he knows most of about this.
Yeah, thank you for the work.
Please people in the back, if you cannot hear me, give me a sign.
Okay.
Okay, I'll find.
So as I said, we have a solution that works for base images.
If we have packages in a base image, we can identify
where exactly they are coming from, so that's all good.
But there are more, let's say complicated cases,
for example multi-stage builds, where these base image
contextualized S-bones are not enough.
So if you take a look at a build like this,
you know, we have a builder stage that
inherits from goal and latest builds,
some kind of go application.
And then in the final stage,
we copy this resulting build binary into the final stage.
And as a curveball, we also copy another binary.
That's completely different.
So we copy a slash-bind slash curl,
just as an example, to identify
actually the weaknesses of base image contextualized
S-bones.
So why is this the problem?
Why is this S-bomb not enough?
So let's say we have a vulnerability that we come up with
image scanning in our hello program.
You know, it has some dependency on some library,
library version 2.0.0.0, right?
And we have a vulnerability in the
slash-bind slash curl program.
The problem is that the path through
mediation for these vulnerabilities is quite different.
In one case, we have to update our program,
either fix a vulnerability or maybe a band
dependency.
But in the other case, it's not really our responsibility,
although it is our responsibility.
But it is not.
We cannot easily fix a vulnerability in curl.
What we have to do is actually wait for
the maintainers of the goal and latest image to update
it and then we can rebuild our image.
So in the case of red hat, the goal and latest image
would actually be managed by some other team.
And so what needs to be noticed,
notify the team and then rebuild our images,
they are affected.
And if we take a look at the base image contextualizes
bomb, we can see that there is no way we can
actually distinguish between these vulnerabilities.
So what can we do?
Because of the fact that in complex,
we have access to the build process.
We have access to some additional context that we can
leverage to identify content and classify them
between different images.
So we also have access to the container file,
which can be very useful because
when copying content into your final stage of your image,
you have to use the copy command.
And what that gives you is the opportunity to
see and list all the paths that you copy from
into your final stage.
So you can see that in the middle,
square above.
We have the goal and latest image,
and we copy slash bin slash hello,
and slash bin slash curl.
Right, so we have that.
But since we are in control of the build,
we also have some other advantages,
like build artifacts.
So when we build an image,
a lot of complicated stuff happens,
but we also have some artifacts from the build,
which we can leverage.
So during the build,
we of course pull the goal and latest image,
and we have full access to its content content.
But we also have a so-called intermediate image,
which is probably not a very often used term,
but in this context,
it means a kind of image that is created
when you run all the instructions in the builder stage.
So if you take a look at the container file again,
once we build our hello application,
we create some kind of temporary partial image,
that has all of the content already installed.
And by combining these two sources of information,
we can select selectively scan images,
and only those past which interest us,
and we can determine which content actually ends up
in our final image.
And so at the end of this process,
that I simplify it quite a lot,
but there's no time to go into it more.
We have some kind of classification of packages,
that were copied to the final image,
and we know exactly where they came from.
So we know that the hello program,
and it's dependency lib,
it's from the go-line latest builder image,
or the builder base image,
and we know that curl is from the intermediate image.
Okay, what can we do with this information?
If we take the base image contextualized as bombs,
and again, utilize package matching,
we can selectively classify which package
originates from which builder image.
So we know that by matching hello with hello,
we get that this was built during our build process,
so it is our responsibility to fix,
and we know that CURL is from the go-line lightest builder image.
So we have to do is notify the owners of that builder image,
and rebuild our image.
At the end of this,
we get the fully contextualized bomb,
which classifies the relationships between our images.
So if we see the green, my component image,
we see that it is a build tool of a go-line lightest,
which specifies that go-line lightest is a builder image for this image,
and also go-line lightest intermediate,
it is the descendant of the go-line lightest,
because it was originally that image,
but we installed some content in it.
And we have nicely classified packages between the different images in the build.
So that's really all from our site.
I hope that it was easily understood,
but probably not.
If you scan the QR code,
you have access to some of our repositories,
where we do the work,
and also our emails, our contacts.
If we have any questions,
we'd love to hear them,
and hopefully answer them.
Good.
Thank you for keeping it very short,
so we have more time for questions.
You have the mics for you.
What about tools,
that allies,
as long as we are full of descendant of relationships
for building an entity for them,
if you respond doesn't want to expose the dependencies,
and what's?
Those,
as opposed to,
is like,
can you repeat the question?
Yeah, I can repeat it.
The question was,
what are the other,
if I understood it correctly,
what are the other software types,
right,
except content images?
If I analyze your conceptual,
let's start with the other things.
Okay.
Set it up.
Relationship,
but that's not far away from building the entity.
It doesn't see a lot of the analysis.
How does that align with the fact,
and what's to use this there?
Yeah, this is a new concept,
so the tooling for me is doing that.
Yeah, sorry.
Sorry about that.
Yeah,
the question was about tooling.
What if the tooling doesn't understand the descendant of relationship, right?
Yeah, this is a new kind of concept,
so the tooling needs to be adapted for that.
Okay, so we are working at delivering this solution in another redhead team,
that they are need to adapt their tooling to understand this descendant of relationship.
So it's a redhead job?
Yeah, this is,
yeah.
The question was, if it is internal,
but it could be used,
and like,
the concept is,
we have basically,
we have the tools, open source,
and can be used,
and that tooling might be adapted accordingly.
Yeah, the approach is kind of experimental,
so like there's no tooling that can actually take advantage of it,
because the idea is new.
Go ahead.
When you're contextualizing it,
this metadata stored in the A to itself,
we'll see our information,
so we're going to install system that now
is always metadata.
If you start programming.
Okay.
I'll repeat the question,
but it's a clarify.
About the data you mean,
which part exactly do you mean the response of the images,
or?
You mentioned that you're contextualizing it,
so you add things that you understood about this specific,
the dependencies may be the versions
or the relationships between the base and the environment.
So if you kind of analyze this image,
and you have to metadata.
I was wondering,
the metadata is one of our analysis,
and then you have to understand how to report,
or is it going to explain the image?
Okay.
To repeat the question,
where is the metadata used for contextualization,
actually coming from,
is there internal system for that?
So there is,
there are the parent's s-bombs.
Those are basically just analyzed s-bombs.
We contextualize them if we can.
If not,
there are simply analyzes bombs,
with the tool like sift.
If we're talking about metadata,
such as the actual container files and stuff like that,
we have access to the build,
so we can see all of the intermediate studies
that get created there.
So there's no special tooling.
All of this is available when building an image.
I'm not sure if that's enough,
or maybe if you want to add something.
No.
You also want to.
What?
Where do you give this information about descendant total stuff?
So is it inside the inventory?
Yes.
The question was,
where we are keeping this information about descendant of,
well actually in the contextual as well itself.
So if you are building your base image,
you will contextualize it.
You will acknowledge about their parents in the chain.
And while you are building your descendants,
you are basically accumulating information
downstream the chain.
Does it make sense?
Yes.
But do you put these both inside of the chain?
Yes.
No way.
We are actually storing it into places,
in a class which is basically instance of TPA,
which is open to our product,
and we also storing them in a repository
where we are storing all content images.
Okay.
Please go ahead.
When you go to all the containers,
you know, from the very base of it,
you've been adding files to the application.
Why are we just generating the next one,
each of those states,
and then looking at the delta change?
Because that will find out which we compose
come in at each state instead.
You have,
Exactly that.
And then allow it to work with both sides of the edge,
as well as actually the edge.
Yes.
So to repeat the question,
why aren't we generating
as bones from all the images in the chains?
The answer is,
we do exactly that.
So this does not reflect well on the presentation,
but basically the process that we use.
Like every time we generate,
we build an image,
we generate an s-bone for it,
and then once we generate an s-bone for the send-end of the image,
we use the already generated s-bone,
and then contextualize based of the whole s-bone,
of the ancestors.
Sorry.
Yeah, pretty much.
I mean, there's no real reason why.
The question was,
why we are not using cycle on the X as well.
Yeah, we are not using that,
because there is no like convenient relationship pattern there.
So.
Oh, yeah.
Yes.
As a simple tool user,
using set-up,
tri-bay for inspecting containers,
I would say an optimal solution would be for me.
This solution would be if those containers,
containers,
scan all the images in the world.
I mean, would come with the content,
I mean, with the information,
do you assemble already in the s-bone,
and having a s-bone,
and with the text connected to it,
so I couldn't,
in the end,
just really say okay,
that's it.
And even if you don't have access to that,
I don't have access to that.
So,
that would not be good.
Yeah.
So it was a suggestion to generate s-bone
for each image in this chain, right?
Yeah, yeah, yeah.
And what is the problem of this approach?
I don't know.
Yeah, yeah.
But yeah,
but there is a one-one-like problem
that you are so suggesting
that you would take a look into this s-bone,
which you,
the up the chain, right?
And you look where is the vulnerability.
Yeah?
This could be an approach.
Definitely.
It might be a simple way,
but the problem is that
the each s-bone you are taking a look
is basically content of this particular image,
for a particular base image.
And while you are doing some installments
downstream,
you are changing this content.
Why?
Because you can delete the content.
You are updating your packages.
And at the package managers,
I'm basically removing older packages.
So this is not like the correct representation.
Downstream, there is a contact show as well,
which is like,
absolutely,
let's say,
most precise representation
of the content from all this chain.
Okay?
If you take a look to steps about,
this is not like,
or rather say,
downstream,
we are not super set of all the packages
that are installed,
M-O.
Yes.
Okay.
That's what we see.
Thank you.
We have time.
I think so.
Yeah.
I'll go ahead, please.
Yes.
I'll go ahead, please.
Yes.
We just use the action way to allow systems.
Because I have to update and I can use this,
for example,
between five blocks,
and use it for,
we know that you designed the license.
Okay.
To repeat the question.
Sorry, I'm deaf in the horse.
Okay.
It's a long question.
So,
okay, I'll touch some rice,
hopefully it's correct.
So, can we use this for different platforms?
Okay.
And,
what do we actually do?
If we scan the layers of images.
Right?
That was the question?
You just need the action way to allow systems.
Yeah.
If we have access to each layer,
to just the response or each layer.
We have access to the layers.
Since complex is a build system,
we are using build data to build our images.
We can basically inspect each layer
and see what control region is there.
And so, we can scan just that content in each layer of the image.
And then we get, basically, the package is present in that layer.
And this way, we can contextualize the response.
I'm not sure that's cool enough.
Okay.
Thanks.
Thank you.