Can you hear me now? Great. Hey everybody, I'm Stefano Ventasugia, and I'm a software engineering
in reddit. And today I'm going to introduce you to a conformist, a pleasure in policy
engine that my team and I are actively developing. Let's start from the problem. As the
audience is in this room, you already know, we are seeing a boom in tools to generate
this bombs. And we have plenty of, to generate, sorry, a secure supply chain, artifacts in general.
And we have plenty of this bombs, a salsa, a provenance, a test stations, and vulnerability
reports in our pipelines. So basically what we spent in the last years is to, so we spent
the last years solving the generation problem. But we have another problem now. We have all
this data, but who is actually very fine it? For example, if an SBOM contains a vulnerability,
if a vulnerability is present in an SBOM number one checks, are we actually releasing our
software securely? Of course not. This is why, if we don't have a mechanism to verify all
these documents, all we have is passive compliance, not active security. And we cannot
rely on human oversight either to verify these documents, because it cannot keep pace with
the modern CICD deployment velocity. This is why we build conform. Conforma is an open source
tool designed to see exactly in the gap between artifact generation and production deployment.
It works by evaluating the documents generated by the supply chain and to evaluate them against
a policy that is defined as a code. And finally, it outputs a clear pass fail signal that
can be used in the pipeline. It also generates a human readable output to make the violations
that are produced during the evaluation clearly understandable and to facilitate the remediation.
But not least, it also, it is designed as a lightweight CLI tool that can be easily integrated
in your pipeline, where it can serve as a blocking gate. This is how it works at a high
level. The CLI takes the input, all the documents that are generated in your pipeline.
So for example, image manifest, image signatures, SBOMs, sub-Provenants, attestations, but
also CV scans, test reports, and basically whatever you want to validate. And evaluates
them against a policy. That is a set of rules that enforce a variety of compliance checks
against these documents. This policy is highly flexible because it can be adapted and
parameterized using a policy configuration file. So that the same policy logic can be used
with different security profiles for different environments. And finally, the CLI produces
a simple output, as I said, that can be used as a blocking gate in your pipeline.
Having the policy logic and the policy configuration version, there is a code in a Git repo.
In a Git repo, offers several advantages. First of all, you can apply the same quality standards
that you use for your application code, also for your security logic. Also, your policy
and hence your security posture is fully versioned and though it will. You can also write
unit tests for the rules you implement so that you are sure that they behave exactly as intended.
And finally, the policy can be jointly maintained using a collaborative governance model
where several teams contribute to the policy definition. We, in a reddit, adopted
a conform on the majority of our bills and we aim to adopt it on the totality of our
bills. At the following link, you can find the full list of checks that we enforce in our
bills. And in this slide, they put the most interesting ones. So, for example, we have
basic magic checks to check that to ensure that the basic magic that we use in our containers
are trusted and come from trusted registries. As bomb checks, that enforce a variety of checks
against the packages listed in our bomb, such as if they are allowed if they have only
allowed attributes and so on. Several checks against the salsa provenance attestations
to enforce salsa security levels. But also checks that ensure that the software is free
of the CVs and checks that ensure that the software is being tested and that all the tests
are green. But that's enough talk. Let's move to the demo. Let's start with a simple example.
So, suppose we want to validate a simple JSON file that contains a list of animals belonging
to different species. We implement our rule. This is written in a regal language. That is
the declarative query language that is designed specifically to write policies. You can see
here that we define some metadata to describe the rule and the steps that are needed to fix
the violation. Then we have a line 10. Let's see if the list works. Yes. At line 10,
the nigh contains result is the default pattern to define a violation in conforma.
And then the rule simply iterates the list of animals and throws a violation if any snakes
are found in it. So, basically what we are enforcing with this rule is that we are forbidden
snakes from being on the list. Then we have our policy configuration that in our case
simply includes our rule we just define. And that's it. Two files is all we need to run conforma.
Is that simple? Let me show you the common. Is Evalidate input is the main CLI common. Then we pass
the input JSON that we want to validate. The policy yamol that is the policy configuration that
will be used for the validation. And then a couple of arguments or optional arguments to
improve output readability. I also add the line to show you how the common produces a simple 0,
1 exit code. And this is the output. Since we had as naked our list you can see that the validation
failed with one violation. And you can see in the violation text all the metadata that we define
to describe it. So, we have the reason why it failed that is pointing out specifically which
any more is throwing it. And the solution steps to remediate it. Not is also how the common
existed with a non-zero exit code because the validation failed. Now let's suppose we have
choose nakes in our list. And let's see how the conforma output changes in this case.
Now we have two violations, one for each snake. And again the reason of which violations
shows exactly which snake is throwing it. So, this further facilitates the remediation.
The common is still the validation is still failing. So, let's fix it now.
By transforming our nakes into rabbits. So, let's run conforma again.
And you can see that now the validation is passing. And the nose nake rules, nose nakes rule
is successful now. Since the validation is passing the exit code is no 0.
Sometimes we don't want to straight up deny something but just warn the user about potentially
bad situations. So, for example, in our case we are okay with cats and dogs being in the same list.
But we want to warn the user to be aware about potential fights.
So, let's define our nangle rule. This is very similar to what we did before. Again, we have the
metadata that describe it and the solution steps. Then we have worn contains result that is the
default pattern to define our nangle. And then the rules simply check if there are both cats and dogs
in the list and if so it throws the warning. Let's run conforma again.
And you can see that now the validation is still successful and the exit code is 0.
Because warnings are non-blocking. You can see that the warning is showing up in the result list
again with all the information that we need. And we still have the nose nakes rule that is successful.
Another useful feature for conforma is the possibility to include or exclude specific rules from
the evaluation. So, for example, we are okay with the cats and dogs being in the same list. We
don't care about the warning because we know that our cats and dogs get along well with each other.
So, we want to exclude that warning. We created this section in the policy configuration file
that lets exclude in the rule and run conforma again. You can see how the warning disappeared
from the results list because the rule is not being evaluated anymore. Of course, you can do this
also with violation rules. And you can also include only specific rules in your evaluation.
Now, let's move to something more interesting. Let's validate an s-bomb.
I created a simple s-bomb in a speed x format. You can see here the first lines.
And you can see here the list of packages. I put two packages in it. One ledget one that is
my app and one that simulates a package affected by a critical vulnerability. So, let's write our rule.
This is slightly more complex than the previous ones. But basically what it does is to
iterate the packages in the s-bomb and check if any of them is included in a user defined list of
this allowed package. That is this rule data package scheme. And if one is found the violation
is thrown and the failure message clearly points out which is the package that is allowed package.
For this rule, I admit I cheated a bit because I imported a library of helper functions
that we inform created to help navigate the main types of attestations.
And this is our policy configuration file. In this case, we are not only importing the
library and the rules definition, but we have also this new section rule data. Where we define
the user data basically. The data, the configuration that the user use wants to use to
for his evaluation. And here we can disallow the package version that is affected by the critical
vulnerability. So, let's run conform against the s-bomb with this configuration.
And the validation is failing because conform is correctly detecting the
this allowed package. You can see again the steps that are needed to remedied the violation.
So, let's suppose now that now we fix our application by bumping the package to a new
major version that is not affected by the vulnerability and that we update this bomb so that it
reflects the change. And in fact, you can see here that this bomb is listing version 2.
So, let's run conform again. And you can see that now now the validation is successful,
meaning that the s-bomb is compliant to our policy now.
The last demo is the most interesting one. I created a simple repo. By the way, don't worry
about coping the link. I will share it with you at the end of the presentation.
And I will show you how we can validate this bombs and salsa program and statisticians
that are being generated by our GitHub actions workflow. So, let's clone the repo first.
Here we go. You can see here that it consists of a simple containerized
going application. And for the sake of simplicity, I put in this same repo the policy rules and
the policy configuration. However, for better management and security, they should leave in their own repo.
By the way, this is easy to do because in the policy configuration, you can reference other
policies, also by also using OCI tags or GitHub links, making referencing sources across repo
very easy. This, for example, is a policy configuration file that we use in one of our redat
repos. So, let's move on. You can see here that the goal and application has a dependency on
0 log 1, 3, 2. The time to reduce the to show you how conformity else we did.
And then we have a very simple container file that simply copies the binary, the newly built
binary inside the container and then runs it. So, let's define our rules. The first one is the
same that we use in the previous example, that just checks that the package is in an S-bomb
or this allowed or not. And then we have this new rule that checks that the builder ID that
is specified in our self-saprovenance is among the list, is among the list of allow the builder IDs.
Again, here we have this allowed builder ID scheme that is as section defined in our policy
configuration file. Similarly to what we did with our, this allowed the packages.
And if the allowed the builder ID is not found, the exception is thrown.
So, this is our policy configuration. You can see again, we are importing the policy files
and we have this rule data where we are disallowing a 0 log version 131.
Remind that in our application, we are using 132. So, this is okay.
And we are allowing only one builder ID that is specific for my repo, specific for the release work flow
that is specifically running on the main branch. So, this is very strict.
And then we define our workflow. It will have three jobs, build, validate, and promote.
What I am showing you here is the core step of the validate job.
It is using the publicly available image of a conforma.
Here for the sake of consistency, IP in that specific digest, but you could use the latest digest for
simplicity. And I am calling here the conforma CLI using the validate image command.
This is different from the validate input because it can run additional checks against the
image being validated, such as image signature check, salsa provenance attestations, signature check,
and format check. Here I am passing the image references input instead of a file.
And we have also additional arguments that are needed for the signature verification.
So, let's see what's the output of this workflow. By the way, this is an actual output
that is taken by acquiring the GitHub API. So, you can also find the outputs of this workflow on GitHub itself.
And you can see here that the validation is failed because we are running the release workflow
on the main branch. And we have the analog version in our application.
So, you can see that the three jobs are successful. The promote job only executes if the validate
job is successful and it promotes the newly built artifact to the latest tag simulating our
successful release. By the way, I didn't include it in my demo, but this workflow actually pushes the
artifact on my personal image registry on Quay, whose link I will show you, I will share
with you at the end of the demo. You can also see here in the workflow output and the workflow
log that it's listing the artifacts that are being generated. So, let's now suppose that we
change our policy and we now forbid a zero log versions up to one three two.
You can also notice here that we have two policy configuration files,
policy pull request and policy release. This is because we are using different security profiles for the same
policy rules. In particular, with regards with the builder ID, because it is different if the
workflow is run against the, against a pull request or against a, actually, this is a different
workflow running on a pull request or the release workflow running on the main branch.
The updated policy triggers a failing workflow. You can see here that the validate job failed.
Let's take a look closer look at the conform output. You can notice here that there is the
violation is expected pointing out that a zero log one three two is this allowed.
And you can also see here that we have the other rules that are being evaluated, the
first successful. So, we have the artistic signature check and syntax check, the image signature
check and then our rule about the builder ID. So, we want to fix the violation. So, we open
a PR. Oh, by the way, basically what we did before is to prevent a release of a software that
is not compliant to our policy. So, we want to fix the violation now to be able to release again.
So, we open a PR where we bump a zero log two one three three. And this is the workflow
output running on the pull request. It is successful. So, it is the change we did made our code
compliant for our policy. And notice here how the pull request workflow only has two jobs
because it doesn't need to release the software of course. So, we can safely merge the change.
You can see here how the change is reflected on the main branch. And finally, the workflow running
against this commit is successful again. Meaning that we can safely release our application
being sure that it is compliant to our policy. And that's it. Thank you for your attention.
And if you have any one to download the link, you can find it here. Any questions?
Questions? Questions? They're up there, someone? Yeah.
Excuse me?
I have a question. If you have, for example, two, the CV is in your code. And you fix one. Are
you able to release again and just improve the situation gradually?
Yes, basically, what you do with your policy depends on you. So, for example, you can
temporarily exclude one specific CV from your evaluation or a set of, for example,
based on severity of CVs. So, you can action it gradually.
Yes? Maybe.
Oh, really? Okay. So, let me share it. Maybe you can see here. It's basically this part.
Without the build sample. So, I had true repo. Only with first them 20, 20, 6 is the one that
links. Yes? I was wondering if you can use other sources to define these policies like
checking on a data with the clinical bots on the end of your analysis.
Oh, like the question is if you can use other sources to evaluate your policy,
other source of truth, right? But you mean, on GitHub, like bugs?
Okay. So, for example, if you on GitHub, you have a list of dependencies and this dependencies
have some bugs open on GitHub, like GitHub issues, right? You could, for example, write in a rule
that queries the GitHub API and checking if there are any bugs. This is an idea of I don't know if
there are any tools that can do it. You could use the output from a task running that specific tool
and evaluate that in a conforma. For example, we do this with vulnerability scans. We use
clear scan in our pipeline that generates the vulnerability report and we only evaluate that
report. We don't actually check vulnerabilities by ourselves.
Yes?
Oh.
I was just wondering, like the inputs to your policy that they asked all the January update
out of stations, like what kind of how are you produced and how are you that see on some
code and use the input, but I just had a wonderful thing to use the support of the inputs
and the policies.
The question is how we generate this box or how we import them in our evaluation?
Yeah, like a little bit of both and like, what things do you generally have, like, support,
like, things like that? What do you support those inputs to get for this?
Okay. What do you support as input in your policy?
Basically, we use, when you call the validate image command, basically what the conforma does
is to use the cosine non-load command that downloads all the
attestations related to that image.
So, basically that, but you could also attach to the, for example, our build system,
attaches in the salsa provenance attestation, the full report of the query run with all the
tasks that have been run parameters and outputs loads, so we are able to check basically
everything that happened in our build pipeline.
No, I'm not going to talk about that.
Well, the question is sort of, as a community, I consider some very common policies appearing,
so it is the way of sort of having a general policy that we can then sort of speak upon,
because you've obviously demonstrated one of the spots specifically.
Yeah, of course.
The depth, actually, to the validating of picking up on that previous one,
the build is because of the critical rule, the values of failure, if it took about a day,
I guess, well, that's what's found.
Yeah, the question is, is there are some default policies, like sets of rules to checks,
like sets of things, no, vulnerability, or maybe just image checks and so on.
We have some, in the metadata, we can define collections, so we have collections of rules,
that checks, for example, salsa attributes, or we have a minimal set of policies that check,
signature and not much more.
So, yes, we have this function, it's not so extended, let's say, we could have more granularity,
but the feature is there. For example, yes, there is the minimal collection,
a collection called the minimal, that, as the basic set of policies that you can enforce,
so you can start from there.
Okay, times up, so thank you again. Right, thanks, Tejman.