All right, welcome back, everyone.
With my pleasure, I welcome your tian, Franken, to talk about back home.
Thank you.
Thank you for the introduction.
So, I'm Giftiana.
My researcher at Distinette, a research group at K-Level.
And I'm going to talk about buckhawk, what we call automated browser
buckby section on steroids.
So, maybe some context.
I think we can all agree that when software becomes more and more complex,
the likelihood of a buck being introduced also increases, right?
And even to that extent, that seemingly small changes to the codebase
can lead to severe security bugs.
Let's place that in the context of browsers.
Nowadays, very complex systems, tens of millions of lines of codes.
And it comes very close to other complex systems that we know being operating systems.
And indeed, this often translates into vulnerabilities.
For browsers, I think on a weekly basis, we see new vulnerabilities being discovered,
being fixed, discovered fixed.
It's like an endless cycle.
And we think we can improve this process by looking at buckfixing process.
I think most of these processes look a bit like this.
First, vulnerability is reported, then it's priaged.
Somebody has assigned to fix it, and then the fix is tested.
Now, we think with a proof of concept of vulnerability, we can do a lot more.
We can do a full buck life cycle analysis.
Because, for example, if you know the cause of a problem, it's often much easier to fix it, right?
Also, there might, in the history of the codebase,
there might have already been a solution at some point in time if the bug is regressed.
And in the long run, we think we can improve the development process as a whole.
Now, we don't want to burden the developer with even more work,
so we automated this buck life cycle analysis process.
Just conceptually, buckhawk works with the version control system often get.
Just a recap, let's say you get it.
If you commit, you save like a snapshot of the codebase.
So here, the green dots stand for snapshots of that codebase.
And let's say we also have a proof concept of a vulnerability.
So then, in theory, we can take that proof concept and check it on every snapshot of that codebase, right?
So here, the red squares stand for snapshots that are vulnerable to that's bug.
And with this, there's really a lot of information that can help us fix the bug,
but also prevent bugs in the future.
And of course, like codebase like Chromium, there's nowadays already more than a million commits,
so that's the main reason why we automated this.
So just to give you an idea of the tool, this is a tool user interface.
So as you can see, we can define what browser we want to use for now.
We support Chromium Firefox.
The proof concept of our bug is here.
We selected and then essentially we start buckhawk.
This is run on a VPS with ATPU course and 16 gigabytes of RAM.
It's not sped up and this was recorded.
I think I did it yesterday.
So in a while, we will see dots appearing here and each of these dots stand for commits.
So on this line, we see commits where the bug is reproduced.
On this line, commits where it's not reproduced.
And in a bit, we will see a general idea of the lifecycle of that specific bug we're testing.
As you can see here, I've seen that this before, so it's not a surprise for me,
but here we will see a dot, and we know that the bug is introduced between these two dots, right?
Because the reproducibility has shifted.
And if we leave this on for about 10 minutes, we will know the exact commits where the bug was introduced,
which again gives a lot of information.
I'll go ahead and this is also more of a high level view of buckhawk.
First of all, we give parameters for the evaluation, being the browser,
while there's a range of versions that we want to consider, and then of course the proof concept.
Proof concept can be quite complex sometimes, but to some extent buckhawk can handle that.
I don't have time to go into the details here, but I can explain the next steps that buckhawk will do in an automated way.
It's essentially a loop, and first, first thing in that loop, buckhawk will choose the next commit to evaluate.
Then it will take that commit, search the executable, that associated with that commit.
And as a last step in that loop, it will evaluate the proof concept against the executable.
So this is loops until the full analysis is finished.
Okay, so I don't think I have time to go really into detail, but in essence we made a sequence algorithm
that's in an intelligent way, chooses the next commit, it's in two phases.
And in the end, maybe the last phase is the most important, it's called the pinpointing phase.
So as you can see, buckhawk will concentrate on these parts because these are the most interesting.
And at the end of that phase, if I can click it, you see that there's a lot more evaluations here.
And if we zoom in, we can see what commit was responsible for introducing or fixing a buck.
Okay, so because we do dynamic experiments, we need executables that associate with a certain commit.
And for this, we rely heavily on repositories of browsers.
For example, for Firefox, we use a nightly for chromium browser snapshots.
In the past, we also built these from source, but it's very resource intensive, so we don't really do that anymore.
And we built a lot of optimizations.
For example, the sequence algorithm in the last slide also takes into account availability of these executables.
Then the most important step is, of course, to evaluate the proof concept on the executable.
For this, we cannot use browser automation tools like Selenium because they only go back to, let's say, version 50.
I don't know it by head, but we want to go further in history even more, right?
At this moment, buckhawk supports browsers until version 20 till the latest release.
And we just call them using the command line interface.
So in essence, each browser, each executable, is executed in its own isolated Docker container and all dependencies are taken care of.
And a proof of concept is essentially like a bunch of web pages that interact with the browser and try to elicit a certain vulnerability.
When this is done, we reject the logs and buckhawk can then say, okay, in this commit, that buck was reproduced or in the other one, it was not.
This is like a summary.
When the analysis is complete, you will see something like this.
We can zoom into that and click even the dots that represent commits and go to the commit web page at public, for example, for chromium.
So we showed that buckhawk can be very useful by analyzing 75 CSP bugs.
We find a lot of interesting findings here, for example, that some of these vulnerabilities are already publicly disclosed before they're even fixed.
This was the case for tree vulnerabilities.
So that shows that there's some room for improvement in these buck handling processes, right?
Also, at the time of our evaluation, tree vulnerabilities were still affecting the laser release of these browsers.
I don't think I can go into this, but what's maybe more important is that we want to extend further.
So before we started with just web browsers, right?
But now we also want to take the V8 engine, for example, and just run proof concepts on that.
So that's now supported in the latest version of buckhawk.
Also, rapidly run times, but this is a bit more experimental.
There's some hurdles compared to browsers.
For example, we have certain builds flags for V8, for example, enabling the sandbox.
And also, we're still looking for repository with Spiderman key executables that are associated with single commits.
So if you know this repository, please tell me.
And even beyond that, beyond the browser ecosystem, we think buckhawk can also be very useful.
Because there's only three main ingredients that we need for buckhawk to be applied.
And that's first of all, like a gold base that is a version controlled.
Also, we need an easy way to execute the builds that are associated with commits.
Either the building can be quite efficient or there's like some repository where we can download them.
And then lastly, we should be able to interact with these executables, for example, automating user behavior.
And these are the most important aspects, and if these hold true, I think buckhawk can be applied on any system.
Okay, so I think I give you a summary of buckhawk.
If you're interested here, some links also of previous presentations.
And if you want to discuss more, I'd be happy to talk to you after this session.
Thank you.
We already have a question.
Did you reach out to Google or Maria, but it?
I mean, some people, but I didn't get a response.
Okay, I'm in charge of that, Maria.
So please, we should chat because it's super exciting what we've been doing.
Okay, right, too.
Thank you, it's incredible.
I hope question actually, how long does it take to find a regression point like for one buck?
Okay, it depends, but how early are you looking for a version?
I think it's safe to say, on average, 10 minutes, something like that.
And it really depends, like, is there a lot of fluctuations in reproducibility?
But in most cases, I'd say 10 minutes.
Yeah.
So you didn't turn on the whole browser like it's just from home?
We do turn on the whole browser.
So essentially what happens?
Let's say you start with evaluating.
You just buckock will choose the commits.
It simply downloads and immediately instructs the browser in RAM to visit the proof
of concept webpage.
And based on the request sent, it can already decide, okay, this one is a committed
process where the buck is affecting the browser or not.
So in that sense, and we also run it in parallel.
So I think in the example I showed, there's 12 containers running at all times doing the evaluation.
Cool.
It goes up and down all the time, so it's reproducible, not very reproducible.
But the proof of concept is that not deterministic or why does it happen that it fluctuates as much.
Like I understand that some point is fixed, but yeah.
But this is like a time axis, right?
The axis is time.
And so this is, for example, let's say 25, I think.
And here, 20, 26.
So that means that between the releases of 25 and 26, the buck was fixed in this case.
So this is going up and down because you go throughout history, the development history of the browser.
But the buck is deterministic and it.
Yeah.
So here you see, well, here the buck is certainly reintroduced.
Here it might be because the policy that we're testing is not yet supported.
But here indeed, it was fixed.
It remained fixed for some time, and it was reintroduced a few versions after.
So do you think your tool can generalize to like other programs, you know, like compilers, maybe, or things like that?
I think it's possible.
Like what do you need from the software being tested in order to do the by-section?
Yes, it's simply these three things.
I think that's all we need.
Get a buckhawk works per commits, fetch the executable, and then interact with the executable to elicit the buck.
And if that works, we can...
I'm sorry, I didn't realize you had a whole slide, I missed it.
I'm not in no problem.
So the reason you're not doing a simple binary search is because this pattern of being fixed and then broken again is really common or what?
We use an adapted version of binary search, but because, for example, not every executable is available, we have to adapt.
And also we first want to have like a general idea of the life cycle, and only then, because we want to find regressions.
Only then, when we have like a general idea, we want to really, in that sense, do binary search on the actual commit.
Yeah, my question is, how is your difficulty to extend this to other web browsers or web engines?
Not that difficult. For example, once we had a running for chromium, it was just looking for how things were done in Firefox.
For example, it they used macro instead of get at the time.
It was just writing an adapter for that, also different endpoints for fetching the executable executables, but other than that, it's quite similar.
And my second question, not one is, how hard would it be to explore back the idea of building from sources?
Let's say that you have a powerful enough machine.
If you have a robust script for that, and you can build versions of 10 years ago, which is, I think, quite difficult sometimes.
It shouldn't be a problem, but I think the bottleneck is some things don't work 10 years later, for example, in a building process of chromium or Firefox.
Yeah, I was thinking more to integrate it with the continuous integration system for instance of WebKit.
What do I mean, trust that you can find in something that might happen?
It started happening in the past five weeks or so.
Yeah, CICD is a good example how this could be applied, indeed.
And I think in the future we will certainly look for that if we find a way to do it more efficiently.
Okay, thank you.
Thank you.
Okay, perfect.
No more questions.
Thank you so much.
Thank you.