The risk of supply chain attacks have been lost here in the media, quite popular, not only security experts,
we are discussing the topic, but also politicians and society.
Not only attacks on the physical infrastructure and on plans, but also on the digital site
have been discussed. The most major incident was the supply chain attack on the
targetting the open message server to gain remote code execution.
How we can detect this attack and similar ones will be the topic of my talk.
Who am I? I am Tobias Cyber Security Researcher from Germany.
I am working at the Final Institute of Applied Integrated Security.
My main research areas are critical analysis and offensive security, both mainly in the area of automotive
and embedded. You can find me online on GitHub and on Matrix.
The agenda of this talk will be as follows. I will start with a small background about supply chain attacks in general
and in particular the accept attack. I will introduce you to build systems on C++ and the DBN packet build system
to better understand the tech vector and the mitigation.
I will then introduce you to my concept called supply graph.
Very well traced build process. We can then analyze the graph to hopefully detect the accept attack.
At the end I will go into the limitations of my approach and to future work.
The background of supply chain attacks can probably be best described by this famous XKCD comic.
It shows that all modern software is not standing on their own but they are built on top of multiple dependencies.
We can also call them supply chain.
In our example of the accept project, we have the open message server on top and the accept library would be the small piece down there,
which is the weakest point in the chain.
If this is attacked, all other components are affected as well.
The accept attack, as I mentioned, target the open message server to gain remote code execution.
The attackers infreate the dependency, so they are not attacked the open message project directly but the dependency, the supply chain attack.
The attackers are not attacked the source code of accept directly, so they have been no trace in the source code.
But they use the build system and the way they build the project to enter their demolishes code.
So they use the test vectors and decrypted test files and then they reveal the demolishes code, which I will show in a second.
This attack was only detected by co-incidence because the log on time of the open message server was a little bit longer due to the attack and this reveals the questions of there are any undetected attacks running, which was the main motivation for my talk.
You can see a schematic of the attack on the one hand side, we have the legitimate source code of the project and on the other hand we have the attacker modified test vector.
The source code is compiled to object file and test vector is extracted to reveal the malicious code both are linked together to the shared library and shipped to other projects.
To better understand the whole process, we go into the build chain itself for C++ we have a very diverse and complex universe.
There are platforms specific build systems like Visual Studio on Windows or Xcode on Mac.
I can now order tools have been quite popular on Linux for some time.
There are Mac files and CMAC to only name a few. Those build systems are often touring complete programming languages on their own.
So you can do right on applications, basically with them you can do the dependency check getting test case execution and whatnot.
They often involve recursive file structures they include internal and external files and all this together makes a very complex and transparent system and as we know those tend to be prone to attacks.
I will now go into a few details about the deviant package system. I chose deviant because it's quite common and it has huge community but the same concept you can apply to other build systems as well.
On the one hand side you have the upstream source code and on the other side you have the deviant specific additional so this can be metadata like the package name version and so on.
What is important to know is that the original source code is unmodified by deviant and it's only extended with the separate David configuration.
The build process then reveals the build artifacts like the binary file, configuration file, icons, whatever and those are all packed into the deviant package and then shipped to the customer.
So when you opt install a packet you only get the binary files but not the source file.
And now I will introduce you to my concept called supply graph. The main idea behind this is that for source projects all program parts should be available in source code.
So that is a common interest we share in the open source community and this is as we have seen the point where the exact attack.
You cannot kind of verify because there is a binary block popping out of nowhere.
So how we can then detect the attack we need to trace the build process then we can build the graph and we can traverse the graph to find the attack.
I will now go step by step to see how we can do.
To trace the build process I used a tool called code checker. It's originally a tool for study code analysis but it can also capture the build process originally in a different configuration.
It only captures the compile commands but we can instruct it to capture also the linking commands.
It uses the preload for those of you are interested in intercepts all the execute calls tracing the external command execution.
I modified it a little bit to also capture additional tools like the sampler, the archive copy and install commands which are used as part of the rebuild process.
I also extended the compilation commands tracing file to add an output entry.
We can see on the right and the example it shows a clung compiler execution it shows the directory where the command was executed.
The file which is the output file of the compilation step and the input file which is then put and the output is the output.
So thinking about the graph structure the input file and output file are nodes and the edge between those nodes is the compile command.
We can do this not only for the compilation step but also for every step in the build process which gives us a graph structure.
An example graph structure can be seen here.
We have the source files on top. They are then built into object files.
The object files are linked into for example an application and shared library.
Both are then packed into the DBN build packet.
Now we can take this graph. We can capture the graph of the vulnerable extension and we can turn it upside down and we can see if we can find the attack.
We start with the DBN packet of the lib and as part of other files it contains the vulnerable shared object file.
This contains, for example, the CRC32 fast and the CRC64 fast object file and an additional CRC64 fast file which is located in dotlips folder which is hidden forward on Linux.
So this is already where it is suspicious.
If we trace it further we can see that for the first two object files we can reconstruct the according source files.
But for the third object file there is no corresponding source file and indeed this is the malicious code of the attackers.
So this we can successfully detect the exact attack.
The whole graph of the DBN packet of the accepted utils is way more larger than we saw in the small excerpt.
So this is a picture of the whole graph. On the top layer we can see all the source files in the middle layer we can see the object files and the shared libraries and on the bottom we can see the DBN packet.
And on the top right corner if we look close we can see the the malicious binaries.
So you can find all this work on GitHub as a proof of concept.
It also includes a Docker container with not only the exact example but also open message edge and open message edge so you can play with it and modify it.
And modify it as you like.
There are a few limitations to my approach.
First tracing the build system is unreliable as we have seen it is quite complex it involves different commands we need to pass the commands to get input and output nodes.
So this is error prone.
There are also legitimate ways of anomalies in the tree for example code generation as an example.
I saw that open message edge uses some press scripts to generate highly optimized assembly code.
So this assembly code was not part of the original archive which is suspicious in the first way.
We have also code generation as part of protocol for example to generate encoders and decoders.
If we can cover those limitations we could scale this up to all DBN packages on a regular basis to detect any attacks.
Also implement other detection mechanism on the graph so the example of finding all source files for the object file is only one example.
I also implemented a check that all source files are part of the original tarball but you can imagine any other mechanism you can implement it on the graph.
If this at the end we hopefully can prevent attacks on the build system in general or at least make it harder to hide them.
This all of course is trying to work with this whole complex build system.
So my vision would be that at the end we can create descriptive build system where we have a strict rule set what is possible and what not.
So we can static analyze certain requirements again to build system so we can in the first they prevent all suspicious work.
As an acknowledgment this work was founded by the German Federal Ministry of Education and Research as part of the attacker project.
That's it from my talk if you have any questions or ideas you can find me around at foster or online. Thank you.
Thanks for watching.