Okay, so hi, so supply chain attacks, so we probably all remember solar winds or heard about
solar winds back in 2020, maybe put supply chain attacks on the radar of everybody, certainly
myself, as sort of revealed the potential impact of an attack like this that could have
far-reaching consequences. Since then, there have been a number of other pretty sophisticated
supply chain attacks like XZU tells recently, which was a very long term package takeover
attack, something like 3CX, which was like a chain, an attack chain of supply chain attacks
to compromise, and then application. And then, you know, just every day, there's we seemingly
hear about more and more supply chain attacks coming out of various complexities, so this thing
is here to stay, and so one of the things we're going to be talking about today is some strategies
that Sebastian, my colleague, and I see all the time in our continuous monitoring of open
source package ecosystems, strategies that these threat actors like to use, and, you know,
basically, like, what do they want? How are they trying to get it? And then also,
make some recommendations for how people can protect themselves from this kind of attack.
So, my name is Ian Cutts. I am a security researcher at Data Dog, specializing in supply
chain security. So, yeah, I'm Sebastian, what are you also? I work for in that I'll, in that
I'll, for about a year and a half, I also work with the unease supply chain, and also with
in application security for another three-thirds of topics. Okay. Yeah, so just to kind of set
the stage, right? So, what is the supply chain attack, what is the software supply chain? So,
at a very high level, right? We have a simple software supply chain here. Basically,
we have some developer who wants to deliver some code to some consumer on the other end.
And so, there's this kind of high level process of like maintaining the source code,
developing the source code right somewhere in diversion control. Then we go to build the source
code and some like build system, right? And when we do that, we're probably going to be pulling in
some third-party dependencies to get involved in that process. And then finally, publishing the
finished artifact somewhere or the source code, if it's like a library, right? In some way,
where the consumer can access it. And so, basically, like, at each step of this pipeline,
there's various kinds of attacks that can take place, so we can be compromising the integrity
of the source code via like a malicious PR. For example, we can mess with the build system
or mess with the imports or outputs. We could go after the package repository itself,
like maybe do like an account takeover of the publisher, right? And then we can also try to
get our compromised or malicious dependencies into that build process. In terms of what these
third actors are after pretty simple, there's kind of three things we see a lot.
Credential theft, so developers have access to a lot of attractive targets for these attackers,
like, I don't know, data or code, right? Very simply, so a lot of times we'll see them going
after API keys or credentials like this. Resource misuse, so things like crypto miners,
you install some package and it's got an embedded crypto miner. And so, there you go.
Or a thirdly something like SolarWinds, so it's a downstream compromise where some, you know,
downstream client of the package is being targeted. And so, yeah,
it's maybe the choice to use up and source is more complicated than it might seem at first,
security-wise, so we'd like to feel confident in, you know, our decision to do so, and so that's
kind of what we'll be talking about here. Okay, so the tick-team malicious packages will go
that we've got though. We're going to explain a little bit on how we are using this CLI tool,
which is a tool that we created in data though internally for scanning PIPI and MPN ecosystem.
So, basically, we take, we use the guardall, which pulls data from the package published in
PIPI or NMPN and this tool makes use of two kind of analysis. One call, metadata analysis,
which is basically the detection of some patterns present in the characteristics of the, of the,
of the package itself, imagining something like the registration email address of a
alpha maintainer, right? In this case, you can be used in a disposal email address, and that's the
discount of the detection. On the Recon, you'll, it may use another type of detection called source
called analysis, which under the hood, we are using two types of two tools. One is called Yara,
which makes use of, it's very good for, in the kind of for compromises, such as domains or
IP addresses. For instance, in this case, we can use like this kind of addictions in a Yar rule.
In, also, we are using some bread, right? This is another kind of tool that detects kind of a
semantic analysis for a, and it's pretty good to detect like a kind of a behavioral pattern
in the code, right? For, in machine this, we are using, it's a, it's a, it shines really well
with the detection of, for example, execution of a, makes 64 payloads. So, this sample execution
of a guard dog, this tool, you can see, there are the, those are the like the, the rules that
detected in a, in a real package in MPM. Once a couple of things, do notice in, of guard dog,
is that, you know, it allows you to use JSON and set it for my output to do a more like,
automated analysis on the outcome. And also, one thing to notice is that guard dog, it doesn't
give you any, like, like a final decision, if the package is malicious or not, one thing to notice
is that it just rises the risks on the package, okay? The, the actual decision has to be taken
by an actual free-husher at the moment. So, this is how we use, you use internally, guard, guard
dog in data dog. Basically, we implemented a set of auto crawlers, use, that are basically
crawling the, the repositories type I and MPM. We, roughly, you can, can imagine that we have,
like, 300K for, uh, PIPI and 75K of, uh, PIPI, uh, sorry, 300K of MPM packages and 75K of, uh,
PIPI packages, uh, amounts. And then we scan this once with guard dog. And the, the, the, the findings
that we were talking about, uh, are filtered then with a set of behavioral, behavioral patterns,
and, uh, a combination of machine learning techniques, okay? Uh, to have an idea where, where,
considering having, like, a 0.6% of suspicious or interesting enough packages to further,
the further analysis with just the, the, the triage stage is, um, it's conducted by a human,
right? And if confirmed, then the, the malicious package is, the sample is uploaded to a public
repository called malicious public data fed. Okay. So, um, so yeah, now we'll just kind of go through,
we have four kind of common strategies we see these thread actors using and we'll just kind of
use some of these packages we've been finding with guard dog to kind of support kind of the case
for making. Um, so the first is to target popular packages to get at their developers,
we were saying earlier developers are attractive targets for these attackers. So, how do you
get developers? You go after the packages they want to use. Um, so basically there's kind of two ways,
you can go about this. Um, if you're the attacker, you can publish a new already malicious package
or you can seek to take over an existing package. Um, the first case is very easy. You can just upload
whatever you want like NPM say, but it's not a very efficient strategy because like 10k packages
get uploaded to NPM every day, so nobody knows your package, right? So if you're going to tackle
how do I get people to take notice of my package? And so the way they often do this is very,
not very sophisticated, right? You just kind of try to affiliate the malicious package with some
targeted package. Um, usually by like a naming trick. So we see a lot of combo squatting where
if I'm targeting requests, I could do something like pie requests or something or type of squatting
where the name of the malicious package is the type of the targeted package. You can see like that.
Not very sophisticated stuff. Um, on the other hand, package takeover is a lot more lucrative
because these packages already have developers that use them, but it's harder for the attacker.
They have to perform some kind of compromise together to work. And so this could be like
compromising the maintainer credentials, malicious PR, other things like that. So if you go back to
our architecture, kind of overview, this could take place anywhere right at any stage of the pipeline,
depending on what the attacker is trying to do. Um, so here we have an example that we found in the
past year of a name squatting attacker, a combo squatting attack. Um, so on the left we have a legitimate
NPM package and on the right we have a malicious one from a threat actor. We've called
tenacious puns on a data dog. So it's a DPRK affiliated threat actor and we use like the national
dog breed of the nation's like name the threat actor. So the puns on is a DPR from Doug from North
Korea. Um, so you can see very simply passport passwords, JS. Um, otherwise like if you look at the
readmees, they're exactly the same although like the graphic didn't load on the left, but you know,
you just copy the code, backdoor it and upload it with a similar name, right? Not very sophisticated.
Um, on the package take over side, the stuff tends to be more interesting, right? There was right at the end of
2024, there was a notable package takeover instance for this Python package called UltraLidics,
where they were running like a vulnerable GitHub action on their repo. You can see in the pink box,
like the the branch name is used like unescaped in a shell command. And so this allowed an attacker
to submit a malicious PR with a crafted branch name, which like ended up running a shell script in
the context of the GitHub action. And so they're able to like exaltrate the pipeline published
token and upload some malicious versions to Python. There's a great analysis of this attack at that
blog post. Um, and so like package takeover is complicated, but the name squatting stuff is very
simple to avoid by just like doing the simple obvious checks. So our advice would be just to use
simple guardrails to minimize the consequences of simple mistakes like a typo. Um, in our research
in Q4, we found that at least 70% of the malicious packages we triage were using some kind of
name squatting or targeting some other package. So if you're an end user, if you're like a user of
an open source package, just make sure you're spelling everything correctly, the simple obvious solution.
To get around package takeover, you could use version pinning, don't use like just the latest version
of the package, okay, something happens. Um, and then if you're lucky enough to work like a company
who can do this for you or in some institution who would do this use like an internal, um, not copy
it, sort of an internal version of the package repository with a verified artifacts to kind of
only pull things from, you'll cut off a lot of these kinds of problems that way. Um, meanwhile,
like for the package repositories themselves, I think we would suggest like introducing some kind of
moderated publishing. So simply like not allowing people to publish on typosquads of certainly
prominent packages, but ideally all packages, whether it's some kind of verification of like
this code looks too similar to an existing package, like no. Um, and then also post-compromise,
take those package names out of circulation and replace them with security placeholders
so that people who didn't upload or sorry update their dependencies don't get re-infected.
Okay, now we're going to talk about the next strategy you used by Threadaclors.
Which is called exploit developers via install time hooks. So install time hooks is a method that
can be used in the package as like a IPI or NPM that trigger at install time, right?
This is a, this is a very like visual way to exploit something. So how, how this actual
works in the in the in the in the wild. I wanted to go over with an example. So let's say,
I'm an attacker and I publish two set of two set of artifacts. One to NPM with a malicious payload,
very visual. I don't know with the install time install execution. And another like a sample repo
into GitHub for doing whatever. This, this packet, this a GitHub repo does have not malicious
behavior whatsoever. The only thing that it has is like is it has like a dependency to the NPM package
that was published before. Then the user, sorry, a GitHub user is lured into download the package,
right? There are a number of ways how Threadaclors can achieve this. One of the favorite techniques
implemented by DPRK, Threadaclors is to use them as a job interview, right? Hey, I need to do this
job for me. You have to do this test, test it out. You have to download this code and execute it. Then
bam, this, this, this code is pulled from the NPM and triggered. So if you look at the
again in the architecture diagram that I was shown before, this attack goes over in the use of
compromised dependency and it's a compromised package. Parts. And in here we can see an example of how
this self-delicious, self-deleted initial attack access vector was used by the Threadaclors test
function. Again, in data though, we name the the known Threadaclors function in this
indication goes after the breath of the the country that was the Threadaclors representing.
And so we can see here that it's executing the reference JS file, which in this case downloads
using a curl. Downloads are DLL. I use this run the allow 32 binary to be a trusted binary execution
of the DLL code that is the downloading. This is a technique very used again for by this Threadaclors.
Another example from in this time, in this case from PIPI, from the Lappardexos package.
In this case was from MUT 8694, MUT stands for miscellaneous and attributed Thread.
Basically, this is the way we name a Threadaclors that are not belonging to a known or a
nation's Threadaclors, but we can cluster the attacks into one specific cluster.
So as you can see, he's already in the installation process and in the file, in the installation
process, he's also using PowerShell to download a binary and execute it. This time was a stealer.
So what can we do with this? Be wary of install time hooks, right? This sounds obvious, but it's
many Threadaclors often disguises or covered this in under like downstream attacks like dependencies.
So that attack is not so obvious, right? So one recommendation or mitigation, we can say,
first of all, 85% of the malicious package we observe during Q4, we're using this kind of attack.
So this is important, right? For picture containers, we can say, leave it to the usage of in
cell scripts as much as possible. We see times that times and times again that developers are
using the installed time hooks for greetings. Hey, thank you for installing this package.
I mean, okay, but there are other ways to achieve that, right? For end users, if possible,
completely disabled in cell script. For a PIPI, you can use wheels. This doesn't implement
set up like installation scripts, but for NPM, it probably will break stuff. You can only
version what are them for that matter. Garlic is the tool ways we showed earlier. So for
package repositories, one, you can use these packages using this, probably not very helpful,
but surely it can help in some cases. And also perform regular audits if packages are using this.
Okay, great. So the third thing we say a lot is the use of obfuscation to kind of hide
with these, what they're doing. So if you're a threat actor and you want to deliver malware via NPM,
I'll probably run into your right away, is that everybody can see the malware. So you need to hide it in
some way. So you can then pin like one thing we see a lot is the use of this obfuscator called
obfuscator.io, which produces this garbled sort of looking texture at the bottom. What it does is
it takes the input code and like one thing it replaces the variable names with random ones.
It removes the formatting, but then it also kind of packs the code in a certain way where it's
getting unpacked at runtime and actually being executed. So it makes it difficult to read
and difficult for static analysis to help you on. Fortunately, it's relatively easy to
partially unobfuscated or deobfuscated. And so indeed, that sample we were just looking at when
you deobfuscated it, deobfuscates to this in part. So this is two screenshots from an info
stealer called BeaverTail, which is a DPRK associated in Foceler. So you know on the lettering
that we have a second stage being downloaded and then on the right, so we have some data exploitation.
So again, you'll see you'll need to be on the lookout if you're like the user of a package
or if you're using a package to build something. There's just another example of this.
On the Python side, it tends to be a little different. So we see a lot of like decode or
an executed pattern, so base 64 or decompression or decryption or like character encoding stuff.
And it just goes to the point that like ecosystems have their own kind of obfuscation
tactics or techniques that people use. So the obvious recommendation is just say no to obfuscation.
There's no good reason to distribute open source code that's intentionally difficult to read.
So don't write it, don't use it, don't else it.
And the last thread she we're going to see is the minimized exposure by using a second stage payload.
So this is an example of how the second stage payload is used.
If in case you are not aware, malicious actors are using this thing that is called a second stage
payload which is deliver one small portion of like just a downloader of the real stuff.
So you only have to deal like for instance, in this case we're seeing this response example
which lures the victim to install the package as we discussed. It runs automatically the script
and then download the DLL and then runs it, right? The point here is that the DLL has the actual payload.
The only thing that was like in the package was the link to download it, right?
Again in architecture diagram, this goes into the use compromise dependency or use compromise
package. Here is an example of using power shell to download that button and stealer. Again,
the malicious payload is hosted in gear up, not in MPN or PIPI.
On our example, using again a fuscation, the method that I was talking about,
is concealing a bash script that's installed a service to run a minor.
So what are you going to do with this? Make sure your code is behaving, okay?
How easier say it than done, right?
So from patchment owners, we can say about using external runtime downloads,
it's harder. It's not always to do it, able to do this, but in several times we see that
it's used, it's overused, right? So for end users,
examine if I package downloads, have some of those buttons, like,
is, again, this is harder to achieve, but Garder can help out here. Otherwise, you can use a
container to somehow mitigate the exposure of the host. Yes, you can try.
So the thing called supply chain firewall. So with the tail end of last year, we released the
open source package that basically is just a drop in CLI tool to filter your NPM and PIPI
install or PIPI install commands, excuse me. So it'll inspect the install targets, it'll check
them against lists of known bad packages, or like, if their vulnerability is, it'll let you know about
that too. And then, yeah, it'll just, it'll kind of, if you set up your environment the right
way, it'll passively scan all of your commands. So if there's the PIPI install command, if you like,
and yeah, okay, it's one final thought. For patchment owners, don't be shady. That means
leave the bad guys do the weird stuff, try to do like the most obviously banning code behavior,
right? For end users, don't be an easy target. If you're installing something that you're on
aware of what it is, do your research check test, probably isolated, right? And for package
prepositories, lastly, increase the barrier to end 20 for predators. It's really, really easy for
predators to publish, bogus, or malicious packages. So there are a lot of good improvement
on opportunities there to prevent that for the techniques we saw today. And yep, that's it.
We have two minutes for questions.
Yep, sure. So go ahead, sorry.
Yeah, that's a great question. So the short answer is, that's kind of it. It's very early
days for this tool we hope to. Oh, I'm sorry, the question was, if packages are being taken
down quickly from these open source ecosystems, what is supply chain fire? Well, actually doing for
you. So you're right. Right now, it's kind of only good for, on the malicious side, for things
that are known to be malicious, right? So if it's been taken down the threat has been kind of
neutralized in some sense, these things do kind of like a voider around in different mirrors and things
maybe. There's, it also consults OSV.dev. So there's like a secondary function for pointing out
vulnerabilities. We hope to integrate like scanning capabilities into the tool later on,
maybe facilitated by guard dog. Yeah, that's where we are today.
Sorry, coming in. You'll focus on the unsupporting and the text port. Almost nothing will
be the example place. So there is any force about the plug-ins in the developer file, in fact,
in addition to the code. Yeah, so if I, I understand correctly, if it's like, we are focusing
a lot of on the package container, but if there's something we can do on the developer side.
So basically, we are, we're given like a couple of good tips for every single part, right?
For instance, for the developers, we said something about like, trying not to run or to provide
a sophisticated code, right? But this is like, this is something like, it has to be implemented,
this controller has to be implemented in all parts. Like, we can say to the developer, hey,
don't office care your code. We can detect it, but if the developers,
package containers or repositories are not preventing to upload to open source repositories.
Office-cated code is like, that's pretty much everything we can do, right?
I think we're out of time. So thank you.
Thank you.