The Rise of Open Source Malware

Jenn Gile and Paul McCarty from Open Source Malware (OSM) discuss the evolving landscape of malicious packages, born from their frustration with existing vulnerability databases like OSV. These databases, while excellent for vulnerabilities, lack the rich data needed for effective malware analysis.

Jenn notes, "It's already the largest source of data vulnerable components for open source malware, period." Paul elaborates, "You don't get any data about the author. You don't get any data about what the severity is. You don't get any data about what the thing does when it goes boom. And that's something that I wanted to provide." Unlike vulnerabilities, which might be accepted, malware in the supply chain demands immediate and aggressive action.

Developers: The New Primary Target

A significant shift in attack strategy has made developers the primary target for threat actors. Attackers now recognize the wealth of secrets, data, and crypto developers handle. This also extends to other roles within tech companies, like marketing and sales teams, who consume open source software.

Jenn highlights this shift, stating, "Threat actors are realizing what a wealth of potential data, secrets, crypto that developers have access to." Attackers exploit the common practice of not vetting dependencies and extensions, and the pervasive issue of secrets stored on developer systems.

DPRK's "Contagious Interview" Campaign

North Korea's "Contagious Interview" campaign targets developers by masquerading as recruiters on platforms like Fiverr, Upwork, and LinkedIn. During these fake job interviews, malware is delivered, often stealing cryptocurrency or gaining access to company environments if a work computer is used.

Jenn explains the broader implications: "But if you think about where are they interviewing from, if you or I are working for a Fortune 500 company and take the interview on our company laptop, then they can get in and get all kinds of additional secrets, company access, whatever." This campaign is a major source of funding for North Korea. Additionally, DPRK agents apply for remote jobs, potentially getting hired and gaining access to company resources.

Why Malicious Packages Keep Increasing

The surge in malicious packages is driven by several factors. Interpreted languages like JavaScript and Python are inherently easier for attackers to use, allowing rapid iteration and evasion of traditional sandbox detections. Registries like npm and PyPI also have weaknesses that make it simple to publish malicious code.

The fast-paced nature of modern development, coupled with the "upgrade to latest" best practice (intended to prevent vulnerabilities), inadvertently creates a fertile ground for these attacks. Attackers can release benign versions initially, then swap in malicious payloads after typical cooldown periods.

What the Industry Needs to Do Next

Combating this threat requires a multi-faceted approach. Transparency and data sharing are crucial. Registries must improve their defenses by verifying metadata, linking email addresses to user accounts, and encouraging developers to sign their commits to prevent tampering.

Organizations need to rotate Personal Access Tokens (PATs) regularly. Crucially, companies must develop comprehensive incident response playbooks specifically for software supply chain attacks. This is a new domain for many, as traditional AppSec and SOC teams may not have the expertise for these unique types of breaches. Paul shares an anecdote of a large enterprise having to block all npm access for six weeks due to a lack of proper response capabilities. This is a solvable problem, but requires collective action and a shift in defensive strategies.

Read full transcript

Mackenzie Jackson: Hello and welcome to another episode of The Secure Disclosure. I'm thrilled to be here with Jenn and Paul here at RSA. Always having fun. You guys are the founders of Open Source Malware, amongst many other things, but I thought it kind of makes sense to start there. Let's talk about Open Source Malware and how you guys came to kind of build this out.

Jenn Gile: Yeah. I won't tell the whole story because a lot of this is Paul's, but we've been friends for a couple of years and he came to me last year and said, "Hey, I've built this thing and it's starting to pick up. Let's talk." As we started discussing it, he shared the story with me about building an alternative to OSV for malware data, basically finding that there was not adequate information in OSV as a researcher and creating OSM. The letters, you're kind of picking up on what I'm letting down.

Mackenzie Jackson: I see. I see.

Jenn Gile: You see where that's going. We're not giving away malware to be clear, but it was kind of his solution. All good projects come from a sense of frustration. And so, he built this cool thing and it's hard to explain just how different it is from what's out there already. It's already the largest source of data vulnerable components for open source malware, period. So, packages, domains, IP addresses, crypto wallets. The community submits threat intel reports. They get human validated. We run our own things as well, and all of it available to the public. I think that's what's super unique in this space is a lot of threat intelligence feeds are totally proprietary.

Mackenzie Jackson: Yeah. Yeah. Yeah. For sure. And there are lots of different types of that, but there's some that are completely hidden.

Jenn Gile: And then you have to believe whatever numbers they tell you.

Mackenzie Jackson: But yes, but there are different levels, but you guys are completely open and it's completely community-driven. I know that you guys do a lot, but still a very community grassroots foundation. All products start from a point of frustration. So Paul, what was your point of frustration?

Paul McCarty: Yeah, I mean, like Jenn said, the problem was that OSV from a vulnerability perspective is a great, amazing thing. It's probably the best thing for going somewhere and asking, is this thing vulnerable, what versions are vulnerable? And that's awesome. But the problem is that we kind of assume that you can kind of stick malicious things in there too. We kind of assume because the atomic unit is a package that you can put malicious things in the same database as vulnerable things, and that's just not what OSV delivers. So their malicious package repository, their advisories, they're good for saying if something's malicious or not, but that's about it. You don't get any data about the author. You don't get any data about what the severity is. You don't get any data about what the thing does when it goes boom. And that's something that I wanted to provide. So, I basically took this idea of OSV and I made OSM, Open Source Malware, to be OSV with all that data in it and a good SaaS and a good API. Because the other problem with OSV is it's got a crap API, straight up. It's just a terrible API. You can't get things out of it. And so, I wanted to make our API super, super accessible. We've got like four or five endpoints, all are available to the free users. So yeah, honestly that's what it was, man. Let me take the things that OSV doesn't do well and do them well.

Mackenzie Jackson: Yeah. Yeah. No, I can see that. And it's not about OSV not having its place or anything like that, but it's about, no, we need something else in this.

Jenn Gile: Well, and if you think about it, the way that you manage vulnerabilities and the way that you manage or maybe don't manage malware, but deal with it, they are very different processes. Honestly, some of the things that you do for management actually can make you more susceptible to consuming malware. And there's not this broad understanding because they get lumped into the same category. Paul and I were at a founder pitch event this morning and one of the categories that the CISOs were talking about was visibility into third party and supply chain risk, and it was like, "But those are really different."

Mackenzie Jackson: Yeah. Yeah. Yeah. It's totally true. When it comes to malware as opposed to vulnerabilities, kind of what you're saying is that you have to accept vulnerabilities in your supply chain, right? You have to accept that they're going to be found there. You have to accept that you have ones now that you don't know about, that no one knows about.

Jenn Gile: Oh, you've got ones now that you do know about, right?

Mackenzie Jackson: And you accept all of that. But with malware, you can't. And in some ways, that kind of gives you an advantage in dealing with it. If you have a right tool, if you can say, I have malware now, we can block it or we can be aggressive on this. But that's the problem. And yeah, and how you deal with that and the vulnerabilities. I mean, we've all plugged in an SCA tool and it's given us 12 million criticals and then we've figured out that three of them are relevant. Right? With malware, it's kind of fundamentally different because it's...

Jenn Gile: Well, even the concept of, you know, with SCA and with reachability and looking to see what's actually being used in the application code, with malware you don't have to be using it in your application code for it to be a risk for you. And so again, it's like coming back to we lump these things together, but honestly, they diverge very quickly once you pick them apart.

Paul McCarty: And the problem is, like I said, people think about them as a package, so they think about the same metadata, but the reality is the metadata and the way you treat it for vulnerability is very, very different from when you look at it from a malicious perspective. Vulnerabilities, versions are very important, right? Most of the malware we look at, with a very few exceptions like the account takeover chalks and the debugs and what have you, most of them, all versions of the package are malicious, right? So you don't really care about the versions, right? The author is super important for a malicious package because then you can pivot on the author. Who gives a crap if you know who the author is for vulnerable packages, right? It doesn't matter.

Mackenzie Jackson: Yeah. Yeah. Well, let's kind of talk about some of the findings because there's been a fascinating pivot. So I've been kind of working with this with Aikido, working with Charlie, so we've been learning about this whole world and what's fascinated me about this a lot is that when I think of malware, and I think maybe this was true and you can correct me if I'm wrong, but you know, you deliver malware into a package with the hope of compromising a cloud system or something like that. Still true, but now it seems to be like developers are primary targets.

Jenn Gile: And it's really changed in the last couple of years. Threat actors are realizing what a wealth of potential data, secrets, crypto that developers have access to. And the nature, the way that they consume dependencies, consume extensions. You know, they know that they're not vetting them. And so, we see this big increase in threat actors pivoting to target developers as patient zero. I mean, we could even talk about they're starting to target other people in software organizations that don't write software. Your marketing teams, your sales teams, anyone who's consuming software really is at risk or consuming open source is at risk for this.

Paul McCarty: Like a lot of those malicious AI skills that we found in Claw Hub were specifically targeting kind of marketing people. So they were targeting TikTok and YouTube and stuff like that.

Jenn Gile: Yeah.

Paul McCarty: They're not targeting developers. They're targeting other people, right? Same malware, right? It still ended up being DPRK and other stuff, but...

Jenn Gile: Different secrets, you know, Salesforce creds instead of database creds.

Mackenzie Jackson: Oh, and it's so frustrating. So, I obviously have a bit of a history with secrets in my previous role. But it just seems to be a problem that we can't seem to get on top of at the moment. And when you look at all this malware, it's like, okay, you have malware on your system. When that actually turns into an "oh crap" moment is because you've got secrets on your system. It doesn't matter until there's something to steal.

Jenn Gile: Yeah. Especially these large breaches because now it's like they're not going to try and spy on your webcam or something like that. No, they're passing information as much as they can for secrets, low-hanging fruit to pivot.

Paul McCarty: Yep.

Jenn Gile: Well, and the reality that we've seen, the data shows that even after you've been popped, secrets are not getting rotated fast enough. Trevy is the perfect example of this, right?

Mackenzie Jackson: Yeah. Yeah. Oh my goodness.

Paul McCarty: Good segue.

Mackenzie Jackson: Yeah. I was delivering a workshop on basically how we can pipe together multiple open source tools like SaaS, like SCA, like all that. So to do it and then I had to stand up there and I was too late at this point and I was about to go on stage. Then you messaged me 30 minutes before to be like, "Just letting you know, Trevy's been popped again." I was like, "Yeah, again." Because I had just sent out a message to everyone to be like, "Hey, Trevy's been popped. Here are the version numbers. If you followed the instructions of the worksheet, you're probably fine, but here's what you can do." And then I had to stand out there and say, "Okay, disregard that last email. Let's not use Trevy right now."

Paul McCarty: And by the way, when you're watching this at home, we're talking about the third time they've been popped. By the time this airs, there might be a fourth time. Or a fourth time.

Mackenzie Jackson: And Checkmarx is now...

Paul McCarty: Yeah. And Checkmarx. Yeah.

Mackenzie Jackson: Now. So, yeah. And the only people happy about this are Delve because now we're not talking about Delve on the podcast. But we'll move on. We'll move on from that. I want to go back to this area of developers because you guys are kind of the leading experts and this is what you've been putting out there about the DPRK campaigns for contagious interview. When we find something like that, you guys are the first call for us to say...

Jenn Gile: Yeah.

Mackenzie Jackson: I don't actually think a lot of people actually understand how this works because it's quite interesting. It's quite different. So, how does the contagious interview process and everything kind of happening? It's about targeting developers now, but in a pretty sophisticated way.

Jenn Gile: Yeah. Let's start one step higher and talk about who's doing this and why.

Mackenzie Jackson: Yeah.

Jenn Gile: So, it's North Korea and they're targeting developers, and we'll say there are actually two arms of it. There's one is the contagious interview route and one is the DPRK workers route. And so they're targeting us mostly, but recent data we've seen shows it is global and it is funding their economy. Last year, what did you say? 2.2 billion.

Paul McCarty: Dollars in stolen crypto. Yeah.

Jenn Gile: So, contagious interview has been going on for what, maybe two years or so.

Paul McCarty: Well, technically 2023, late 2023.

Jenn Gile: Time is a weird thing.

Paul McCarty: Yeah.

Jenn Gile: It's still 2008. But it's been going on for a couple years. And the whole concept here is somebody pretending to be some kind of recruiter, who in fact is a DPRK threat actor, reaches out to a developer through a platform like Fiverr or Upwork or even LinkedIn and says, "Hey, love your profile. Would love to interview you for a job." And so they're going through the interview and at some point, it's evolved over time, they get sent something that will have malware in it. And that malware will have, you know, it steals crypto and it gets into their environment and sees what else they may have there because let's be real, a lot of people take interviews on their work computers.

Mackenzie Jackson: Yeah.

Jenn Gile: And so, where I think people have been shortsighted is thinking, "Oh, this is just targeting people who are interviewing, it's just taking their crypto." But if you think about where are they interviewing from, if you or I are working for a Fortune 500 company and take the interview on our company laptop, then they can get in and get all kinds of additional secrets, company access, whatever.

Mackenzie Jackson: Well, one thing I was kind of curious about though too is like the idea of kind of having like a sleep agent in there unknowingly. Like I don't know if this would work, but if you had an employee, someone do an interview and then eventually they end up in the government.

Paul McCarty: That works the other way. So what they say is anybody who has remote job postings right now, DPRK agents are applying for those jobs and interviewing for those jobs.

Mackenzie Jackson: Okay. So, okay, so...

Paul McCarty: That's the other arm and they have gotten hired places. There are ways to detect both, right? So are they primarily doing it to be sleeper agents? Honestly, no. It's a money-making scheme, but the side effect is they're going to, you know, you're going to send them a company laptop, right? Yeah. You're going to give them access to all kinds of stuff. Yeah.

Jenn Gile: You're probably hiring them to be a developer, so presumably they're going to have some creds. So, yeah, both routes are offering lots of both things of explicit value to DPRK as well as things that they can leverage for the next campaign. Who knows where some of these secrets have come from that have given them access to get into GitHub repos, start the next thing?

Paul McCarty: Well, I mean, and let's be honest, this isn't an existential thing for North Korea, right? Because they've been sanctioned into a corner. They have no way to make money legitimately. Their GDP has now been replaced by stolen crypto, straight up. $2.2 billion last year and like six billion dollars in the last four years. Like that's a lot of money and that's been keeping their ICBM program alive and a lot of other things. So, it's real serious business for them and they are now doing it at scale.

Mackenzie Jackson: And they've gotten really good at it. Last year there was the Bybit attack where they managed to steal a billion dollars in one go from that.

Paul McCarty: And the thing that makes me so like so crazy about that attack is that they remained hidden for so long active in the environment because they knew eventually a big fish was going to move some big money around.

Mackenzie Jackson: They could have stolen $2 million.

Paul McCarty: Yeah, very easily.

Mackenzie Jackson: But be patient.

Paul McCarty: But yeah, be patient.

Mackenzie Jackson: Which, you know, someone could have found it and blocked it, but it was like, yeah, that is. And as far as I know, the money's never been recovered or at least no. No.

Paul McCarty: But I mean, every time we post about contagious interview, you get DMs from people who have been popped by it.

Jenn Gile: Who said, "Oh my gosh, this happened to me."

Paul McCarty: And I think that's part of the problem, Mac, is that people don't see it. Like when people don't realize that there are real people out there that are getting both their own personal crypto drained and any crypto for any organization they work for drained too. People hit me up and they send me screenshots of the LinkedIn people and the person owners and then I can go and find those people and pivot on those, pull that into the database, into the OSM database and enrich that data. But the problem is people don't see that, right? They don't see the fact that this is actually generating revenue. And the problem is especially bad when somebody's like a freelancer, when they're not working for just one company or working for multiple companies. And that's why Fiverr and Upwork are such vibrant, rich sources to put them off. So they're not getting one GitHub organization. They're getting like 15. And I've seen it personally. I've seen that happen.

Mackenzie Jackson: Yeah. So how do we actually start to combat this in like where, like what, what's, because we've kind of talked about a couple of different things here already. We've kind of talked about the package registry type stuff and then we've talked about this which isn't related in terms of it could be related but but like, you know, they're getting, they could probably get malware in too. So...

Jenn Gile: I think there are lots of things that need to happen and we're behind. As a society, we're behind where the threat actors are. Certainly some of it is transparency like what Paul's talking about. If you don't know these things are happening, how are you going to be vigilant for them? I talked to somebody at BSides SF this weekend. The DPRK workers thing came up in conversation and he was like, "That literally just happened to me a couple weeks ago. I had somebody interview with me who gave me a reference and I called that person and they said no, this person does not exist." So, you know, some of it is just transparency and data sharing and to the extent that your legal department will let you share this information out, like threat actors are not protected, you know, it's not protected PII. If you get this so some of it's there. I don't know Paul.

Paul McCarty: Yeah, that's a really good point. So one of the frustrations we've had is that when we deal with some of the platform companies, whether it's GitHub, GitLab, or whoever, cloud providers, is that when we come to them and we prove that somebody is malicious. Here's a campaign. Here's hundreds of repos or whatever the case may be. They don't want to share data back to us to help us, you know, kind of tie those things together. And that's unfortunate. So we need to get better as a culture at saying, hey, listen, if we identify that somebody really is malicious, then let's share that data together, right? Because then that can help us protect ourselves and the whole ecosystem against those threat actors.

Jenn Gile: Well, and to be clear, the individuals that we engage with, they do want to share the information. It's legal departments. Often this is a new thing that's just not covered.

Mackenzie Jackson: Yeah.

Jenn Gile: You know.

Mackenzie Jackson: And there are a lot of businesses that are built around this too, you know, and I guess that's something like, you know, and I kind of say that from someone that works for a company that does have, you know, that does have some interest in this. But I guess like one of the great things about open sourcing something like this is that you devalue everyone's private data, you know, kind of in a positive kind of way, you know, like and then we can all stop the little squirrels about it. You know, we publish ours publicly, so I like it's not, it's not quite.

Jenn Gile: You're right. And I don't think it's an explicit goal of ours. But I think what's been super, I think inspiring for me, is some of the people, community researchers that we're working with who are contributing to Open Source Malware, we can then watch them use that data and publish research using it that helps them move their security programs forward, right? And then they contribute it back. And we have this very organic relationship with them. So I think, you know, there are some interesting things evolving there. I also think as an industry, we need to have some serious conversations about what it means as a company who has a security program. How do you actually deal with open source malware? Because like you mentioned earlier, we're kind of used to different types of malware or malware through a fishing type of vector. They're not really used to dealing with malware coming in through the developer or the employee as a software type of thing. And so we don't really have programs built around that. We don't have a natural place where that sits in the organization. There's some inclination to have it sit with application or product security because we think, oh, well, it's applications, so it should sit there. It's code and, you know, to some extent they definitely have a role in it in working with engineering in reducing the probability that they consume this stuff, you know, encouraging them to pin dependencies, you know, set cool down periods. But fast forward, you got popped. Your apps person can't do the investigation, you know, they're not going to have all of the experience that's needed to actually go deep on this, that's going to live in your SOC and your IR and your threat. You know, to say that it's apps responsibility is shortsighted.

Paul McCarty: Because there's a proactive component of this that's really outside the application. Kind of if you flip it over to the SecOps and the InfoSec teams too as well, like they think when they hear malware, they're like, "Oh, sweet. Our EDR has got it covered." Right? And the reality is that 99.9% of this is interpreted code, JavaScript or Python, right? I've got CrowdStrike on one of my machines and Sentinel on the other and I've got like I don't know 15,000 malicious, you know, samples on my machine and it doesn't detect any of them, right?

Jenn Gile: So the problem is by using that term malware, right? It's generic, it's too generic, right? That's why we really try to hammer home this idea of open source malware. Let's get into the kind of like the really open source stuff and how it's and how it's really changing. There has been such a dramatic increase in the amount of packages on places like npm and I know PyPI is not as big but there's still an awful lot of malicious packages on PyPI.

Mackenzie Jackson: Yeah. What's driving this big increase? Is it that the attackers have discovered a new vector and they're just like, and they've had some big successes, right? And there's been some really close calls that we just got away with. Somehow is it like it kind of looks like it could be AI related? Is it the ability that more people can create AI? Now...

Jenn Gile: I want to start with interpreted malware and how it's or interpreted languages and how it's different and what an advantage that gives. You know, we're used to as a security culture looking at malware that you can sandbox and detonate and that's how you manage it. You know JavaScript, Python malware, it doesn't do that.

Paul McCarty: It's file hashes like because you'd have to compile it and you deliver the artifact, the binary artifact, right? And it takes a while for you to create another one. There's, you know, there's a latency there. Whereas with interpreted software, they can just push a new npm package. Boom. Boom. Boom. 17 versions a day.

Jenn Gile: And so by default, just because of how these languages work, this malware is easier to make. It's more likely to be able to get around your legacy practices if you haven't caught up and realized that your sandbox isn't going to catch this because this malware is, you know, I don't want to say smart because it's not intelligent, but it can recognize, oh, I'm in a sandbox. I'm going to chill out, right? I'm not going to detonate. So, that's some of it.

Mackenzie Jackson: It's got very like you can adjust the logic very easily based on it. Yeah.

Paul McCarty: And I think the problem too is a lot of people might think that post and pre, like for npm for example, like everything depends on post and pre-install. That's the low-hanging fruit, right? Like the good stuff is all on import. Like, you know, last year I found a package web3 parser had been on npm for years. It did what it said on the tin. It actually parsed web3 events, but into where you told it to, but it also sent them to the bad guys as well.

Mackenzie Jackson: Yeah, because like the pre and post install scripts are kind of like you're going to roll this out. You know, you've got a limited window from the moment, right? Because you've found it, we've found it, you know, all the other teams have found it and we're like and we're fighting each other to post first on Twitter. But...

Jenn Gile: So like let's start there. The malware itself was different. We'll call a spade a spade. NPM has some issues.

Mackenzie Jackson: Yeah. Got lots of issues.

Jenn Gile: Lots of issues. And so it just makes it incredibly easy to publish malware. You know, whether you're publishing novel malware or whether it's an account takeover. And let's talk in a second about the changes that have happened to kind of help prevent ATOs that don't, I mean they can help. I know I'm being a little dramatic. It's a podcast, darn it. So yeah, there's, it's just easier to publish malware there. And then you look at again the way that we consume software now, the way that we consume open source, it's heavily biased towards speed. You know, we are past the point of we update our software once a year or every six months or monthly. Like for a lot of tech companies, it's weekly, daily, hourly. So the velocity that things are getting pulled down is just so fast.

Mackenzie Jackson: And there's a serious friction point that we're both well aware of, anyone working in security is aware of, that the developers don't want to be slowed down by security, you know, getting in there and that's fair. They have their own KPIs.

Jenn Gile: Yeah. And so there's just this perfect storm. Oh, and the best practice in software is upgrade to what? Upgrade to latest, right? Yeah. And so there's just these things that make...

Mackenzie Jackson: To prevent vulnerability.

Jenn Gile: To prevent, to reduce your tech debt. Totally legitimate. Yes. Like these are good things. You should upgrade.

Mackenzie Jackson: Um, but also, you know, immediately pulling down a release within, you know, seconds is not great. And so it's all kind of coming together to just make this like a nice place for threat actors to publish malware.

Paul McCarty: Yeah. And on the speed thing too is like we kind of know so with the debug chaw compromise, some of these were shut down within six to eight hours and...

Mackenzie Jackson: Yeah. Yeah. And Wiz found out that it was up 10% of their cloud that the actual...

Paul McCarty: I didn't hear that stat. And from first notification, well, from when the malware was published to when Josh regained control of his account and deleted the versions and all of them were gone within six hours.

Jenn Gile: Yeah. And let's be real, he caught it pretty quickly.

Mackenzie Jackson: Yeah.

Jenn Gile: And it took some time for npm to help shut it down, which...

Paul McCarty: And even with the Trevy thing, it turns out the Brew install wasn't 0.69. Okay. Which is great.

Mackenzie Jackson: So, you're at my workshop.

Paul McCarty: We're good. In July, during that small time period, 700 people downloaded that version, right? Do you like what risk that is to pull the latest version? And so, everybody's response is, well, let's add cool down periods, right? Yeah. Well, here's what I'm going to do as a bad guy. I'm not going to put my malicious crap in until after the cool down period, right?

Mackenzie Jackson: I'll put my payload. They'll swap it out later.

Jenn Gile: Yeah, it can be a totally legitimate thing that does what it says on the tin. I'm going to, I'm going to wait, you know, for seven days after it or...

Paul McCarty: But also like legitimately if it's an ATO, probably the maintainer is going to realize it at some point. Like I think cool down periods are good, but...

Jenn Gile: Sure.

Paul McCarty: Like all things, life finds a way. I think, yeah, I think the problem is that we're focusing a lot of these big kind of account takeover attacks, right? Which are huge and have big high-profile...

Jenn Gile: The reality is that lots and lots of these new packages, these like, you know, not pre-existing packages, are actually compromising people as well.

Mackenzie Jackson: Yeah. And we don't really talk about those and we don't spend a lot of time scanning those because we think all the SCA vendors are all spending time scanning the high impacts.

Jenn Gile: Well, that TensorFlow package that we had a community member report a couple weeks back. Somebody reported a TensorFlow package that's been around for what, four years or something like that.

Paul McCarty: Oh wow.

Jenn Gile: Cryptominer straight up in it. Obfuscation.

Paul McCarty: And it had a lot of weekly downloads. It was getting used. People have been using it for...

Jenn Gile: PyPI has got some things they need to do too as well. Right.

Paul McCarty: Right. Right. Right. Okay. So solutions from this like what do we need to see in the next couple of years? Is it the responsibility of the package registries to help solving this? Who's like how do we support? How do organizations support like what do we need to do to help this?

Jenn Gile: I, so that's a great question. I think that the GitHub and npm and PyPI all need to up their game, right? I think we all realize that. I mean, for God's sakes, GitHub and npm are, I know I've bitched about that on your podcast before, but the reality is it's still true now as it was the last time I said it.

Paul McCarty: Well, it's a solvable problem. Yeah, it is absolutely a solvable problem. Unfortunately, the somehow that the team at Microsoft, the security team keeps getting smaller and smaller and smaller. Here are the specific things that I would suggest. I would suggest that both npm and PyPI implement policies that the metadata can't be fake. So basically what I mean there is like and when you make an npm package right now, it basically asks you the author and the contributors and you can just put anybody in there you want to, right? Yeah, it's fake. PyPI similar thing but like basically they don't tie an email address to a PyPI user in the metadata, right? So without that you can't tie that package to other packages inside of PyPI. Both of those things are easy things for those package registries to fix. You know, and then developers themselves, they should be signing their commits. This whole Trevy thing, right? And like the checkboxes thing. If people had been signing their commits, a lot of that wasn't attackable because they can't force push over the top of a commit. So there's things that both developers can do and there's things that the registries themselves can do and we just need to get better at it because what we're doing right now is not working.

Mackenzie Jackson: No. No. Like it's definitely not broken and it seems like a solvable problem. It seems like we can, we can...

Jenn Gile: Well, you know, the reason that the recent Trevy compromise happened is traced back to a personal access token. Rotate that stuff. Don't let it sit around. I know. See, it all comes if it's not DNS, it's secrets.

Paul McCarty: Well, the Argon Vodica account is from 2023, right? Right. See, correctly. Jesus.

Jenn Gile: Jesus. Yeah.

Mackenzie Jackson: Yeah. No, terrifying. But I think but it is solvable. I think we will, you know, we will get there and hopefully we can. But at the same time, we solved SQL injection in 1998 and we still have but so I don't know if we have the best track record. Yeah. Hi. Well...

Jenn Gile: I think like as an industry, we need to demand some accountability from these repositories, these registries, and then, you know, it's really like thinking through this at an organization level, threat modeling it, like the, you know, standard security stuff.

Mackenzie Jackson: Yeah.

Jenn Gile: Have a playbook for if you get popped. How are you going to handle this? How are you going to find out who the threat actor is, what their next step is, how bad is it going to be? Who are you going to call?

Mackenzie Jackson: Do you have to call your cyber insurer or not?

Jenn Gile: Yeah. Yeah. Seriously. Like, do you have to call legal or not? Right? You know, we're talking to to large enterprise users and they, you know, they've been popped before and they they've like, "Wait, we've never done incident response for a software supply chain attack. What do we do?" Right? It's just a new thing for them.

Paul McCarty: I mean, I'm going to give you an example. I talked to somebody who's at a very, very large enterprise. AppSec is reasonably new to this company, a couple years old, and he shared with me that during Shy Hallude, they had no way to shut down access to compromised packages. So, they pulled access to npm until they could purchase a product that would let them do that, right? It was not days, right? It was like six weeks of no access to npm.

Mackenzie Jackson: That's going to cause some issues. Yeah. I have to play a fun game. I'm very curious to hear what your response is. It's called Would You Rather? Yeah. I'll give you two really crappy options and you have to pick one from it. So the first one, Jenn, we'll start with you.

Jenn Gile: Okay.

Mackenzie Jackson: And I'm going to ask you to pull the same one so he gets a moment to think about. Would you rather auto update all of your packages or pin and not be able to change the pinned package version? So you're stuck forever with current versions or you automatically pull?

Jenn Gile: Oh my god, those are terrible choices. Yeah, that was the kind of like the point of the game. I think I would choose auto update with an asterisk of we need to be doing more as an industry to support open source projects and so you know auto-updating there's risk of breaking changes right and there's a lot of risks that are not malware when you're auto-updating but I think I would probably go with okay let's auto update there's probably something else that we can do as an industry to deal with malware I think I would choose that over tech debt Okay. Oh...

Paul McCarty: I'm going to surprise myself and say pin.

Mackenzie Jackson: Yeah, I didn't like, I had to really think...

Paul McCarty: Until you heard my terrible answer.

Mackenzie Jackson: No, no, I was having my own internal kind of dialogue about it. Um, yeah, oddly enough, I would pin it.

Paul McCarty: Yeah, I don't know what I would do. No one's ever asked me. So, hopefully that would...

Mackenzie Jackson: What would you do? Well, there actually was, I would pin as well, but I'm going to steal someone else's answer. Okay. They said I'm going to pin because I can input other mitigations and if I have a vulnerability, if I have a vulnerability in this, then I can try and harden that. Whereas I, I have never thought about that. So I'm going to clip that, make myself sound smart.

Paul McCarty: Here's the other thing is that like we know that only like a small percentage like five or four, whatever, 8% of these vulnerabilities are actually exploitable, right? Because of reachability. So, and you know, I don't want to get into this whole kind of reachability thing, but the reality is that if you pin, not necessarily all the theoretical vulnerabilities that are in those pinned older versions are absolutely slow.

Mackenzie Jackson: Yeah, it's like less. All right, another one here. Claudebot. You have Claudebot with overprivilege in your systems.

Jenn Gile: Okay.

Mackenzie Jackson: So, let's just say access to GitHub repositories, access to cloud accounts, or all of your secrets are stored in a text file in a private GitHub repository. It is private. So...

Jenn Gile: Okay.

Mackenzie Jackson: Where would you run?

Paul McCarty: Wait, wait, let me just understand the question. So the two choices are run Claudebot with all my AWS credentials?

Mackenzie Jackson: Yeah. Or your GitHub access to like something like this very highly privileged Claudebot or put them all in a private repo?

Jenn Gile: Yeah. Yeah. Claudebot is like, it's a no.

Paul McCarty: That's a no-brainer. Okay.

Jenn Gile: Thousand times over, I'd rather put it on.

Mackenzie Jackson: All right. I'm going to change my, I'm going to change my question now because that was resounding. That was resounding. It wasn't that much of a terrible option. Same problem with the secrets in a private repository but in a text file or no firewalls on your application. So everything so no way to kind of filter out malicious traffic from...

Paul McCarty: True. Well, I still control the operating system, right? So I basically control what services are running on the operating system. So, um, I'd go with, I think I haven't fed this through very well, but I think I would go with the latter, which is no firewall.

Mackenzie Jackson: I like that I'm getting, I like that I'm getting this is the goal. It's not as easy as Claudebot.

Jenn Gile: That was too easy.

Paul McCarty: Maybe, maybe still do an MD file. Actually, I'm going to go with the MD file.

Mackenzie Jackson: Okay.

Jenn Gile: I'm inclined to too because like the idea of crashing my website, my service with...

Paul McCarty: Yeah.

Jenn Gile: Just a boatload of traffic, you know, whatever.

Mackenzie Jackson: Yeah. Because at the core of the question, you're really thinking about like security versus availability.

Jenn Gile: Yeah. Yeah. You know, like so...

Paul McCarty: There's a lot of other components that come in around the firewall. There's a lot of other things. It's like are the, what, what network interfaces are, you know, the ports attached to and then a bunch of other things there that like mitigate that. But I would still go with the dot.

Mackenzie Jackson: You're complicating it too much. You're thinking too deeply about I'm a network guy in the 90s, right? This is all like, well, I'd love to have you chatting you here. Um, I'm sure I'll get you back on the podcast because I keep asking and keep showing up. So that's...

Paul McCarty: Thanks for having...

Mackenzie Jackson: Yeah, it's fun.

Jenn Gile: Yeah, that's awesome.

Mackenzie Jackson: And yeah, can't wait to see the evolution of Open Source Malware and keep killing it.

Jenn Gile: We weren't understood. People assumed we're a criminal. Yeah, we kind of liked that in some ways, but it limited our ability to actually do things that helped if that makes sense.

Suggested episodes