Mackenzie Jackson: Hello and welcome to The Disclosure, the show where we bring you the latest and greatest in cybersecurity and tech news. We have an amazing show planned for you today. We're going to be diving deep into the Microsoft SharePoint breach and we've invited onto the show the Chief Hacking Officer at Eye Security, the company that discovered this being exploited in the wild. We're going to dive into exactly what on earth happened and whether or not a patch by Microsoft actually made things worse. We're also going to be looking at what on earth is going on in NPM at the moment with a massive increase in supply chain attacks. We're going to be joined by Charlie Eriksen, malware and supply chain researcher. And of course, I get to sit down in the Cyber Bar, drink some sake with Berg Severens, who is an AI researcher and founder. We're going to be going into how AI is affecting security and vulnerability scanning. This is a fascinating topic and a really interesting subject. If you have been living under a rock in the last week, you may not have heard that Microsoft SharePoint had a pretty catastrophic vulnerability that was being exploited in the wild that allowed for remote code execution or basically complete takeover of SharePoint on-premise servers. We saw large organizations like governments and financial institutions being targeted by what we suspect are nation states. This is a pretty worst-case scenario for a lot of organizations and it's still ongoing, but it could have been a whole lot worse if it wasn't caught early thanks to the analyst and research team at Eye Security. So, I invited Eye Security's Chief Hacking Officer Vaisha Bernard onto the show to discuss in technical detail exactly what happened, how they uncovered it, and what actually went wrong because this vulnerability was actually known about all the way back in May this year. So without further ado, let's dive into it with Vaisha Bernard, Chief Hacker at Eye Security. Welcome Vaisha, thanks so much for joining me here on the show. It's a pleasure to have you. I thought we'd jump and get straight into it. You have one of the coolest titles that I've seen for a while. You're the Chief Hacker at Eye Security, which I love. I don't know how one gets the name Chief Hacker, but maybe if we just start off by explaining a little bit about this crazy week that you've just come through and what exactly unfolded with the SharePoint incident. You're the Chief Hacker at Eye Security, which I love. I don't know how one gets the name Chief Hacker, but maybe if we just start off by explaining a little bit about this crazy week that you've just come through and what exactly unfolded with the SharePoint incident. ️Yes. So, yeah, I am the Chief Hacker at Eye Security. We are a Managed Security Service Provider. That means that we monitor the networks of our clients to keep them secure. We're also doing incident response and we combine that. That means that if we see something happening at one of our clients that looks malicious, we're also the ones to respond to that, to interact, to stop the threat in place as soon as possible. And that's exactly what happened. We saw that at one of our clients, an exploit was taking place at a SharePoint server. At first glance, it looked like that a webshell was written to disk, an ASPX file that contained code to extract machine keys. And this was the first sign that we got and we didn't know about any recent SharePoint vulnerabilities. This came in at our analyst on duty because it was after office hours. Immediately her spider senses started tingling. This is something strange, right? She never saw SharePoint exploitation and it was on a SharePoint server. She called in her second line to have a second opinion and he also thought this is definitely something that should not happen there. But at first because we didn't know of any recent SharePoint vulnerabilities, we started looking at a different angle. We thought maybe the VPN was breached. We've seen a lot of VPN breaches in the last year and maybe it was lateral movement by an adversary. But it didn't feel right. So we helped the client recover as soon as possible. But as we do in such measures when we see active exploitation, we first isolate the machine to make sure that no attacker can interact with the machine. We have a little more breathing space to do a forensic investigation, look into what exactly happened. And that's when he called in the research team. This is something bigger than we know. I can't get a clear picture of what's happening. So, please could you help investigate? And this was at 11:30 PM local time and of course on a Friday night. ️Mackenzie Jackson: What else would it be? ️Vaisha Bernard: It's always those time frames. And I remember I just came home from dinner with friends and we had a couple of glasses of wine and I was sitting on the couch just before going to bed and I read, I read on our emergency signal app that something is going on at a client that can't be determined exactly what's going on and we started digging into the details. And that's when we found that only 10 days earlier the Zero Initiative released information about a CVE that was patched by Microsoft on the 8th of July and it contained a little information. It contained this tool pane endpoint that was used in the exploit and we recognized that as being exploited at our client. So our first hunch there was that this new vulnerability was exploited and the client didn't patch yet. That is very possible in 10 days' time. Not everybody patches in 48 hours as we always recommend. But then we dug deeper into the vulnerability. Everything that was stated about the vulnerability was that it wasn't a remote code execution vulnerability. It was, I think Microsoft classified it as a spoofing vulnerability with a CVSS score of 6.5. So it didn't look important from what we could find. But we noticed that our client it was definitely remotely being exploited without any authentication taking place. So in our opinion, this should have been a score 10. This is the most critical type of exploit out there. And that's when we realized we're dealing with a zero day here. This is something new, something that is exploitable directly from the internet without user interaction. And that's when we also realized if we see this at one client happening, what is going on? And that's also when we immediately kicked off an internet-wide scan to see if we could identify more victims because what we saw at our client was that it left an artifact that was reachable from the outside. So that meant that we could identify other servers that had been compromised in the same way. So we did an internet-wide scan immediately found dozens of compromised servers belonging to government agencies, universities, companies in the vital sector in the energy sector worldwide. And that's when we realized this is big and we need to warn everybody. So we sent out emergency messages to worldwide teams to warn them of what was happening. And we also immediately started writing down everything to inform everybody who needed to take action at that moment. That's when our blog was released and we kept working through the night to update the blog to update it with all the information coming in, information from the local government who also responded by that time with some more information. And yeah, from there on, everything is public. So that's when people started waking up somewhere on Saturday and thinking something is going on. ️Mackenzie Jackson: Yeah, I must say that blog is fantastic and I think that it's responsible for preventing a lot worse, or a lot of more breaches from this, because it was very spot on. You mentioned something in there that there was a previous vulnerability that kind of used a very similar exploit path that used the same kind of endpoint, that tool pane endpoint. That vulnerability was first discovered in Pwn2Own in Berlin. It was reported, it was patched, I believe, July 8th or July 9th. That's what you're talking about. Is this the same kind of vulnerability and I guess the underlying question is, did Microsoft release a patch that perhaps solved one exploit path but didn't actually solve the problem of it? Or are we, are they similar but different? ️Vaisha Bernard: Yeah. So we dug into it and what's the, so the original vulnerability was an unsafe deserialization vulnerability in SharePoint where attackers were able to execute malicious code by embedding them in payload in an Excel data set object in an XML file and sending that to the SharePoint server. That was only possible if you were the SharePoint administrator. But a second vulnerability was actually an authentication bypass vulnerability where if an adversary was able to send a specific referral header in the HTTP request, it was possible to deliver the first vulnerability to the correct backend endpoint. This tool pane endpoint that was discovered during Pwn2Own. And looking at the timeline now, it's interesting to see what happened because it is now known that the first active exploitation of this in the wild took place on the 7th of July, one day before Microsoft released the patch. We don't know if Microsoft knew that. It could be possible that Microsoft knew that it was being exploited in the wild and that they released the patch on the 8th whilst it was not complete, whilst it was not perfect. And looking at the patch now and dissecting the patch, we see that it was also, it looks a bit like patchwork, like very rudimentary patchwork. So the way they patched the first vulnerability, the unsafe deserialization, was in a way that it actually required a configuration reload of these SharePoint servers after the update, which was not mentioned by Microsoft, which meant that everybody who installed the patch was still vulnerable without knowing, as nobody reloaded the configuration. And the authentication bypass actually was also quite rudimentary. It checked the referral header if it ended with toolpane.aspx and if that was the case, it denied the request. But HTTP works in a way that if you add a slash after a file, it still accesses the file. So if you simply add a single slash after the ASPX to the toolbane referral, you could bypass the fix and still completely exploit these servers even though they were patched for both vulnerabilities. That was, I think, on the somewhere on the 17th, that was discovered by a security researcher who actually asked Google Gemini for a bypass and Google Gemini came up with this bypass. ️Mackenzie Jackson: Oh my god. Okay, okay. Wow. That's crazier than actually I thought. ️Vaisha Bernard: Yeah, it's a crazy story. And it might be that this was read by the adversaries who did the mass exploitation attempt on the 18th and 19th of July. And then on the 21st, Microsoft released the new patch which looks a lot more solid than the original patch. So now it really, they really rewrote the whole validation of this deserialization routine in the backend of SharePoint. They did not do that in two days. So they probably already started doing that earlier and the check for the referral header is now also a lot more solid in a way that it is not possible to bypass it. So, it's funny to see that the original patch actually looks hasty while the emergency patch released on the 21st looks more solidly built. That's just an observation here. ️Mackenzie Jackson: Yeah, that is really interesting because I also, because what I also was wondering is scrolling through Twitter, I did, or X, whatever you want to call it, I did see a tweet from someone around July 14th that said they were able to reproduce it and they specifically mentioned using the patch, the diffs in the patch to be able to recreate it. So that kind of led that, and from everything that you've seen, it says that they patched something but I think it would have been, but using the patch and using Gemini you could basically figure out, so I guess unintentionally Microsoft made the situation a little bit worse, although as you did mention it had been exploited in the wild beforehand. ️Vaisha Bernard: And that is the most pressing question at the moment is how did an adversary come to know this vulnerability and abuse it on the 11th of July already before any patch came out. ️Mackenzie Jackson: Yeah. And we don't have the answer to that question. ️Vaisha Bernard: And there's only speculation that you can pull from that which we won't go into. But yeah. Yeah. ️Mackenzie Jackson: Now let's talk a little bit about the threat actors themselves. What again we're diving into a little bit of speculation, but I think some attribution like some attribution has been made. Do we know what threat actors were exploiting this in the while at scale? But I know now there's probably going to be more looking for these vulnerable SharePoints. That initial kind of attack on government. Do we know who they were? ️Vaisha Bernard: Yeah, we're relying on the attribution that the big threat intel firms put out there. Multiple of them have now attributed this attack to Chinese based actors. Yeah, we don't have more information than these organizations. So, we have to rely on their assessment of this. But it is, it is quite likely if they assess this to be Chinese-based actors, it has all the characteristics of it. We've seen a very similar pattern four years ago with the mass exploitation of Exchange that was also attributed to China. ️Mackenzie Jackson: Yeah. And I think this is kind of like the, yeah, the scary thing is that I guess the initial targets would have been big boys, big governments, financial institutes, areas like that. Yes. But now we're seeing people that are probably less prepared to handle these types of attacks as they come in. Yes. What can those people do? So you're running SharePoint, you're running it on premise with the affected. What is your advice to them? ️Vaisha Bernard: Yeah, first of all, if you haven't updated yet, do it immediately. Then rotate your machine keys. And third of all, if you haven't updated immediately, also do thorough forensic investigation on your SharePoint server because there are now countless places that backdoors could be hidden already by adversaries and they could very definitely be already in your network. You probably also want to deploy a solid monitoring detection solution to catch these adversaries if you don't have one already. ️Mackenzie Jackson: Yeah, absolutely. And I guess too, if to have a good company like yourselves looking on your back if you don't have the expertise to do this in-house. And I hope to have you back but let's hope maybe under different circumstances after not quite such a worldwide incident. ️Vaisha Bernard: Yeah, so that we have some interesting things coming up in the next couple of weeks. So I'm actually presenting at Black Hat USA in two weeks' time. So there's more content coming. ️Mackenzie Jackson: Excellent. I will be at Black Hat myself. Do you know what day you're presenting so we can give a shout out to the audience? ️Vaisha Bernard: Thursday 1:30. I'll be presenting what I call Consent and Compromise, how I found a new type of misconfiguration in Entra ID that I was able to abuse to access 22 different internal Microsoft services. ️Mackenzie Jackson: Awesome. I'll provide a link to that talk in the description if anyone listening wants to check it out. Thank you again for coming and hopefully I'll get to see you in Vegas. ️Vaisha Bernard: Yeah, come say hi. ️Mackenzie Jackson: Now, the other big news event that we've been seeing is a lot of very popular NPM packages, Node packages being poisoned with malware. And these are packages from really popular and reputable maintainers whose accounts keep getting compromised and malware added into these packages. So to uncover how on earth this is happening and why we're seeing such a large increase in the last couple of weeks. So I brought onto the show Charlie Eriksen who is a malware researcher that specializes in detecting and finding malware inside open-source packages. Charlie, thanks so much for joining us and welcome to the show. ️Charlie Eriksen: Thank you. It's great to be here. There's a lot going on at the moment, but it appears that there's been a lot of elevated activity in terms of NPM packages, Node packages being turned malicious. Can you talk about what is the cause for this and why we're seeing these particularly sophisticated type malware attacks for the supply chain? ️Mackenzie Jackson: Yeah. So last week we saw the start of a phishing campaign targeted at NPM package maintainers using a very clever kind of typo squat on the NPM subdomain called npnjs.com. And obviously everybody who publishes a package on NPM their email address is public. So somebody went through, scraped the whole registry and sent out all these phishing emails targeting especially kind of high-profile maintainers and it was so well done that they had people fall for it, like really big ones. ️Charlie Eriksen: We saw one maintainer that kind of had a total of around about 100 million weekly downloads. So this is almost worst-case scenario and they actually, yeah, they expressed that they had fallen for this phishing campaign. What was the result of that? What kind of, what happened after that? ️Mackenzie Jackson: Yes. So the good thing was that this person very quickly recognized what happened, that they had been phished. So they were able to notify the community, delete the malicious packages and fix it really quickly. Yeah, there's still going to have been a lot of exposure. There's a couple of mitigating circumstances. The specific malware that they were using only targeted Windows, the right small window of attack, all these kind of things. But it wasn't the only thing they compromised and frankly we don't really know what the fallout might be from just the people that might have been exposed. So in the coming days and weeks, it's likely that we're going to find out more. Perhaps some more packages have been compromised because if I understand correctly, this phishing email was trying to capture credentials and session tokens for NPM. Is that how it was working? ️Charlie Eriksen: Yeah. So this is a very classic spear phishing kind of attack, right? They had a login page that looked like NPM and I believe what how it mostly will work is that it basically does a login attempt for you, proxies your request for you. It's like a man in the middle. So even if you have two factor, they are getting your session token kind of on the wire because they are just like a middleman. So now they have the ability if you actually go in and log in through the link to be logged in as you and then they basically just add an automation token and then they have persistent access to your account to do whatever. ️Mackenzie Jackson: So this works even in the scenarios that MFA, multi-factor authentication is enabled they can still access your accounts via this pathway. ️Charlie Eriksen: Yeah. If you go in, put in your username and password and then put in your two-factor token, then yes, because effectively you now both get that session token in theory. ️Mackenzie Jackson: Terrifying, I got to say. What's the kind of greater implications? Let's say one of these attacks makes it through. Let's say a popular package, couple of million weekly downloads. What's the kind of and it's not detected under cover. What's the risk here? What's the worst-case scenario that we're looking at? ️Charlie Eriksen: The worst-case scenario, it could be really bad. Some of these packages, right, like the ESLint, Prettier of stuff, 20 million downloads per package. Usually you use both of them at the same time. I think most well-set up projects will probably use those to be honest. If you use ESLint, most of the TypeScript projects I do will have it. So the amount of like systems and developers they will get into not accounting for the fact that obviously a good portion are going to be on Linux and Mac really depending on what the actual binary will do, right? I'm assuming it's a RAD or something like that. It could be used to then get more credentials, right? More tokens to then go and compromise more packages, get into other systems. ️Mackenzie Jackson: You can really see it like you compromise with this package, you get into a bunch of different systems, you select a few that are very interesting, you move laterally, you compromise those systems which allows you to move laterally into more like the follow-on effect could be huge in this case which is I guess why we've seen such a sophisticated attack. To me, this doesn't appear to be some kind of script kitty playing around in the basement, right? ️Charlie Eriksen: No, I would be surprised. This is well organized. The binaries, from what I can tell, they were quite well obfuscated. ️Mackenzie Jackson: Yeah, they know what they were doing. This is not small fish. And what can people do to protect themselves from this if they're freaking out going, "Oh my god, how do I prevent malware getting into my systems?" What can they do? ️Charlie Eriksen: Yeah, the truth is that there's very few things that you can really do because of just systemic kind of problems in the ecosystem in my opinion, right? Because there are very few ways for these maintainers to communicate, hey, something happened, right? You have your NPM installed in your command line and there's not really any way of communicating back going, hey, something happened. You might have installed a bad version. Really what you can best do is protect yourself at the time of editing packages, right? Installing a package, upgrading a package. Definitely use lock files. Use lock files with fixed versions. Don't allow these like fussy matches, right? Where will allow an installation or an upgrade to be done to a like a slightly higher like minor version that can get you in trouble. Use lock files. And there are tools out there that will check against databases to see, hey, is this package flagged as malicious? That is free. And for all of these like really obvious cases, right, where it gets detected, at least it protects you during that window of vulnerability where the package is still live and hasn't been taken down yet. ️Mackenzie Jackson: Got it, got it. That's the, that seems like a scenario where we can't do a whole bunch to protect ourselves, but there are a few things that we can do and making sure that we're at least in control of what packages we're upgrading to. Charlie, thanks for this terrifying update. It was great to have your insights in here. I'm sure we'll be having you back on the show when it pertains to malware and supply chain incidents. So, thanks again for coming in and yeah, hopefully you stay safe out there. And now is the perfect segue to talk about our sponsor of this show, Aikido Security. Aikido Security is your complete security solution from code to cloud, detecting vulnerabilities in the first lines of code that you write all the way to your runtime environment with your cloud infrastructure. But very topical to today's episode, Aikido Security has just released a new tool that is open source called Safechain. Now Safechain is a wrapper that wraps around your package managers like NPM. So when you run the commands like npm install, it actually checks your packages for malware. It means you don't need to do anything new. You can stay in your total environment. Aikido Safechain protects you from malicious packages that may have got there by account compromise, typo squatting, dependency confusion, or any other way. With Safechain, you can regain some confidence in your open-source supply chain. But enough from the sponsor's segment. Now we get to go into my favorite part of the show because I get to go into the Cyber Bar and sit down with someone and drink sake and ask them a whole bunch of questions. Today I got the privilege of sitting down with Berg Severens, who is an AI security researcher and founder and we go pretty deep into some conversations about what the future of security looks like with the rapid development of AI. It's a great segment. So let's get into it. All right, welcome back to another episode of Sake and Cyber. I'm here with a very special guest, Berg Severens. I was practicing your name, so hopefully I got it right. ️Berg Severens: Great. Yeah. ️Mackenzie Jackson: We're going to be talking a little bit about kind of AI, its role in security. You have an interesting background because you're more from an AI engineering point of view and history rather than security and now working in security. But maybe we can start with telling us a little bit about your background. But before we forget, I almost forgot the most important thing and reason why we all come here. We're going to try some sake today. So the sakes that we have in store is our first sake is going to be from the Nada region in Japan again. And this time this is from a specific region that is famous for soft water. So it should be fairly soft. The second sake that we're going to be trying today is from Hiroshima. And this is kind of famous for its Ginjo brewing technique. So we'll be a little bit sharper and a little bit different. So let's try our first sake. Cheers. ️Berg Severens: All right. ️Mackenzie Jackson: Yeah, it's actually sweeter than I remembered. Well, let's start off talking about your background. So you've got an interesting background, but kind of at what point did you start becoming interested in AI and kind of what's your history with LLMs, Artificial Intelligent Neural Networks? ️Berg Severens: So basically I'm a former founder of two different startups. One was in software for metal 3D printing which got acquired by Octon. And there were three AI engineers on the team. And this started an important spark of interest in AI. And so I decided to start a second startup which was called Mon. This is an anagram of human because we wanted to develop human level intelligence. And so this is what's still, still in the more early days of OpenAI and so OpenAI was like almost a direct competitor. But we developed like new kinds of neural network architectures, basically trying to compete with a transformer. The transformer is now the working horse of all LLMs. And we tried to challenge that statement. That's how I rolled into the AI world actually before ChatGPT was released and before like the huge boom of how AI is used today. ️Mackenzie Jackson: That's super interesting. So I'm curious like for people like me the role of the transformer. Can you dive into that a little bit of like how that works in these models that maybe we're using today? ️Berg Severens: Yeah. So maybe let's take one step back before the transformer is like neural networks in general. Neural networks in general are some kind of simulation of the human brain. And this simulation is like very basic. And the way how neurons are connected in our brain is still very different from how it works in neural networks, but at least we're trying to mimic those networks. The transformer is basically one way of connecting these neurons that show to be very efficient. And the great deal about transformers is that they are very scalable. So if you make those networks really, really big, the accuracy is getting better and better and better. And that was not the case for all neural network architectures that were developed prior to the transformer. So the key component of the transformer is scalability. ️Mackenzie Jackson: You've had a pretty wide background in AI before AI was kind of cool, let's say, before everyone was using it. When did you start to look at the implications in security for AI specifically? ️Berg Severens: Well actually like almost a year ago. And at that time, I was looking like what would be the next challenge for me. And I was always intrigued by the companies that William started. I know William for something like 10 years now I think. And so I reached out to William and we immediately discussed okay there is a huge opportunity for AI here at Aikido Security. And so we yeah talked more about it and was clear that yeah there was a real match to join Aikido and now I'm part of the Auto Triage team and so basically we try to use AI to filter out false positives and also to give a better prioritization of the existing SAST issues. ️Mackenzie Jackson: So let's talk about triaging then. So let's start, let's just start off with like when we say auto triage or just triage in general in relation to security kind of what, what are we talking about in this in this situation? ️Berg Severens: Yeah. So we're currently focusing on SAST issues. When we're talking about SAST issues, that means like there are vulnerable patterns used in source code. And so software files these are then scanned. And we detect certain patterns. For instance, if let's say a join operation is being used for creating a file path, then there is a likelihood that there could be a part traversal vulnerability, but that's just a likelihood that there could be a part traversal issue, but that doesn't mean that it's a real one. And so this detection with hard-coded patterns, it creates a lot of false positives. And one of the main USPs of Aikido is to filter out false positives to do a real aggressive noise reduction. And that's really hard with hard-coded patterns. And LLMs have an opportunity to understand much more of the context of the code and to understand when something is a false positive and yeah, to do a lot of noise reduction. It's just a tool in the toolbox. It's not just a tool. I think it's a bazooka in the toolbox. ️Mackenzie Jackson: I like it. ️Berg Severens: And I think auto triage alone reduces false positives with around 60%. And together then with all other tools in the toolbox, I think that false positives reduction is around 85, 90% or so for the SAST issues. So it's a bazooka in the toolbox. ️Mackenzie Jackson: So, so the auto triage really plays a massive role in kind of taking let's say they kind of capture all these results and then filtering out into like what's actually, what's actually kind of exploitable or reachable or these words. How does an AI model do that? Like what is, like what extra information is it providing? Is it analyzing? What is it doing behind the hood? ️Berg Severens: Yeah. So it starts with giving enough context to the model. And so typically we will construct a full call tree. So it's not just a vulnerable function but also the callers of that function and the callers of the callers and so forth until you reach the endpoint. And so the next step is we will try to include all relevant libraries that are being used. So sometimes it depends on the used version of an OM or a database or a schema validation library. And so we give all that context to an LLM and then the LLM will try to rule out the possibility of an exploitability. And then that's always a first step if it can really rule out all the different possibilities. For instance, there is accurate sanitization or validation in place, then it can rule out that exploitability and it will mark it as safe and ignore the issue. But then there is also the second step where the cases that if the LLM cannot really rule out the possibility of being exploitable, then it will also look at hints like might something be user controlled, like is it probable or improbable, or is this likely in a context where the severity would be high or not very high, like there is a big difference in a login endpoint or you're just managing the styles, the layout styles of a user. And so we will look at both severity and likelihood that something would exploit it. And so based on that, we will also do prioritization. ️Mackenzie Jackson: So in this kind of context, I guess what you could you could say is that the like the typical SAST rule-based SAST, let's call it, is kind of capturing, it's like a big net. It's capturing a lot of things. It's capturing some fish, but it's also capturing some junk in there. And the AI's role is to kind of like filter out that so we're just getting with, with, with what we want. And it's doing that through, through various different methods. Is there then a risk of it getting it wrong in the sense of missing a true true positive? Because the good thing about having these like rule-based kind of functions that they may be noisy, they may have false positives, but you still could see everything. Now that I know the reality is different. I know that you know that these, these systems just get ignored if they're too noisy. But to play devil's advocate, isn't there a risk that we then start trusting AI systems to really help us and then something slips through? ️Berg Severens: Yes, definitely. It's definitely a risk and the way we account for it is by building a huge benchmark of code samples. And so we, we run our samples against, we run our LLMs against that benchmark to check how many true positives would be filtered out. And what we see is that it's less than 2% that gets filtered out for the wrong reasons. And like more than, yeah, it's, it's around 60% of the false positive reduction that I mentioned earlier, which is the advantage. And so that's, yeah, in fact a no-brainer that that that is acceptable because at first it sounds like oh we're losing 2% true positives but that's actually enabling the SAST team to make much more aggressive rules and which, but with aggressive rules, I mean they can flag much more and they can, yeah, a bigger net and they can also find more true positives which is bigger than the 2%. And so that helps to actually find more true positives because we can also filter out more of it. ️Mackenzie Jackson: Really interesting point because when you said that, like my brain immediately went to exactly what you said, I was like, oh, so we're missing out like 2% of the stuff. That sounds terrifying. But actually, I get what you're saying is that because we can capture more things, we're actually capturing more information because we can be looser with the rules. ️Berg Severens: Yeah. And it's a misconception that deterministic rules are perfect. Like deterministic rules also have been tuned to code samples and there are ways to make them more aggressive, but it's always a tradeoff between how many false positives would you allow and how many true positives do you care about. ️Mackenzie Jackson: Yeah. Yeah. No, this is super interesting. I have some more questions, but let's, let's move on to our second, our second sake here. Okay, so we have here the sake from Hiroshima with the different brewing style. Now, I'm not going to lie, last episode we had two sakes. I could not tell them apart. So, let's see if the palette has improved in one week. I think that's how it works, right? Cheers. No, very different. ️Berg Severens: Yeah. Very different. ️Mackenzie Jackson: This is stronger. It's not sharper. It's like stronger at the back, I feel. ️Berg Severens: Yeah. For some reason, I have to think about the color of brown. ️Mackenzie Jackson: I don't, for sure. Yeah, absolutely. Yeah. Yeah. Yeah. Not the most appealing color, but definitely, like, definitely that's what it's, that's what it's feeling. I feel like there's going to be like a sake person like watching this. It's just like dying on the inside that we're getting it all wrong. But that's all right. You know, we're trying to learn. Let's talk about the economics of AI. AI economics at the moment. You know, there's crazy budgets, like insane things. We've seen the headlines that Meta is offering $8 trillion per week for someone to, to come, to come do that. They're building like entire cities that are data centers. Is, is this going to be a case in every scenario where biggest budget wins? And how can you build these really important finely tuned models, products without $8 trillion? ️Berg Severens: Yeah, it's a very big question for a lot of people currently. Yeah, so currently there are indeed a few, maybe three top players like Anthropic, OpenAI and Google. And then a bunch of followers like Meta. Maybe also Mistral, DeepSeek. ️Mackenzie Jackson: Zack Z, if Zach's listening, he's, he's squirming in his seat. They're doing the follow. ️Berg Severens: Oh, I'm not sure, but it's all right. There's a very small chance that the Zach is in the, is in the, is in the episode probably. I hope isn't listening. Head of AI. But, um, yeah, there are quite a lot of followers and they are trying to challenge like the, the ones on the front. It's also surprising like what something like, like someone like DeepSeek could do. They could surprise everyone suddenly with a lower budget than was expected. ️Mackenzie Jackson: Much lower like by, by a factor of about 100, you know. ️Berg Severens: Yeah. Well, debatable because they, they had probably much more GPUs available than than was known first. And it was only like the last stage of training before all the R&D that was published, like oh, we spent only this amount of money on the final training after doing so many experiments that also cost a lot of money. But it's still surprising that a relatively small lab compared to Anthropic and OpenAI and Google was able to come up with with that kind of results. So also a very important player to mention here is Nvidia because I think still 98% of the published papers in AI use GPUs provided by Nvidia. It's still not really challenged as a company. But also apart from the competition landscape, I think it's also interesting to mention Moore's Law in this context for AI. I don't know if you ever heard. ️Mackenzie Jackson: I'm not familiar but let's, let's, let's dive into it. ️Berg Severens: So Moore's Law has been posted like decades ago where a man named Moore said that the number of transistors on a chip is doubling every two years. And this law has been a fact like for the last few decades. And this is actually what enabled the development of AI. That compute has become much and much and much cheaper compared to several decades ago. And it's still continuing. And that's a very important factor in the economics of AI. So if you follow that law and let's maybe try to formulate it a little bit different like the price of compute is being cut down by half every two years. That's something that's still going on. And so if you take this law of Moore, then yeah you can imagine that in a few years from now or a decade from now, like the, the economics around AI will also be completely different again compared to what they are now. Because it will be again much cheaper than what we are seeing. Now the question is also, will we get the same level of intelligence at a much lower price or will we see a much higher level of intelligence for the same price as what we are paying now? And this will also affect the cybersecurity community a lot. The answer on that question, and probably it's somewhere in between. I'm not sure if, if the intelligence will keep on increasing and increasing at the rate that we are seeing, but at least it, it will certainly become more intelligent than what it is today. It's just not obvious how much more intelligent and it will also get cheaper to what it is today. ️Mackenzie Jackson: Yeah. Five, 10 years in the future, where are we in security as it pertains to LLMs? Do we have any traditional scanning as in rule-based scanning anymore? Is everything AI, you know, and, and I, you know, I guess if, if that's the case, like how, how, how do vendors, how do products differentiate themselves? ️Berg Severens: Easy question. Wow. Um, yeah. Well, it's, it's interesting because the economics will play a big role in it, not just the accuracy. Like one big reason that we don't do detection with LLMs today is because LLMs are quite expensive to do just detection while you can use hard-coded patterns. There are other obvious reasons as well like, like also the accuracy is actually still better with hard-coded patterns to detect them. If you read like the academic papers on, on the topic. But yeah, it's, it's a very difficult question to answer right now like there are so many different ways that the scientific community could go in. So I think also something that I'm waiting for is is like a certain development in LLM that's still not there that surprises me. And this development is the, the ability of being able to learn from what it's doing. So we as a human we, yeah we have learned a lot during our education then we come in the working field and then we learn by doing how we should adapt our knowledge that we learned in education to a certain niche and then during working we also think again and again and again about the things we are doing and for AI there is like a similar thing that they are just first pre-trained on a lot of knowledge. ️Mackenzie Jackson: Then they are fine-tuned. But the fine-tuning is also very, very general for like any number of people and then there is like the inference time where do they do reasoning and they reason about their own thoughts. And so there is like a big similarity between how we humans think and how AI thinks. But a very big difference is that we humans we learn from what we do in the niche we're doing. And I'm kind of thinking about this in the way like if I went to school and university and then I came out at the other end but I'd learned nothing else from that point on like that, that is my knowledge right and then yeah. ️Berg Severens: And you would actually work at a company but you would forget everything you do every day again. That's, that's like the state of AI today. ️Mackenzie Jackson: Okay, that's really, okay. That's very interesting. Yeah, that's very interesting. So, it doesn't continue to, to, to evolve like we do. ️Berg Severens: People would probably challenge my point by saying like there exist fine-tuning methods. But these fine-tuning methods, what they basically do is they take a pre-trained LLM and then they fine-tune on a number of example answers and then it learns how to respond in a similar way. But it has very big disadvantages. Like for instance, uh one idea could be like let's train it on Slack. Then we have all the informal knowledge that's present on Slack and we have like a huge knowledge base of what's going on in the company. But what comes out of it is that it's, if you ask something like oh, can you give me a summary of the financial report of last year? Then it will respond like okay I will work on it tomorrow. ️Mackenzie Jackson: Oh, I love it. ️Berg Severens: So the problem with today's fine-tuning methods is it learns how to answer but not what to answer. And so building up knowledge is something very different from learning how to answer. ️Mackenzie Jackson: We've almost come to the end, but we're going to finish off with a little bit of a game. So yeah, we need some more sake for this game. The game is called Would You Rather? And I'm going to ask you two options. You have to pick one. Would you rather never use AI in a response or only use AI in response? So every time someone messages you on Slack, every LinkedIn post, every email, either entirely written by AI or never written by AI. ️Berg Severens: If it was today, never. ️Mackenzie Jackson: Never. But in the future there, there may be just maybe a complete AI Berg. ️Berg Severens: Maybe, maybe I'll work on this tomorrow. Yeah, sounds good. ️Mackenzie Jackson: Okay. Uh, in application security, would you rather only scan using an AI model to detect vulnerabilities or only rule-based detection? ️Berg Severens: Ouch, that's a painful one. If it was today, today I would still go without AI. ️Mackenzie Jackson: Without AI. Okay. AI is, AI's taken an L in this one here. ️Berg Severens: Well, the problem with AI is like if you only trust AI, then I'm also assuming that it's, it's just incredibly hard to trust it everywhere. And then yeah, it will miss a lot of stuff. So a lot of the work we do is, is also like finding those niches where it's, it really shines. So I wouldn't trust it today to, to do it everything. ️Mackenzie Jackson: Okay, I have one more. You can probably guess what it is. It is AI based. Again, very similar. Would you rather only use AI generated code or never use any AI generated code? ️Berg Severens: Uh, okay. But I have some stipulations. You're allowed to, you're allowed to continually prompt. ️Mackenzie Jackson: So, you're allowed to like it can produce something. You're allowed to change it with, with, with a prompt, but you just cannot do the code yourself. And can I still ask questions to an AI? You can, you can ask. So imagine, let's just say that you don't know how to code. Right. And you're using Cursor. This is the scenario. Would you rather that or be the legendary engineer that is Berg? ️Berg Severens: Um, let's go for being the legendary engineer myself. ️Mackenzie Jackson: Ah, I thought I was going to win this one for sure. I was like, all right, AI took a big L in this one, but Berg, thanks so much for joining me. I had lots of fun. Super interesting chat, and I hope to have you back on the show in the future. ️Berg Severens: Thanks for having me. ️Mackenzie Jackson: And we've come to the end of the episode. If you liked this episode, please make sure to subscribe to the channel, like the video. It actually helps way more than you realize. We're fairly new, but we're trying to bring great high-quality content. So, we would appreciate if you helped us out there. I'll see you all next week. And remember, stay safe.
