Mackenzie Jackson: Hello, you are listening to the show where we bring you the latest and greatest in cybersecurity news, research, and leaders. My name is Mackenzie Jackson and you are listening to The Secure Disclosure. We have a great show for you today. First up, we're going to dive into the new attack path that's becoming pretty prevalent, that is turning VS Code extensions malicious. We're also bringing on to the show Rami McCarthy, whose recent research highlighted the risks of VS Code extensions just before this attack happened. It's a fascinating segment. Then in our Leaders and Legends section, we're bringing on to the show Patrick Debois. Patrick is a legend in the DevSecOps and DevOps space, but is also becoming very prominent as we make our transition in the AI code revolution. We get to sit down and talk about the fundamental changes that are going on, and it's a fantastic segment and you don't want to miss it. Before we get into it, please do me a quick favor. Like this video, subscribe to the channel. Every single subscriber that we get makes an adversary cry. That is a known fact. So get those tears flowing. Hit that subscribe button. Enough about that. Let's get into our first segment. I've talked a lot about malware on this show, in particular, malware in open-source packages, but it seems that there is a new attack path that is becoming very prevalent, that is turning VS Code extensions malicious or even creating malicious VS Code extensions. Security researchers have been bringing this up as what will probably be the next big attack path, and it seems that this has all come true. On Friday, security researchers at Aikido first announced and publicly reported that multiple VS Code extensions were turned malicious: Shopify Liquid, CodeJoy, Beta, Nunjucks, Rust Analyzer, and many, many more. In total, it had the potential to infect tens of thousands of developers. Now, there is something important to state here is that these extensions weren't turned malicious on the official VS Code marketplace. They were turned malicious on what is essentially a mirror of that, which is the Open VSX marketplace. So first to kind of understand this, I asked Charlie to explain what's the difference between the VS Code marketplace and the Open VSX marketplace.
Charlie Eriksen: The Visual Studio Code marketplace is meant for Visual Studio Code. We now have all of these forks of Visual Studio Code, like Cursor, Warp, and all of these products that have become really popular thanks to AI. I believe that they use Open VSX instead of the Visual Studio Code marketplace because it's Microsoft proprietary. So it's an open way for people to share extensions for these forks of Visual Studio Code.
Mackenzie Jackson: The malware in this case was really interesting because it uses invisible Unicode characters. But not only have we seen this technique before, we've actually seen this exact malware before. Aikido reported on this way back in May, and the exact malware, line for line, was used in this.
Charlie Eriksen: So what they have is that there are two things that come together. One is that you have the actual JavaScript code where there is this call to a decode function. If you just look at it in many editors, it will just show you like a single character because if your editor is not configured to show non-printable characters, it will not show these Unicode characters that actually are present that don't have a symbol on machines, whether it's Linux or Mac or Windows. So it's just not going to show them to you. So it's actually like a big blob that it's decoding and there is this native module that it's calling, a Node.js native module, and that is just a native module that takes this string, well, I guess it's kind of a binary or actually a binary literal in a sense, and it is doing a little bit of decoding to return an actual reasonable ASCII piece of code to be run. So they have obfuscated the decoding into this native Node.js module to make it more difficult to figure out what is going on. You actually have to execute it or do a lot of reverse engineering to figure out what it's doing. Then what it gets from that, it just runs; it's just JavaScript code.
Mackenzie Jackson: So essentially there is code decrypting a Base64 string, but it only appears to be one character. Behind that is a whole bunch of invisible Unicode characters that then takes you to a payload which then takes you to a Google Calendar invite and that Google Calendar invite name has a Base64 string. This is the exact same malware that we saw in an npm project earlier, and it is very clear that this is the same attack. To walk us through it, I asked Charlie to give us a breakdown of what that malware did.
Charlie Eriksen: Yeah, so the malware was basically, I mean, it had a lot of components to it. There are some good write-ups out there about what it's actually doing. It was basically a grab bag of all of the different things like a remote access Trojan would do. It tries to put in some backdoors, remote access, stealing things, you name it. Yeah, it's kind of similar, but just a much more sophisticated version of what we saw from the RAT that was in like rant user agent and got fetch from some months ago. Just way more complicated. It does so many things. These people put in a lot of work into that.
Mackenzie Jackson: Malicious VS Code extensions aren't all that new and we've been seeing a lot of attacks on these since February. One company that's invested a lot into this and recently published some interesting articles is Wiz. A researcher at Wiz, Rami McCarthy, has published a pretty interesting article just before this breach happened. In this article, they stated that numerous VS Code extensions had secrets inside them. Why this matters? It is very likely that secrets were stolen from these extensions and then used to turn them malicious in Open VSX. The first question I wanted to ask Rami is why start looking into VS Code extensions in the first place, remembering that this started quite some time ago.
Rami McCarthy: It's definitely been an ongoing concern, these malicious VS Code extensions. Just to close the loop for folks, the recent publication I had that we're referencing was a focus on secrets being leaked in the VS Code marketplace that would have allowed an attacker to take over a whole swath of very active, very popular extensions. To roll back the clock and also to relate it to what we're seeing ongoing in the threat landscape, this all started because starting in around February, we've seen a drum beat of attackers creating and publishing their own malicious extensions. What happened was, as I was tracing VS Code malware, I set up my own scanning pipelines. I was looking and programmatically detecting malicious code being introduced. I picked up two or three malicious extensions. They did slightly interesting things. I sent them off and got them removed from the marketplace silently in the background. I wasn't really sure how to talk about that problem because fundamentally these were attacker-created extensions that were doing a fairly bad job getting adoption, that were doing fairly uninteresting things. So it felt very much a lot of these malware findings that had already been published. By happenstance, I noticed that there were secrets in these extensions, right? Extensions are bundles of code. They're effectively zip files. Some of my code that was attempting to surface evidence of malice was popping up authenticated API calls, which indicates that there are secrets being used. That was really the fundamental underlying insight that led to nine months of work: Oh shoot, there are probably a lot of secrets here.
Mackenzie Jackson: So a lot of this comes back to secrets, which probably will not surprise that many of you if you've been in security for a minute. So I asked Rami, why are we still struggling with this so much, particularly in these extension marketplaces?
Rami McCarthy: There are a variety of reasons. Some are really about how we use third-party services and manage secrets generally, and some are specific to the marketplace and ecosystem. So generally, I would say if you think about how we as an industry have evolved, we have created these sort of access-bearing credentials. We've created keys, we've created secrets, and we don't have a standard way that they are at scale stored and used on developer machines. If you talk about the cloud security ecosystem, where I spend a lot of time, there are different command-line tools you can use to manage your cloud access, and it is a bullet point on the GitHub page whether they store credentials in the keychain or just in a file on disk. Yet, as a security person, that's a critical differentiator. Are these keys just sitting around in files? But then also fundamentally, the packaging and publishing pipelines have needed to evolve in both Open VSX and VS Code. When you create a VS Code extension, you're basically just zipping up a package on your laptop and sending it off. Very little of this happens in CI/CD pipelines. Very little of this happens in GitHub Actions. Mostly it's happening on individual developer machines because of how the ecosystem has evolved, how it's gone from maybe more of a hobbyist tooling to something so critical to enterprises. When you're doing that bundling locally, there's a lot higher chance of accidental introduction of sort of development files you didn't expect. Whereas if you go through a CI/CD pipeline, you have to upload your code, you're using a version control system, you have more checkpoints where secrets can be detected, removed, prevented. Of course, something that, credit to Microsoft here on the fix side they did, is they added secret scanning on the publisher side. This is also ecosystem-specific, but something we've seen GitHub have a lot of success with and Microsoft invest in broadly is this idea of protecting users from themselves. Fundamentally, every secret that was leaked was a publisher making a mistake. But Microsoft does have a choke point here to do some level of fairly commoditized scanning detection and frankly they now have a blocking rule set in place for risky secrets being put into these extension bundles on their side.
Mackenzie Jackson: So we're seeing attackers push in lots of different areas, from npm packages, PyPI packages, now extensions. We've also seen research from Chrome extensions. I asked Rami, what do we need to do? What do we need to do to actually solve this problem?
Rami McCarthy: So this gets into a nuance, right? We talked about supply chain. A lot of the code and risk here has actually been in open source where you don't have suppliers. You're pulling code off the internet. You can't really set expectations on open source maintainers to invest heavily in security as a platform. However, I think that as consumers and as security engineering teams, as organizations, we really need to start to be more thoughtful consumers of third-party code, third-party packages, third-party utilities. This is something that it's really hard to put Pandora back in her box. If you are at an at-scale organization, you have 10,000 engineers and 500,000 npm packages floating around. However, as you find that threats are evolving in these ecosystems, you should have a standard set of controls for basically binary allow listing, and you should be using it judiciously against ecosystems where there is risk. I do think we have to play catch-up here from attackers. We've seen this in PyPI, npm, Cargo, VS Code, Chrome extensions, right? It's in all of these marketplaces already. But fundamentally, as a single engineer, you can download something off the internet. There are two problems there, right? Can random code be downloaded and run, and do you want the point of control to be there, or is your point of control internal to your business where if a single developer laptop is compromised, you move on with your day? That latter approach is something that we've seen with like some of the push for Chromebooks a few years ago that was really active, was this idea of minimal developer workstations where there just isn't as much attack surface. So CISOs need to stop and make that choice, right? Am I putting my control on bounding what developers can download and run on their machines? Really hard decision. Or am I putting it on if malicious code is run, the blast radius is kept low? That decision will have sort of knock-on effects on your entire way of doing development.
Mackenzie Jackson: A question that you might be having is, how come this didn't really affect the VS Code marketplace and affected more the Open VSX marketplace? Well, one of the reasons for this can actually probably be attributed to Rami himself. Rami concluded his research, he actually reached out to Microsoft. As a result, Microsoft now does secret scanning in the VS Code. So I think we have to give a pretty big hat tip to him there. Open VSX hasn't yet introduced such limitations. I'm not here to bash Open VSX. They have a much smaller budget than Microsoft, but I did ask Rami if this potentially could have solved that problem.
Rami McCarthy: Yeah, and there's a world where Open VSX sort of implements that first too. It's really the challenge here is, there are benefits to a homogenized ecosystem, there are benefits to a fractured ecosystem. We've seen the same thing on the malware side where if you actually cross-compare removed extensions, you can frequently find that extensions were only reported to the VS Code marketplace and are still present in Open VSX or vice versa. I found examples where extensions were removed from the Open VSX marketplace but hadn't been reported to the VS Code equivalent as well. So just keeping these platforms in parity, both from a platform controls as well as a management of malicious, vulnerable, compromised extensions, it duplicates work, and as a consumer, you have to sort of think about, especially as an enterprise consumer, where do you invest your security engineering effort? Which of these platforms are you preferencing? How are you enforcing that folks are only using your preferred distribution vectors? How are you limiting which extensions are able to be deployed? There's a real enterprise management problem at scale.
Mackenzie Jackson: Now, just before we get into our Leaders and Legends segments, while I had Charlie Eriksen on the line, I wanted to discuss something else that has recently happened. On the show, we've talked a lot about npm breaches. We've talked to malware researchers. We've talked to maintainers of open source that have been phished and their projects turned malicious. We've all been screaming for one thing, and that is for npm to update its security policies. Now, I'm not going to be so bold as to say that this show influenced it, but I will like to say that it was part of a community effort to try and get npm to take action on security. I'm so glad to say that actually they have. So I chatted with Charlie just to see what actually has changed and what this will impact. So Charlie, what are some of the main points that have been updated in npm security?
Charlie Eriksen: The most recent changes came into effect starting about a week and a bit ago, on the 13th of October, which are changes around access tokens. The first thing is that you have these things called granular tokens, which are these tokens you can create where you can define what access it has, and they have changed the policies there so that the default expiration for these tokens are seven days instead of 30, just to reduce that window where a token could be vulnerable. They have set a maximum expiration of 90 days instead of unlimited. That's really important because what we saw, like with Shy Hulum and Singularity, is that they would have these tokens, they would steal them, publish them, and then for Shy Hulum, they had been sitting on them for a while and then decided to use them. So that's the first thing. The other thing is you have these classical tokens, which are basically tokens that have access to everything and have no expiry date, and those are really scary. So what npm has done is that they've gone, okay, we are going to, we've removed already the ability to create these types of tokens, and over the next five weeks, so the next four weeks from the day of recording today, they are revoking all of the existing tokens.
Mackenzie Jackson: Is this the only change or are there further security changes that have been made by npm?
Charlie Eriksen: Yeah, so they have disabled the ability for people to use time-based one-time passwords. You'll only be able to do web authentication and passkeys for two-factor authentication because that solves the problem that Josh ran into of one-time passwords or TOTP ones being phishable.
Mackenzie Jackson: Lastly, I wanted to ask Charlie the big question: Do you think that we will see less malware on npm because of these changes?
Charlie Eriksen: No, but it will be less impact.
Mackenzie Jackson: And that segment gave me a good segue to talk about the sponsor of the show, Aikido Security. Aikido Security is your complete code security platform. Aikido brings together multiple different tools like static code analysis, dependency scanning, but also it can check things like your API endpoints and your cloud infrastructure for vulnerabilities. Aikido have just launched a very cool open-source tool called Safechain. Safechain wraps around package managers like npm and blocks installations if the package contains malware inside it. This is especially important with all of the attacks going on. You never know if a package has been compromised or not. But with Aikido Safechain, you do have added protection. You can try Aikido for free today at aikido.dev. And now for a segment I've very much been looking forward to. Patrick Debois has been a big influence in the DevOps and DevSecOps space, and he's been pretty outspoken along that journey about some of the challenges and potential issues that it actually brought about, as well as how to do it securely. Now Patrick is providing those insights and experience into the revolution that we're going through with AI, how we dealt with the cloud-native revolution, and how now we're dealing with the AI revolution when it comes to development, and code and deployment. So I was fascinated to get Patrick's opinion on this. So without further ado, let's get into Leaders and Legends with Patrick Debois. Patrick, thanks so much for joining me here today. I really appreciate you coming on the show.
Patrick Debois: Yeah, it's my pleasure to be here. It's been a long time in the making for this podcast, but yeah, let's do it.
Mackenzie Jackson: Yeah. Awesome. I want to start off kind of doing a little bit of a curveball because there's been some recent news that is unavoidable recently. Your background is deeply in DevOps, DevSecOps, and now you're kind of a front thinker in AI. Recently there's been some AWS outages which has caused a fair bit of chaos. Famously, AWS announced that they had kind of relinquished 40% of their DevOps team to replace them with AI. This kind of fits perfectly in your wheelhouse. So I'm curious to see, do you have any comment on kind of what we've just kind of witnessed? Do you think that this is kind of something that possibly could have happened anyway? Do you think that the transition to kind of AI for DevOps is too soon or what's your kind of opinion on what we just saw without being too speculative?
Patrick Debois: So my first reaction to these outages is usually what is called in the community 'hug-ops', right? If you've been somewhere responsible for a production system, this is not your best day. You learn from it, but it requires a bunch of empathy for you trying to solve something at scale, right? I guess that's what a lot of people dismiss is, hey, you know, Amazon or whoever has a lot of money and they should be able to do a very good job at staying up and they have all the IDs and all kinds of things. There is just a certain, sometimes, alignment of events that you didn't foresee and it kind of, very similar to, hey, a user does an unexpected thing or an operator, there are still some pieces that are human-based, there are pieces that are automated, but it doesn't matter too much; both can go wrong for various reasons, right? So yes, I assume companies prepare for that, but there is also a certain cost with preparing things and some things you can't just foresee. So I'm pretty much on the side, was this avoidable? Hindsight is easy, right? Oh, yes, we could have done this, but in reality, that's kind of not how it works. As long as we're learning, and that's usually the thing, like will it happen again or the same thing or something else? Will we have prepared for that? Is it actually worth preparing for? I know the impact is big, so you don't hear me say that, yes, it happens. But yeah, a small thing can lead to several things. So I have maybe more empathy than other people just using it, and I'm still okay with that. Now the conclusion that you draw like, hey, people got fired or is AI kind of not doing that? I don't follow the path in a way that is this related to people being fired? No. Is this due to a stress? I have no story on the inside. Is this something that happened for a reason? Was this even AI? Whatever I found on the internet, it says like, oh, it's a DNS problem, but it's a DNS problem with seven events happening at the same time. So, try to prepare scenarios on that side. But yeah, anyway, we're always hoping that it doesn't happen again. But over my career, I had it. Hey, system goes down, we have to restore from a backup. It takes like a day to restore the backup. A month later, you thought, we'd learn from that, same thing happened.
Mackenzie Jackson: Well, look, I mean, it's all about improving, isn't it? Things like this are always kind of, I mean, I don't want to say they're inevitable, but when we're always pushing the boundaries of what we're doing in various ways, then we're going to find new stress points.
Patrick Debois: Yeah. I think the scale and the impact is something we should reflect on like, hey, this was one region, but then all of the users reusing that region and being the impact.
Mackenzie Jackson: But you can't blame them, they have multiple regions but the customers only use a single region, something like that. But yeah, you assume it works, you don't want to put in the work to make it redundant. Some companies did and they survived, and others didn't. It's a risk game that we're taking.
Patrick Debois: Yeah.
Mackenzie Jackson: As much as you can lay fault at AWS, there is a shared responsibility in this for yourself. So I think that's a good conclusion from that. To move away from the speculation and the news today, I really wanted to get this opportunity to discuss with you and pick your brains a little bit about this big shift that we're seeing in development. One of the reasons why you're such a good guest to have on here is that you've gone through this shift of DevOps and going to cloud-native, and you've had a lot of opinions about that process, and now you're becoming, let's use some tacky words, a thought leader in this AI space. So I want to get into the shift that we're going through now. We're moving into this not just AI coding but AI-assisted coding. We're also using AI for our infrastructure and multiple things. What are kind of the big changes that we're going to be going through immediately? How will our processes, our systems, and how we work change, introducing AI? I know that's an impossibly big question, but I'm going to try and give it, let you have a stab at it.
Patrick Debois: When AI came into our realm of coding, we started to explore what the kind of difference were that we were doing compared to what we currently were doing. As with the technology of LLMs, it is about code generation, right? So the whole starting point was, hey, we can do faster code generation, ask it something, and it has that seamlessly infinite wisdom of doing stuff. Now when people start coding with some of those tools, they learn that they still have to do the review in that way. You can, like you mentioned, you can do AI coding and just say 'accept, accept,' and that's all, but if you're anywhere serious, you are responsible for running things, and that's maybe my background of DevOps. Yes, okay, this is good on your laptop and you're done, but what if it happens and being prepared for when it fails and kind of understanding and being responsible? That is that part. I have written a blog post around the four patterns of AI-native dev, and it's mostly explaining where people are moving in their head. The first thing they notice is they are less about producing the code themselves but becoming a manager as such, in the way that we are managing coding agents, or similar to our interns, that say, hey, this is good, this is bad, you should do this, and maybe we still jump in, but that kind of puts quite a load on us because we are the reviewer as such. The more it generates, the more we're doing the reviewing, the less we still do the work. So we are responsible for when it fails. We're responsible for how much it costs us. We're responsible for anything, but we're not doing that work. So that is probably the first shift that people feel when they start moving with the code and doing stuff. This is, ironically, a story that I tell is all the dev people have now become an ops person because they receive code from people, somebody they didn't do it themselves, they are responsible and they have to figure out how to bring it to production. But yeah, that's the first part.
Mackenzie Jackson: Yeah, I really love that analogy. When I was kind of preparing for this and looking at some of your talks and your posts and everything, and looking at the four patterns of AI-native development, that idea really resonated with me. When I first started thinking about AI-generated code, I'm thinking about me typing into ChatGPT, generating code, pasting it into VS Code, that type of thing. I know we have Cursor now and it integrates, but you transformed that in my brain into thinking about, okay, now we're, if we think about AI agents and these tools as people, we are instructing them what to do. So the direction of the application comes through us. The responsibility of review is still us. So almost in a way it feels like AI coding is going to give us more work than previous before. Sure the output may be more, but this idea that we're going to begin coding Monday to Tuesday and have the rest of the weekend, I think that's kind of sailed out the door.
Patrick Debois: Like all technology trends, they have a way of shifting complexity around. What you gain in one thing, you maybe lose in another part. So depending on how important you find certain things, you kind of wonder. Obviously, the hope is that AI will get quite good, and so the review is kind of less tedious and it gives good results. Now the downside is, and it was, if I draw again the analogy with DevOps, if you have your pipeline working and your security is working and all good, you kind of become a little bit more relaxed. So it's ironic because the more secure you are, the less you actually care about your security, right? The more you care about you're automated, the less you care actually about the actual process.
Mackenzie Jackson: Within DevOps, we had a concept called chaos engineering where it was almost like we were deliberately inducing faults in our architecture to keep training, to keep us on our toes.
Patrick Debois: So it's quite similar to this. Yes, we do the code review, but if you're not writing, how do we actually keep up and still understand what good looks like? So maybe that shifts to training, maybe that shifts into preparing like a fireman, dealing with an issue, or somebody training. So we might not be saving time on the generation and maybe on the review, but then we have to spend the time on training. Now you don't have to, that's a choice, that's your risk. If you just say yes, yes, yes, that's fair. That's kind of where you can take different directions as maybe we did in the past to go with that.
Mackenzie Jackson: I'm curious what you think about something because you mentioned a point there that just sparked something inside me. You said if we're not on the, basically if we're not on the tools anymore, if we're not coding, how can we keep our tool sets to do effective review? So one thing I'm kind of curious about is when AI writes code now, the one thing that I notice is that it's extremely well commented. It breaks down exactly what it does, and it's very human-readable. It's clearly designed for me. At some point there's going to be a shift where these AI models stop writing code for our benefit and start just writing efficient code. Perhaps it's trained in, perhaps that happens naturally. Do you think we'll get to a point where code may be less and less understandable to us? I mean, you could even go so far as an AI model writing a binary for us. Would we get there or do you think that is kind of a different direction that we will end up in ultimately?
Patrick Debois: I think it boils back to that remark of review. Why do people have tests? It's not actually only for the sake of testing. It's also when things fail that you're actually able to see what fails at what level, right? So having that, why do we put things in multiple files? Because if everything is one file of the code, we can't understand, we can't mentally grasp what the code is doing. Now we imagine that the AI is turning away and crunching code, and like the leaked code that we had, completely un-understandable, and it generates, how are you going to do the review then? It comes back to that, how do you say yes, no if you don't understand this? And then when things fail in production, how are you actually going to make a right call on how to solve this? Now the flip argument that I see sometimes on the internet is, well, we're just going to at that time ask AI the questions that we need to ask. But if you don't know what question to ask, that's a hard thing, right? So give me all the options that you think might go wrong. That could be a lot, and it could be wrong. That's one of the things that it is so happy to give us a result is one of the things with the LLMs. Yeah, thank you, yes, I will definitely do this and next time I will do better. It's not like we should not treat this as a human and speaking in secret language as such. It's still a prediction thing, and that's why with this technology of the GenAI, maybe there's new technology or other technology that we combine it with. We cannot rely on saying this is going to be 100% correct. We don't need to understand, just push the button to production, even with all the tools and the pipelines. That is true. Now, another aspect of this is, and in pattern number two, I mentioned, okay, instead of telling it exactly how to code, we just want to specify what we want, the intent, right? That's great. We as a human, we get a ticket or product owner says, please do this, please do that. They don't care too much about how the code is written as long as it passes this test, the users are happy and they can work from that. People have been experimenting with, say, hey, regenerate the code completely different way of coding, still passes the test, is that good enough, right? In our CI/CD system, we tend to believe that when we push things through tests, it is okay enough to probably, when we're really in the continuous delivery stage, to push it when it succeeds to prod. The problem then is if the AI writes the tests and rewrites the tests, we don't know, right? So if I come back to that human understanding of making that call, as far as I believe. Now there's the industry hope as, you know, let's throw a lot of money at it and maybe we'll succeed. Then I'll bring up the story of the self-driving car, right? Okay, it works in certain cases and certain setups and configurations. That's probably what we're still looking for. In what cases are we confident enough to accept this? Ada is one of the coding tools, agents, and it just accepts and does a commit. It doesn't ask you to, it just already does the opposite. If you don't like it, revert it, but I'm going to commit, right? So it's interesting to see the confidence level. For me, it is always about, okay, what's your risk appetite? And that is something we know in the security world, yes, you can secure your whole house and put it in concrete and nobody can enter, you cannot live like that, right? So it's kind of where you draw that line and maybe it's about re-architecting your application. So certain pieces have a certain risk profile, other pieces have another profile, and you deal with that in another way. So yeah.
Mackenzie Jackson: Yeah, I completely agree with that and at some point we can't allow ourselves to give up control. That gets to a real Terminator situation there, but you can, I could for the first time see the path that we could take to get there. One of the things that you've referenced when talking about the transition into DevOps and into cloud-native was a lot of the pains that it took to get there. Whilst this revolution promised and I believe ultimately delivered on efficiencies and securities, there was a lot to get through to get there. So now that we're going through this transition into AI-native, where do you think the major pain points are going to be that organizations will face or developers will face that they're going to have to overcome and that's going to cause a little bit of chaos in the process?
Patrick Debois: So we're specifically talking about AI-native coding, right? Because otherwise it's quite broad. A lot of people are trying to obviously get the AI to do what they want. What's ironic is if you distill those patterns, they boil down to good engineering practices.
Mackenzie Jackson: So, if I'm writing code in one file compared to multiple files, that's something I would like not do as good engineering.
Patrick Debois: If I write code without tests, I cannot actually see if somebody changed the code, whether it's still going to keep working. If I overload the context window of the coding agents with too much information, it is going to fail like a human overloading with too many features at the same time. So you need to kind of take smaller steps. If your documentation is not up to date, the AI will do bad things. If you have seven different coding conventions or ways that you structure or restructure your code, the AI is going to get confused which is now the standard that I should do. So that's kind of like for a lot of people that aha moment like, hang on, we talked about technical debt, bad documentation, all that stuff. You tell me that to get the most out of AI, I actually need to do best practices. Ironically enough, people do it for the AI, which we're in 'deming' it for the humans, so that's a good thing, I would say, that come out of that.
Mackenzie Jackson: Yeah, I immediately kind of think to all the crappy documentation I've had to sift through. Now it's kind of like, I guess you perhaps we could say that you have opportunity to invest more in things like that when you have the AI there.
Patrick Debois: There's an example of an agent called Devin that, if you chat all the time with your agent, it all of a sudden says like, oh, it gave you a suggestion, and you say, well, that's not true, please change this. And it says like, you know, what should I update the documentation with this new information? Or I think this is important, should we document this? So this becomes like a seamlessly kind of documentation update with a live execution by the agents, and that's something that's quite powerful to eventually lead to almost like a knowledge management where people kind of keep track of what they tried, the features they did, and all those things that are often in an engineer's head kind of now become more tangible and reusable. If your engineer maybe leaves, knowledge is still there, good documentation, you want to onboard a new engineer, that information is there and up to date. So there are kind of ways that AI can help us in improving that kind of chore that we used to have, like documentation later, tests later, stuff like that. Now we kind of see a different value into that.
Mackenzie Jackson: One really interesting topic that you've talked about, you've done some posts about this, you've also mentioned it in some of your talks, is kind of team typologies.
Patrick Debois: I love that, I love that.
Mackenzie Jackson: Team topologies. And why I thought this was interesting is that it kind of took a framework that existed and something that was structured. A lot of AI is, when we're moving so quickly, everything feels pretty chaotic. I feel perhaps it's just me. I feel like I'm hanging on to the back of a very fast-moving train, holding on, swinging on the Eurostar and just trying to keep up with everything. But I thought, can you explain kind of this framework or this idea of how it can actually help kind of put some structure around this and manage some of these pain points as we go through?
Patrick Debois: Yeah, I think when I talked about this, it was more about scaling AI inside of your company and working on that. If we think about team topologies as a way that collaboration works, one of the ways that team topologies not prescribes but describes is using a platform and then feature teams on top of that platform, and that's the collaboration. Now the way we did it in cloud was, hey, we have that cloud team that is actually running the Kubernetes platform or something like that, and they kind of help the devs run on that. Now instead of making everybody a DevOps, where some people might have dreamt from, we went to, hey, let's enable a champion, similar to a security champion in a certain team and scale this out. A lot of the current narrative is also thinking about the platform as a product, and in my head, a platform team actually takes care of three pieces. One is obviously running that platform or infrastructure or APIs or whatever you call it, right? There's a piece of running things. The second thing is about enabling your users, whether that's your developers, your product owners, your external users, on that platform. The third piece is the governance. If you look at all those three things, this is also what you do when introducing AI. The new stack with AI had new components. We needed a RAG database. We needed to have connectors. We needed prompt repositories. We needed to have a guardrail system for AI and so on. All infrastructure. The second piece was enablement. Well, the devs didn't know what kind of model to use, and is it secure and is it correct, and can they have access to newer models to do testing and sandboxes and play around and education, very similar as we taught them with cloud and DevSecOps to learn and have a better understanding. Then with the governance, it was the same thing like, hey, you cannot just install any model, you cannot install any tool, you cannot leak your code to one of the things if there's no opt-out. There's some legislation like the European AI Act. So that awareness and governance is quite similar. So the scaling is simple. You go to one team, which is similar to your first customer, you listen to what they want. Then hopefully you have two or three others that can actually help you understand how you're going to scale this out in your company. Then you build that shared service and then you scale it out to the rest of the organization. So the same patterns apply to, hey, I'm introducing cloud, hey, I'm actually introducing DevSecOps, and I'm introducing GenAI platform. MLOps went a little bit different, they kind of kept in their own island, but it is because that was maybe not that close to the devs in general, similar maybe to like the mobile team or something that was more like a global theme on certain things. But that scaling concept of team topologies is applicable in that way of scaling out like a new transformation, and in this case, it's the AI usage transformation or the impact of that.
Mackenzie Jackson: Yeah, and I think the reason why I resonated with that too is that it just, I think a lot of people struggle with what to do and how to do this. Because it's all moving so fast and so new, it's difficult to kind of come up with a playbook. But if we look back, we've actually gone through changes, big changes in software development a lot, and in organizations. To kind of come back to a lot of that, it brings it to, oh, we've actually dealt with change and we can deal with change in a similar structured manner as we have done before.
Patrick Debois: Yeah, that's true. Ironically enough, the team topologies used to be called the DevOps topologies. So there's, but actually before that, there was like the Agile transformation, and it was quite similar but with different, like, hey, I had this, the first Agile team and it's scaling out and multiple teams. Yeah.
Mackenzie Jackson: So it's just a way of doing, like, introducing something new to an org or scaling a transformation. I want to kind of keep on this, but I'm conscious of your time, but I have to ask a couple of other questions before I let you go. One of the very interesting things is that all of a sudden metrics and how we measure success and everything has changed dramatically. We can't kind of measure success. I don't know whether this is true or not, but the analogy, the story will kind of ring true, is that there was the idea that when Elon Musk took over X, he fired everyone that wasn't producing a certain amount of lines of code every week. Doesn't matter. We don't need to talk about if that's true or not, but that is a metric that is fundamentally would be flawed to measure something by. How, when we go through this transition, how will the old measures and metrics that we used to measure success on or progress, how will they change going forward?
Patrick Debois: I'd like to nuance that. It's not useless to measure it. It's not a good idea to only make decisions on top of that. So when you kind of look at your pipeline, there are many aspects that you'd like to measure, and based on where you put a certain value. Let's say I'm looking at how many lines of code they write, and there's a certain truth to this is if somebody never touches the code, I'd like to know, right? But it doesn't mean they're not useful. But I'd like to know, hey, maybe they have a different role, and is this correct or are they just paid to do code changes and not doing code changes? Then we have a bigger problem. Now I think when the Dora metrics came along, and more on the CI/CD, it was more about the flow, like can I get something to production? Can I learn from production? Can I do a change? How long is it going to take me to do that change? So those metrics kind of took the upper hand, but on the other hand, if you say, hey, I'm doing 10,000 deploys a day, that's meaningless as well, right? If you only look at that metric, it's a vanity metric, or they're kind of gameable in certain ways. So I just urge you to look and have various metrics, and then based on what you see in the metrics, you build a hypothesis on what you want to make your decision on. The other day I got a question like, do you really need the Dora metrics anymore on the AI? Because that's just going to go on itself. And I'd argue that do you still want to have something into production, right? Is it like, how long is your agent taking? Okay, that's maybe instant, but how many bugs does it introduce? How many customers did you lose? So even if the one metric is getting close to perfect, the others will fail. So that's why you need to look at that and they are still relevant. Whether that's your code complexity or your lines of code that you produce, it's just interpretation of the signal that is what changes. There has been the idea about, well, we try to measure developer productivity, but it's like a very fuzzy thing, multiple things, and it's a nuanced thing, which is great. What do we do with AI? One of the metrics that I see coming up is how many times do I have to 'rench' the AI before it actually gets what I mean? Now is that fair to say that that is actually you're measuring the AI or are you measuring your own effectiveness to say, hey, I'm giving the AI all the context, the thing it should do, and I'm just not like wanting it to mind-read what it's like I want, right? So that's again interpretation on different metrics in there, but your security metrics still apply, your incidents still apply. So I don't think that goes away until maybe we don't run anything more in production. I asked on ChatGPT, can you come up with very creative ways or novel ways of dealing with code reviews? And it says, well, what if you republish your code every time? And then there's no time to do the review and no necessity anymore to do a review because it's always different.
Mackenzie Jackson: Okay, I don't think we're going to go there. You might not have any users left.
Patrick Debois: They don't like what they see.
Mackenzie Jackson: We're almost coming to the end now, but I wanted to play a game with you here called the lightning round where I'm going to give a series of questions, and you can interpret them as wildly or as widely as you would like, but you have to try and give an answer. So we're going to start off here. If there was one tool that you could ship to any product team or development team, what would it be?
Patrick Debois: I'm biased, but currently it would be Claude. Yeah, I think that's a good one.
Mackenzie Jackson: We were just talking about metrics. What is one metric you think you can more or less ignore now, and a new metric that is becoming super important that you would focus on?
Patrick Debois: How many pods are in your cluster? No, I don't know. But I think it's the biggest underestimated thing is what's bringing in the value. Like that is still a problem for engineers.
Mackenzie Jackson: This one here, a little bit left field, but we're going to do it. AI incident response. What do you think AI incident response should include or look at that traditional security incident responses may have ignored or didn't focus on?
Patrick Debois: Yeah, a question I get often, and there's a lot of obviously, yeah, we give the agent more docs, and it's kind of coming up with hypotheses. If it's only that, you could say it's a human on steroids, but if I think it actually spins up instances and comes up with hypotheses and tests hypotheses, then we are a step further in that way instead of just giving suggestions and handing that over to a human.
Mackenzie Jackson: Yeah. And last one here, 2026 prediction. Will we have an agent orchestrator as kind of a core platform or is it too early or will we not have it all? Agent orchestrator meaning kind of an orchestration platform for all of the AI agents that we will be managing.
Patrick Debois: I think agent orchestrators for coding is already here.
Mackenzie Jackson: Yeah. Okay. That one, that one was easy. I might have to cut that one out. But Patrick, I want to thank you so much for coming. You're full of wisdom and I was really pleased that you agreed to come on here and talk to me. It was really awesome. So thanks for coming back, and I hope that one day I can twist your arm again and bring you back on the show. But thanks so much for being here and sharing your knowledge with us today.
Patrick Debois: Right. Thanks for having me. It was a pleasure.
