Mackenzie Jackson: Hello everyone, welcome to another episode of The Secure Disclosure Podcast. We're here in the Cyber Sake bar. I am here with Matias Madou, who is one of the co-founders and CTO of Secure Code Warrior. Matias, thank you so much for joining me here on the show. Let's start at the beginning. Secure Code Warrior. First of all, let's kind of dive into why you felt the need to create that business, that company? And how did it all just kind of start to come about? And also for those that don't know, I guess we can go into what Secure Code Warrior actually does as well?
Matias Madou: Absolutely. So, I actually started here at Ghent University where one of the first days I met my business partner, my co-founder, Pieter De Neve. At Ghent University, I pursued a PhD in application security. I was working with static analysis solutions, and static analysis solutions find the bad stuff. At that point in time, I'm going to date myself 20 years ago, there was only one company that was making a dent with static analysis, and that company was called Fortify. They were a small startup out of Silicon Valley and with my PhD, I actually joined them. I joined them, worked on their static analysis solution and seven years into the journey, I realized, "Oh my God, it's really easy to find problems in code if you never ever help the developer. If you never ever tell the developer how to do it for real, then, well, guess what? You're going to find problems in code." And that's essentially why I wanted to make something to help the developers. And as many things on planet earth, I was not the only person with that exact same idea of, "Hey, we want to help the developers." So, together with six people, with six co-founders, we started Secure Code Warrior with the mindset, the philosophy of, "Hey, how can we be there for the developers? How can we help the developers and get those security people off their back?"
Mackenzie Jackson: It's interesting that you say that because I guess you were one of those security people for a while.
Matias Madou: I was. I was throwing it over the wall and then running away as fast as I could.
Mackenzie Jackson: You can really see it going full circle there. And I've worked with lots of security vendors before and there's always this dilemma that you face, right? If you solve this issue well enough, are you going to develop yourself out of a need for the software or for you as a position? And I guess the other interesting dilemma too is that often there's a separation between who you build the product for and who uses it, by which I mean, often you build something for security people because they are the ones that are going to sign off on it. They purchase it, right? But developers may be the ones that are actually using it on a day-to-day basis. They're the ones benefiting from it. So there's, you often have that separation there.
Matias Madou: That's true. I think it works both ways if you're asking me. On the one hand, you need buy-in from your managers. So developers also need time quite often. So here's the thing. Quite often developers do not want to do the wrong thing. They want to do the right thing. They want to build secure software. They do not know how. They are very skilled. They are very educated, but still, you know, they want to know, "Hey, how do I build a secure application?" And to me, I think it's a journey of a developer in general. When you start learning to code, you make syntactical mistakes. "Oh, Java lines end with a semicolon." That's the first type of mistake you make. And then you start to think about architecture, different files in an application. And it's only at the end of the journey that you're like, "Ooh, there's some hidden functionality in my application. And suddenly people are misusing my application and, oh, it has security implications. Oh my God." So it's only at the end of the journey that people start to realize, "Well, people can do bad things with my code," but that is not intended by the software developer. The software developer wants to build the right software.
Mackenzie Jackson: With that, I wholeheartedly agree. Developers don't want to do a crappy job, and developers especially are not trying to write insecure code. In fact, they're really trying to write secure code, but often we don't give them—it's not that we don't give them tools—we don't give them the right tools or perhaps they don't understand the true dilemma of it or the true implications of it. So I want to get into the education piece from that, but first, because I've just forgotten, let's try some sake. We have the 0% sake here today because it is in the morning and I'm not going to do that to you quite this early. Look, I would have done it just for you. I would have done it. But this one here is still brewed in the traditional way.
Matias Madou: Cheers.
Mackenzie Jackson: Cheers. Yeah, most people are surprised. Yeah, quite fruity.
Matias Madou: Yeah, it's fruity, but I like fruity.
Mackenzie Jackson: Yeah. Sake can often be kind of nutty or fruity, depending on the brewing technique in the water. But I quite like this one. I think it goes well for the non-alcoholic.
Matias Madou: Yeah. It's very refreshing.
Mackenzie Jackson: So, let's talk about the education piece. Because we have all of these security tools, right? We have DAST. We can find most vulnerabilities. We can solve them even, and then immediately in the next pull request, inevitably more security issues come in. So, we haven't been able to stop the bleeding. We've been able to find vulnerabilities but not stop the bleeding. So, we're Groundhog Day a little bit, especially when it comes to maybe your security team. So, the education part of this is really essential. I'm sure you will agree. So let me ask you this then: Why are we struggling so much with security still in 2025 when we kind of have at least some of the tools to be able to solve it?
Matias Madou: It's true. So it seems to be a very tough problem. Although technically it's quite often very easy. You know, you just use the right syntax, you use the right methods, and you should be okay. In practice, it seems to be much harder. Developers are quite often focused on functionality and not on security. They're talking about, "Hey, it has to be functionally correct. It has to work from a functional perspective." And it works from a functional perspective, but again, there's hidden functionality, you know, there's hidden security functionality. So one thing that we see in the market is, for example, with AI fixing or in general fixing, if another tool fixes a piece of code for the developer, the developer is quite often even hesitant to check that in. That's weird, right? Like somebody else or another tool is fixing it for them and they're like, "I'm not sure if I'm going to check that in." And the reason is fairly simple. They do not understand why this is a fix. They do not understand, "Hey, but my code is working. Why is this better?" So, they're hesitant because they do not fully grasp why that thing is better, why that fix is better, and they're like, "Oh, but my name is going to be attached to that, and if that is not the correct solution, you know, I'm making it worse? I don't know." So, a lot of these auto-fixers, they fix the code and still the developer is not checking it in. I think that is the crux of the problem. The crux of the problem is the developer needs to be confident about what they're doing. They need to know what they're doing and if once they know, they're going to feel confident and going to check it in. So it has a lot to do with confidence, their knowledge, their skill set, and if they are educated, they do it. Maybe can I segue to AI?
Mackenzie Jackson: In 2025, we can't not segue to AI at some point. So, let's do it.
Matias Madou: So, interestingly enough, what we've seen is we've done a piece of research on models and what they're good at and what they're not good at. And interestingly enough, what we've seen is that we may actually get rid of SQL injection in the next five years because most of the models, they have it essentially correct. So, you say, "Hey, I want to use a query. I want to retrieve some data from a database." They always tend to use parameterized queries, for example. Oh, that's pretty interesting. So, we may get rid of SQL injection. There's stuff that AI models are not so good at. For example, more of the architectural problems. Authentication, authorization, those type of problems. Models seem to struggle with that. So, probably we will not see the end of those type of vulnerabilities in the next five years. Interestingly enough, there's actually new problems that arise, and things that we didn't have to tell developers five years ago. The most simplistic one is hallucinations.
Mackenzie Jackson: Five years ago, we didn't have to tell developers about hallucinations.
Matias Madou: And these days, you have to say, "Hey, it's possible that it is saying things that are totally not true." So, we go from things may get fixed with AI to things may be roughly the same to, "Hey, here's a new piece of category, a new category of problems that you had never heard of, but these are the new type of things." So, I think it's going to evolve. AI is going to make sure that the application security market is evolving in terms of threats and problems.
Mackenzie Jackson: Yeah, there's so much to unpack there. And it's interesting you mentioned SQL injection specifically because I did some research about six, seven months ago where I was wondering how much of vulnerabilities that we find are actually SQL injection. And I thought about this because it's one of those issues that has been solved from a technological point of view. And you said it yourself, prepared statements, right? That's how we solved SQL injection. It's how we've always solved SQL injection. So, I wanted, like, does it still exist? And when I looked into it, what I found is that about 15% of all vulnerabilities, and especially those in open source that were reported, were SQL injection. So, it kind of leads me to believe why, and why are we still struggling with this? But you said that AI is potentially going to solve this. Why is it that AI is going to be the thing that tips us over the edge? Why will AI finally help us solve these types of issues?
Matias Madou: So that's a good question. So, I don't think my philosophy is we're hopefully not going to introduce new problems. There's still a trough of legacy problems. If you think about COBOL, for example, COBOL is very old. You would be surprised, but on the Secure Code Warrior platform, we still help developers writing secure COBOL applications. Even more interesting is the fact that I'm not sure if you know, but COBOL came out roughly 40 years ago with the philosophy to have less software developers because they said, "Hey, you know, you have to be super expertise technical, so let's make something to make everybody a developer." And that's how they coined COBOL with the philosophy of, "We will have less of those software developers on planet earth and it's going to cost us less money," while they clearly failed. Even worse, so today, there's actually a shortage in COBOL developers because guess what? They are retiring right now.
Mackenzie Jackson: And these systems are still running. They're still the backbone of some financial institutions, and they're still running. So, to answer your question, are we going to get rid of SQL injection?
Matias Madou: No. Are we hopefully not introducing any new problems? Hopefully yes, right? But there's still, you know, all the legacy that we need to get rid of.
Mackenzie Jackson: Yeah. It's really interesting that you mentioned COBOL because I have a COBOL story actually. Most people don't. I do. It doesn't really involve me. So, when I was going to university, we had one buddy who was learning to do code COBOL, and this was kind of React had just come out and Node had just come out, and it was like the whole world's going to be JavaScript. Like it's just going to be JavaScript. That's just the way of the world. But my buddy was like insistent that, "No, he wants to learn COBOL because he's going to have a job forever." And we're all like, "No, no, no. Like bank mainframes, they're all going to be Node in like a couple of years. Like you, you, like, just, just wait. You have no job." So, he definitely has a job now, 10 years later, 12 years later, actually longer than that. And I think he's doing quite well according to his Instagram. So, I think he won. He won that one. So, like and COBOL's probably not even going to die in the AI revolution. COBOL's probably going to keep going until everyone that knows how to program COBOL is dead.
Matias Madou: But that is exactly the problem. The thing is working. Don't touch the thing. It is working. So the same is true for SQL injection in general. The thing is working. Do not touch the thing because the code runs. Nobody knows why it is running, how it is running, but it runs. If people say, "Well, but there's a SQL injection in there. It can be exploited," they're like, "Whoa."
Mackenzie Jackson: Yeah. I don't know. Like I'm not sure if we need to. Yes, please go and fix that. But the philosophy is still, if it runs, do not touch it.
Matias Madou: Yeah.
Mackenzie Jackson: It's really interesting you mentioned hallucination. There's a lot to go into there with hallucination. And part of it is because you have these machines, let's call them, that are so brilliant, or they're perceived to be so brilliant, or when you ask them something, especially something you don't know, they give you such an amazing answer back that when they do hallucinate, and it's let's say something that you know, then you kind of even question yourself. Like I've experienced it where it has clearly hallucinated something that is incorrect, and then I will then go along and do research and be like, "Well, maybe for the last 10 years I've just thought wrong about this." You know, it's an interesting problem.
Matias Madou: If I can pick up on that one. So, there's actually a study from Stanford University where they gave two groups of people a task, and one group of people was allowed to use an AI solution to create that application, and the other group was not allowed to use an AI solution. After the fact, they simply asked a question. So, they didn't ask about the functionality or anything like that, but they said, "Hey, do you think your code is secure? Do you think you coded that application securely?" And the group that was not using an AI solution, they're like, "Uh, you know, maybe we think so. We hope so, but we're not entirely sure." And the group that was using an AI solution, they said, "Yes, it's secure. We know it's secure." You know, I'm exaggerating, but that was the difference. Ultimately, they looked at the code and there was it was only marginally different from a security perspective. Even the people that were not using AI were slightly better than the people that were using AI, which is a very interesting fact, exactly what you've mentioned. People are overly confident. They're no longer critical. The critical thinking and the mindset of developers is almost gone. And that's something that we want to bring back to developers. We want to bring back critical thinking. That AI solution is just predicting the next best word, the next best piece of code. It isn't that smart. You need to use your own critical thinking and make sure because your name is going to be attached to that piece of code. So, you need to do critical thinking. And it's super interesting. Maybe the last analogy that I would like to use is, if you know your stuff about a particular subject, go to ChatGPT and ask a question about something that you're familiar with, you're going to say, "Uh, it's not entirely correct because this is not entirely correct. This is not entirely correct." And then go to ChatGPT and ask a question about a field that you do not master. And you're going to be like, "Huh, that's interesting. I did not know," without any blink of, "Oh, I should verify and check that." No, that's not what you're doing. And then the kicker, of course, is when are you using ChatGPT? When you know your stuff or when you do not know your stuff? That's a good point. So then you're like, "Oh, one second."
Mackenzie Jackson: Yeah, I recently actually had this experience where I was writing an article about secrets detection. It just happens to be something that I know a lot about. I know a lot about detecting secrets, how they're detected, the trouble of secrets. And I did the deep research on ChatGPT and I was like, "This is going to be interesting to see what it comes up with," because I needed an article and I was like, "Well, maybe it will provide something good enough that I can tweak." And it was very convincing but wrong. But you only would have known that it was wrong if you really knew the subject because it was very, very subtle. But there was just a couple of things it's like, "That is not how it works." And at a high-level glance, I guess most people wouldn't know and it wouldn't matter, but when you get into the technical weeds, it does really matter. How has AI kind of changed the direction of Secure Code Warrior? So, for the last eight years, you know, or for the first eight years, you were very heavily focused on education. Yes, empowering developers to learn security, and you did it in a very intelligent way with your platform, which you can get into. But how has the direction now shifted?
Matias Madou: Okay, I'll go through two shifts in Secure Code Warrior that we've experienced. So, first of all, oh yeah, so we're 10 years in business. I'm super proud of it. We're 10 years in business, and 10 years ago, yes, we were a training platform, education platform, and our philosophy is, or our philosophy was, "Hey, how can we help the developer, be there for the developer?" And key in that journey was the fact that it's not super sexy, right? Security and writing secure code. So, we wanted to make sure that developers at least could enjoy learning about security. That was our main mantra or philosophy. So, 10 years ago we started as an education. We used gamification to get people on our platform and so on and so forth. And that worked really well. That worked really well up until two, three years ago. Two, three years ago, companies came to us and they asked questions like, "Hey, is that actually working, that training? Like, they're spending time, but is there an ROI? Is there an improvement? What are the proof points here, you know?" And really good questions. And you're like, "Huh, we do not know the answer ourselves, but let's find out."
Mackenzie Jackson: So, you're a customer. Let's find out if it actually works.
Matias Madou: So, some of our customers, what they did is they scanned their application based on the top three. They rolled out a training program and they upskilled their people and then at the end they scanned their application again and they said, "Huh, look at this, you know, they're introducing less problems. They're even fixing problems, and if they need to fix problems, they're fixing it faster." And it was all measurable. So that happened two, three years ago where we said, "Oh wow, that's so they were telling us." It was not that we had to fabricate data to make sure that it was working. Our customers were telling us the use cases that they used. So like, "Okay, maybe we should use that as an entry point. Maybe we should change our model and maybe we should evolve as a company." And what we toyed at that point was developer risk management. So today we're no longer an education business. We call ourselves the leader in developer risk management. What does that mean? That essentially means that, so first of all, as a CISO or a director of application security, I hope that your goal is to only have secure developers creating code. I hope that you set a minimum bar where you say, "Oh my God, I'm responsible for all these developers, and I cannot manage all of them because I'm one person that needs to manage a hundred developers. That's quite often the ratio." So, how can I make sure that these developers are writing secure code? And that's where we came up with a trust agent platform. So where we were education before, we came up with tools, a toolset to figure out if a developer was really educated and could check in code into a repository. And that's our mantra from the last two, three years. Now your question was, "Hey, how does AI? What is AI? How is AI coming in?" So, first of all, what I've mentioned before, critical thinking, big problem. My philosophy is with Secure Code Warrior, we want to offer everything possible to the developer to create secure code, and it's very complementary to what AikiDO is doing. I think what we're trying to do is we're setting up the entire environment like, "Hey, learn about this, learn about that. Do you know how to use Cursor? How do you know how to do threat modeling with AI? Do you know what rules you need to use?" So, we're trying to make the most optimal environment. We teach the developer the most optimal environment to create secure code. And if they're still making a mistake, companies like AikiDO can find the problem and help to remediate that problem and help them fixing that problem. So that is the first step of our journey of how we're transitioning into that new world. It's very, very complementary because ultimately my grand vision is, if you want to know how software is built, you need two components. You need to know about the software and you need to know about the people. And I think companies like AikiDO can tell you everything about the software. It doesn't matter who built the software. It doesn't matter at all. But you can give an overview of the risk in the software. How good is the software? Where is the risk in the software? Companies like Secure Code Warrior, on the other hand, we can give you a complete insight on the people that are building the software. Do you have the right team to fix that software if there are problems, or if the software is stable and good, maybe you have a new team that is going to make a mess and we can essentially predict that six months from now you're going to have a problem because right now it's good but the people are essentially not so skilled and they're going to introduce problems. So if you want to know how software is built, I'm convinced you need both components. You need the people and the software. Maybe the last portion to answer your question about AI. I'm sorry it's a long answer.
Mackenzie Jackson: No, no, it's good. It's good. No one watches this to hear me speak.
Matias Madou: I don't know about that. The last portion is now that AI is generating code, the question I quite often ask to CISOs or directors of application security is, "Do you know what portion of your code is created by AI?" And then quite often I see blank stares. Not quite often, always. I always see blank stares and they're like, "Huh, the average is around 40%, right?" "Oh, yeah. That's." "But in your organization, do you know what models they are using?" "Well, the policy says, 'No, no, no, no. I, you know, there's the policy and that, what people are really doing.'" And then the last question is, "But and do you also know if the people know what they're doing with AI? Like, are they security skilled?" And that is the last portion of the journey right now with Secure Code Warrior. We want to give that visibility. So we created a solution called Trust Agent AI where we give that visibility. We can tell confidently within Secure Code Warrior, within a group of developers, that have this particular solution, that 40% of the code is AI generated. We know that 65% of the check-ins contain some portion of AI. We know what models they're using, and we also see what models are introducing problems and that is interesting. So, we see for example, "Oh, GPT-3.5 introduced a SQL injection," to just give an example.
Mackenzie Jackson: Yeah.
Matias Madou: But with the research that we've done, we know for example that, "Oh, Mackenzie, you're creating Java code and you're using GPT-3.5. Well, that's not the best model for you. Did you know that Claude 4.5, for example, is a better model for you?" Because we've done our research and we see that if you're creating Java code in this language, this framework, the better model for you is this particular model because we know it's going to create more secure code. So if you're interested in that, we can essentially guide you again in trying to create secure code from the start.
Mackenzie Jackson: Yeah, it's really interesting. And that point on models that I just want to touch on is like there was some really interesting research, off top of my head right now, I can't remember, but I'll superimpose it in post. But it was that a GPT between 3.5 and 4 actually got worse for security. And I was really trying to figure out like, how on earth did that happen? And then I was talking to someone about it and they said, "Oh, it's quite simple. Essentially these models are getting trained on larger and larger data sets, right?"
Matias Madou: So that just meant more crap.
Mackenzie Jackson: So, you know, you've kind of, you know where all the good stuff is. So let's start there. You know, we can see the code in these open source projects that are high quality. We can start training on that. But now we need to add 12 trillion data sets or whatever they're doing now. And when you kind of just open up a random repository on GitHub, is that high quality code or low quality code? Right?
Matias Madou: You do not know.
Mackenzie Jackson: You have no idea. And neither do these models. So, it's kind of interesting that a lot of the times like these models, we are to tell you why the code that they suggested is bad, but they were still suggesting it. It's very interesting.
Matias Madou: There's also a couple of other things here. So, first of all, one thing that we've seen is variability. You ask a question today, you get a particular answer which is correct. You ask the exact same question tomorrow, it gives you another answer which is incorrect. That's one thing. And maybe a final point is also, today AI is very heavily subsidized with VC money. So if you're paying $50 for your Cursor IDE, well, AI is, Cursor is heavily subsidized. They're using a model which is very heavily subsidized. It's on chipsets and they want to sell more chips so it's heavily subsidized. So if you're using, I've once seen a graph, I'm not sure if it's accurate, but if you pay $50 for AI, ultimately the real cost is around $5,000, and that has to come down at some point in time. So that is a question, you know, do you use an older model, do you use a newer model, do you have the money to go with the newer model? If the newer models are better, like your point is also very valid, it's not necessarily better, but then all these things come into play because you know VCs will not endlessly fund this whole movement. It has to end.
Mackenzie Jackson: At some point it has to be profitable, right?
Matias Madou: At some point, even more people so that means more people on the thing which drives the cost down or they need to drive the cost up to make it.
Mackenzie Jackson: Yeah, there's so much more that I want to get into. And maybe we can go deeper at another time, but I'm conscious of your time. And before you go, I do want to go into a little game that we have. It is a game called "Would You Rather". And basically the game works like this. I'm going to give you two options. Both of those options are going to suck, and you have to pick which one you'd rather. So let's bring up the graphics for "Would You Rather" and let's play the game.
Mackenzie Jackson: We'll get onto this second non-traditional sake, let's call it. First question, seeing as we're on the AI train, would you rather have AI write all of your code, right? You can prompt it. So if you're using Cursor or GPT, you can correct it, but you can't get into the IDE and change it yourself. Or you could never use AI for your code. Which would you rather?
Matias Madou: Well, actually that's an easy one for me because I'm not allowed to write production code. So, I'll go with one AI.
Mackenzie Jackson: You know what? I would probably be the same because like I'm not committing any code to AikiDO. It's a good question. So, give me one second. So, either, you know what I would do? I would not use AI. I would do it myself, but I think I would do some shadow AI on the side. That is a lot. Get a VPA. Yeah. Look, you're bending the rules a little bit, but I'll allow it. But I do, I do like that you do it. Yeah. Cheers. Talking about security, as I said, I've been the secrets guy for a long time, so I'm going to ask a secrets question. Would you rather have all of your secrets hardcoded in your application and in your Git repository? Oh, they're not exposed publicly, but they are in your private Git repository. Or would you rather have no firewall on your running application?
Matias Madou: B. For sure B.
Mackenzie Jackson: This is this one. A lot of people answer very differently on this one.
Matias Madou: Yeah, absolutely. I would definitely ignore the entire firewall then. And for the very simple reason, my philosophy has always been software needs to be built robustly from the inside out, and I get it. You know you need to get layers of protection. I'm not fighting that. But in this particular scenario, why are we building walls? Oh, because the thing is broken. Well, how about we fix the broken thing?
Mackenzie Jackson: Like, you know, I love that. Yeah. And it's kind of like a lot of people will say, "I'd prefer to have secrets there because they feel like look, it's private. There's a layer of security there," but yeah. It's a matter of time.
Matias Madou: It's a matter of time. Yeah.
Mackenzie Jackson: Would you rather always update your dependencies to the latest version automatically? So, you can't pin any versions. Or you could never update your open-source dependencies?
Matias Madou: A. Always. Always. Yeah.
Mackenzie Jackson: Well, Matias, thanks so much for joining us on the show. I really appreciate having you. I'd love to have you back at another time and go a bit deeper. But until then, let's enjoy some more sake. And thank you very much.
