Mackenzie Jackson: Hello everyone and welcome back to another episode of The Secure Disclosure. I'm super excited to be here with Frederick Ryckbosch.
Frederick Ryckbosch: Pretty good actually.
Mackenzie Jackson: Pretty good. Okay, we were practicing a little bit before. It has been great to have you here because we've had a lot of people, we've had a lot of discussions about AI on the show, but you're someone that's kind of in the weeds really building the technical implications of this. So I'm really curious to have this discussion with you because I kind of want to get into a little bit of the technicals. I want to start off with just understanding, when it comes to cybersecurity, there's a whole lot of AI being thrown around. Everywhere. I mean, you can't be a security vendor today and not be using AI. But where is it actually useful in your opinion? Because you can use AI to help find vulnerabilities, to help fix vulnerabilities. You can use it to market a crappy application if you want to. You can do anything you want. But where do you think its real use cases when it comes to securing applications and securing code?
Frederick Ryckbosch: The way I think about that is about what it enables versus the old world. So we had the old world where you have code. People have vast amounts of code. They're written in very particular ways. So people have their own styles of writing code, their own habits of doing things. All code looks slightly different, which makes it very difficult for things like scanners to pick up these patterns because people will do things slightly different. I think that's where AI really shines, where it's not just about the scanning part, but you take the code, you can analyze it, you can then see like, okay, actually this thing is not vulnerable at all because it's just a static variable that's being used. There is no kind of injection possibilities here. In other cases, it will be, okay, this is actually coming from a web request, so you have this field that's just being passed through and you can do whatever on the machine. So you have some code execution vulnerabilities for example. LLMs make it possible to analyze these things but then afterwards also to fix them. So you can actually see, okay, we have this particular flow and we now understand the code. We don't just have a regex that matches a certain thing, we understand what it's actually doing and we can make really good fixes for these things. To sum it up a bit, it used to be the case you had a lot of trouble dealing with code. Basically parsing code is very difficult, understanding the code is super difficult. LLMs make that a lot easier and that's super useful both in the scanning part, in the triaging, seeing if it's actually an issue and also in fixing the code. So I think it enables a vast array of things.
Mackenzie Jackson: Right. Because things have changed a little bit. I remember, I mean probably going back 10 years, the big problem that we were all having is finding vulnerabilities. Then the problem became fixing it. And then the problem became finding too many vulnerabilities that weren't it. So AI can kind of sit in there. You said something that I want to dive into though. You mentioned the word understand the flow of it. What do you mean by understanding the flow of an application or understanding the flow of code?
Frederick Ryckbosch: Yeah, sure. For example, if you're executing a database query somewhere in your code and it's passing in some variables there. So you're setting a name for some user or something like that. You want to know where that name is coming from. So it could be that it's coming from a static list in your application. There's just a static list of names there and it's just picking one of those. In that case, you know this is not going to be vulnerable ever because it's just restricted to this static options. If it's coming from a web request or a user input from a database, then we know that users can tamper with it and we should be protecting that value. We should make sure that you don't just put it into a SQL string and open yourself up to SQL injection attacks for example.
Mackenzie Jackson: Yeah, SQL is actually a great example. I'm really glad you went into that because a lot of people really question, why do we have SQL injection today? It's 2026. SQL injection was discovered in 1996 or 1997 around about there.
Frederick Ryckbosch: It's been around.
Mackenzie Jackson: It's been around. We solved it the same year by creating prepared statements and parameterization. So it should be solved, and yet here we are still talking about it 30 years or something later. But I think a lot of why it still exists, when you look into it, when you actually look at the big breaches that happen because of it, it's because the data flows through. So where it actually kind of runs in the SQL query, it's not like it's take user input, run SQL query with user input. It's gone through that whole flow. So I guess understanding that is maybe we'll finally be able to get on top of SQL injection or something through this.
Frederick Ryckbosch: Yeah, there are still cases that complicate things basically. If you have an in statement that you're using, you say like this username is in and then a list of variables that you want to pass in, you need to make sure that the number of question marks or placeholders that you put into the query matches the number of parameters that you have. So you actually have to write a bit of code for that. Some frameworks do it for you, but not all of them. So if you're using like a pretty low-level framework to write your SQL, you have to write some code to really understand, okay, I'm passing in parameters, I have to match a number of question marks. And then people are sometimes like, okay, I know that everything that comes into this function is actually safe.
Mackenzie Jackson: So it's not a problem that I make this a bit simpler because I know it's going to be safe. Basically.
Frederick Ryckbosch: What happens then after a few years or so somebody else comes into the codebase and they see this function where they're like, oh, this is a super handy function. I'll use it. And then they use it in a different case and now there is a flow that actually leads to user input being passed to that function.
Mackenzie Jackson: And so I think people are conscious about these things but it can happen really fast so you can have it just over the course of different people switching in the project, stuff like that. Yeah, that makes a lot of sense. I have a question I've been dying to ask you. Now, I'm going to be a little bit of a devil's advocate here because of it. One of the things that you've really been building out are sophisticated systems for auto-triaging, but also auto-fixing vulnerabilities that are found. So, you can find a vulnerability, and we've talked a little bit about kind of AI's role in that, but once you found it, you need to fix it. And AI can do that. Now, we know that AI introduces a lot of vulnerabilities from vibe coding, right? You know, there's so many memes out there all about this. There's so many examples of keys being leaked, of injection vulnerabilities, everything like that. The idea then of finding a vulnerability and using AI to fix it seems a little bit counterintuitive because there seems to be inherent risk in asking AI to fix something if AI is capable of creating a problem in the first place. How do you kind of deal with that or what do you kind of respond to in that case?
Frederick Ryckbosch: Yeah, so I think that's a really interesting observation. On the one hand side, we have a lot of opportunity for AI to be used in security to secure all of the things. On the other side, we have AI creating a lot of insecure stuff.
Mackenzie Jackson: Yeah.
Frederick Ryckbosch: And I think these two can be combined though. And so it's a matter of giving the LLMs the right tools, the right skills to be able to detect these things and to fix them. So if you say an LLM is writing some code for you, it actually contains a SQL injection attack for example like we talked about before. If you tell the LLM, hey wait, there's a SQL injection here. Then it will be able to fix it for you, but you have to tell it. And so you need to put your scanners in contact with your, well, you need to have your scanner attached to your LLM in your IDE for example to make sure that it can detect these things right away and it can self-correct itself. The thing is we already have a lot of existing code bases where there's a lot of these vulnerabilities already present as well. So that's where auto-fix comes in where you can see like, okay, we have this piece of code. You could say it's just take the piece of code, give it to ChatGPT and say fix it. However, there's different ways of fixing these things, right? So what we do at Aikido with auto-fix is we provide a remediation that's very specific for the pattern that we have detected. So we have rules that detect vulnerabilities in your code and then based on those rules we will write remediations. So for this particular remediation make sure to parameterize the queries, these things are parameterizable, these things are not parameterizable and so on. So we'll give it more instructions basically. So it can give you a fix where we are confident that this fix will be good. So it's not that we just say, okay, fix it, we say fix it in this particular way. So we've detected this problem, this is a good way of solving it. So we give the LLM more context basically to be able to create a good fix.
Mackenzie Jackson: And how can we be sure of that fix? I understand what you're saying. It's like if you tell the LLM there's an injection vulnerability there, if you, and if we give it parameters, we give it all this information. I've seen enough memes to know that they can still make mistakes or at least I'm not comfortable doing it. So how can you be sure that the fix that you create with the AI auto-fix is actually secure? Is there a way or do we have to have some trust in the system?
Frederick Ryckbosch: There is a way. So you need to make sure that you create a feedback loop. LLMs are really good if they have a feedback loop. So what we're doing now is when you are generating a fix for a suss issue, what we'll do is we'll also ask the LLM once the fix is done to write a unit test for it. And we give instructions to write a unit test that is very security specific: try to do this attack on this function. What it will do then is it will write those tests based on the code that you have. It will execute the tests and then see, oh, I missed something. So in this particular case I didn't catch this corner case, so I have to remediate that. Then it can change the solution until the tests are passing. So that way the LLM by having it execute a part of the code, by executing the test, it gets a lot of feedback about how the solution is working and it's able to iterate a bit and to come up with a great solution.
Mackenzie Jackson: Yeah, I think that does make sense and as long as you have that feedback loop because a lot of it just seems to be like, you know, trust me, bro, it's going to be fine. I want to continue on this and understanding kind of what we can actually start to auto-fix because I want to talk later on about the future but I want to kind of form some foundations first. We're talking now about code and I think most of the audience here is pretty technical. We could all relate to that, but there's a whole bunch of other areas other than code that introduce risk into our environments. There's our cloud environments, there are containers, there are dependencies. Does AI also help in securing these and how does it help or is the main use case just code which is totally fine, but which is helpful, right? But are there other ways that we can use it as well?
Frederick Ryckbosch: So as you mentioned, I had talked about code this whole time because it's a very tangible thing to talk about.
Mackenzie Jackson: Yeah, it's a nice starting point.
Frederick Ryckbosch: But we also have auto-fixes for dependencies, for containers, for infrastructure as code issues. So we apply it across the across the range, basically, across the whole stack for dependencies. For example, what we do there is we will try to bump your dependencies just like Dependabot does and stuff like that. But in many cases there's for example there's breaking changes. So we want to say, okay, these are the breaking changes that are in the packages and give you an overview. Then some of these breaking changes are for example just changing imports. So it's this library changed from this GitHub repo to another GitHub repo and now for version two you have to import it in this different way. So what we'll do is we'll use the LLM there to rewrite your imports basically. So that the code still runs when the dependency is getting upgraded. So that's one of the things where we take deterministic flow of how do we update this dependency and then we check with the LLM like we try to compile it, we see that there are some problems there and then we'll fix certain issues at this point.
Mackenzie Jackson: Yeah, one of my pet peeves has always been the auto-fix in dependencies and how it's traditionally done because this isn't like auto-fixing and dependencies isn't new. As you said, Dependabot did it. Snyk also does it and other companies, but to me it's it's been one of the most annoying features ever because it's the solution is like upgrade your dependencies. But I mean when there's big changes, even in big changes in frameworks and big changes like your version of Python that you're using, right, you can't just update that and then go to bed and sleep well, right? So this, I love the idea of understanding breaking changes, but how the heck do you understand what's breaking in it? Like how can you have that context? And I'm sorry to be devil's advocate again, but you know, like just looking through the code seems very heavy from a computational basis to kind of do that for every dependency ever. So what like how do you actually understand what's changed and how do you apply that in a meaningful way?
Frederick Ryckbosch: Yep. So the way we are approaching that is also again an AI based approach where we are looking for change logs for these libraries. So we'll find the change logs for the library that you're trying to update and we'll summarize that. We'll from the change logs we'll take the most important items, like what are all of the breaking changes between these different versions. And these changes, change logs will contain very detailed information. They will say, okay, we've added a parameter to this particular function or this function is deprecated now, it's been removed and so on, or this function has been replaced by another one, so you need to change your callers, stuff like that. And so by going through those change logs, we can actually look at your code. So we'll go through the code, we'll search for occurrences of those functions that are mentioned in the change log and we'll see, okay, actually you're not using this function at all. So you might have a breaking change between version 2.1 and 2.2 and it's all in the use blah blah function, but I'm not using the use blah blah blah function so there's no problem for me. So that's the way that we are tackling that. So looking at the change log, searching for the things that have changed within your code and then checking off the boxes, basically, do we use any of these things, do we use it in that particular way? Because in many cases it's like one parameter that changed and if you don't use a parameter then it doesn't matter and so on.
Mackenzie Jackson: And like how confident can we kind of be in all this? Because looking through all these change logs, so I know from a different area of a different tool that looks at change logs, which is finding undisclosed vulnerabilities in change logs. And the reason why I'm bringing this up is that I know that change logs aren't always that straightforward in like the language that they use, the systems that they use. So, you know, how confident can you be in understanding that and getting a whole picture and are there other kind of methods around that that we strengthen this with?
Frederick Ryckbosch: Yeah, that's a really good point. So the thing is, if you don't have any change logs or if the change logs are really bad, then this approach is not going to work well, right? What we've started doing there with code quality is to automatically generate change logs as well. So whenever you create a PR, Aikido will do is it will generate a summary of what's changed in this PR, what are the things that have been added, what are the breaking changes and so on. So and from that you can really easily make a change log then. So we are trying to help people with being better with the change logs to get all of this information out. So looking at the actual PRs that went into into the version, getting all that information in your change log and making sure that that contains the right information.
Mackenzie Jackson: I want to understand what you just said because it sounds insane, but I think it is correct. In order to help solve the issue of bad change logs, not having change logs, you're creating high quality change logs for other people to be able to basically to be able to use this in a meaningful way.
Frederick Ryckbosch: Yeah. And at this point, we're doing it for our customers. So, if you have your repo connected to Aikido to the PR checking, we'll do that automatically. I think at some point we'll want to do this for open source projects and stuff like that, but we're not doing that today to be completely clear.
Mackenzie Jackson: Super interesting. I want to talk now about trust in AI systems. A big barrier, and I actually think it's a very reasonable barrier and, you know, one especially when AI first came out it was kind of like you'd be insane if you let AI code for you, like now we all to some degree, but we all let AI code for us a little bit, you know, help us write things. So trust is evolving but I think in security as it should be, it's a higher bar. What do you kind of get feedback from developers, from engineers, people like yourself that are technical boots on the ground to like how do you combat the resistance of integrating systems like this? Like what are some of the key resistances and, you know, are they founded or is it kind of unfounded resistances?
Frederick Ryckbosch: I think they're definitely founded but there's also some fear mongering going on. So,
Mackenzie Jackson: Never, security never.
Frederick Ryckbosch: So we have we have both sides. So we are seeing some interesting behavior from the LLMs as well where we are telling the LLM to do an auto-fix and then we're asking it to write some tests and we have seen cases where LLMs they become very creative in finding ways to make it work. We give it a limited set of tools to run code, but it will put the code in a special spot where it can actually still run it then. So for example we are letting it run npm test and it will change the package.json file to actually be able to execute any code. So LLMs are quite smart at circumventing any barriers you try to put there. So I think it's super important for us to make sure that these things are running in sandboxes and that have very limited access where the LLM cannot do a lot of harm basically because it is quite unsupervised and it's able to run code. I must say with the cloud models that we are using that they are very much instruction following. If you tell them to accomplish a certain task, they will really aim to accomplish that task. They will not go rogue.
Mackenzie Jackson: Yeah.
Frederick Ryckbosch: The way they go rogue is in how they implement it. So they will try their best to actually achieve the goal. So sometimes they can do very weird things to get around the system that you're trying to build here.
Mackenzie Jackson: Yeah, yeah. It's really interesting to hear you talk about that because it's it's kind of like they act like humans in so many ways sometimes these LLMs, of kind of like we've all been there where we need to get a test done and we kind of...
Frederick Ryckbosch: Exactly.
Mackenzie Jackson: We fudge it through. I want to now talk about the future because I think this is really interesting. It's amazing to be on this AI journey and it's going to be amazing to be through it because it creates a lot of anxiety for me because I have no idea what the future looks like. Will I have a like will I have a job? You know, like is any of this going to like what's going to happen? So I want to ask you let's do kind of short-term future meaning five years. Let's do long-term future meaning kind of, I don't know, 15, 20 or however long you want to go but what do you think coding looks like in five and into the future? Are we still writing code? Are AI systems so good now that we don't need to kind of write code ourselves? Like what do you think's going to happen?
Frederick Ryckbosch: That's a very good question, a very difficult one as well.
Mackenzie Jackson: I think I only asked difficult questions.
Frederick Ryckbosch: I find it interesting how you mentioned the timelines like short-term five years and then long-term 10 years. For me short-term is like what's going to happen in the next year. Then long-term is five years.
Mackenzie Jackson: We can do that. We can change it up. I mean like five years in AI is one year.
Frederick Ryckbosch: Yeah, exactly. If I think about the change we have seen in the last year where vibe coding was not really viable like last year to give you some stuff but it would not be really good. I'm seeing that we're having, starting to have a lot of good results from it now. I don't think it's pure vibe coding. So it's not just like give it a prompt, let it run and just look at the output and keep typing. I think looking at the code, understanding what it does is still really important. But having said that, it's really good at writing the code. You need to channel it in the right ways. You need to make sure that you have some instructions in your IDE setup for example that say this is how we work in this particular project so that it follows your rules basically, that it understands a lot of the context. But it's evolved tremendously over the last year and the results are starting to become very, very usable. So it's not that you can only generate slop from these things. I think if you do it the right way today, you're able to achieve a lot of productivity gain as an engineer by using these tools. Having said that, I think you will still need a human in the loop that looks at these things and that is like, okay, we should look at this corner case or have you taken into account this thing and so on. So I think we will be more of a reviewer of the code than actually writing the code. But I personally think we will still be reviewing the code. We'll have a look at it like, okay, we should take this into account and so on. So I think that's the short-term future. So more of that and us being like reviewing the AI's code and having it iterate on it.
Mackenzie Jackson: It's really interesting. I agree with you too, but I also kind of worry, like maybe this is me, just me. But when I'm on a little bit of a vibe coding adventure and I'm creating something at the start, like prompt one, I know exactly what it's doing. I can see it creating something. Prompt 15, prompt 20, like when I'm down there, I've kind of lost track of my own flow in a way. It makes it really hard to review that code.
Frederick Ryckbosch: Right.
Mackenzie Jackson: When you don't understand it, like I'm super curious. I'm going pretty far off script right now to be honest, but what do you think's going to kind of like impact with that and how do we solve that?
Frederick Ryckbosch: Yeah, I think as an engineer you'll have a more higher level role. So you'll be saying like, okay, I want a system that achieves these properties basically. I want to be reliable, has high availability, all the all of those things. And you will let the AI take care of that, but you will still give it a direction basically. It's you won't just say, okay, make it secure. If you look at a workflow for an engineer, if you give an engineer a ticket like make it secure.
Mackenzie Jackson: What does that mean?
Frederick Ryckbosch: Yeah, right? What does it mean? At which level does it need to be secure? What is the type of application? All of those things. You as an engineer, you need a lot of context to actually do that properly. So I think the same thing goes for AIs. Basically you need to make sure that they understand what is the project about, how it's being built, where are you going to, what are the functional requirements, what are the non-functional requirements, all of those things. So I think we'll become more like an architect that is telling the system how that's telling the AI how to build the system and less of a being down in the weeds. Although I do think at this point we'll still be reviewing a lot of code.
Mackenzie Jackson: This has been super interesting.
Frederick Ryckbosch: In five years I have no clue. So that's a super difficult question.
Mackenzie Jackson: Yeah. And in 15 or 10 is kind of like, like.
Frederick Ryckbosch: Because we don't know how the, if there's going to be a plateau if this is what it's going to be or whether it's going to evolve a lot further AI. So, it's super difficult to make any predictions there. But I think like the people that are saying like there's not going to be any software engineers anymore because we'll just vibe code everything, I don't believe that. I think you will still have people who need to understand systems that will be able to give good input to these tools that will build a lot of stuff for you. I think we'll have a lot more tools, a lot better tools. It will go faster, but it will not eliminate like the human factor.
Mackenzie Jackson: It's been a super interesting conversation. We've come to the end and I'm conscious of your time, but we do have a quick game to play here. It's called Would You Rather? I'm going to give you two options. They're both going to suck and you have to pick one. All right. So, the first one is to test like how confident are you? Where do you think we are with AI right now? Because the first question is, would you rather only vibe code? Which means you cannot touch the code. You can only prompt. You can update your code via prompts, but that's kind of all that you can do. Or you cannot use AI at all. What would you rather?
Frederick Ryckbosch: Oh, that's really difficult. I think I would rather vibe code than not use AI at all. I think it would be quite hard. But if we see the trajectory over the past year, if that continues a bit more, then in a year, this will be the right the right choice, I think.
Mackenzie Jackson: Yeah, I think I would rather only vibe code. I I'm the same, but you're the first person that said that. But I think most people just are scared to say it. Yeah. All right. I have another question here. We've talked about dependencies today. So, let's do a dependency related one. Would you rather automatically upgrade to the latest version of every dependency? So, as it comes through now, there's no fancy check and change logs with AI systems to see breaking changes. It's just upgrading to the latest version. That may be malware. It may break whatever or you're frozen in time and you cannot upgrade any dependencies. What would you rather?
Frederick Ryckbosch: I will go with the vibe coding and not updating any dependencies. Loving the AI fix.
Mackenzie Jackson: Just to be clear, I'll go with clips there. Like this is what Fred's doing. He's coding, not updating dependencies. All right, we have one more. This one's kind of my favorite one. Would you rather have all of your secrets stored in a file inside your Git repository? So a private Git repository but in there or you can have absolutely no web, no firewalls in your in your systems. So everything all the ports are open. What would you rather?
Frederick Ryckbosch: I think I will go for the private Git repo because I can always rotate the secrets.
Mackenzie Jackson: Oh, this is the first time that this answer has come up. Like rotating secrets, that's like that does change things a little bit. That makes it a little bit easy if you like rotate regularly or have dynamic secrets.
Frederick Ryckbosch: So, there's a mitigation to that one if you don't have any firewalls at all. And it stays like that, that's going to be very tricky.
Mackenzie Jackson: Yeah. Well, Fred, thanks so much for joining me. It's been awesome and I can't wait to kind of see what you created and get you back on the show to talk about it.
Frederick Ryckbosch: Great. Thanks for having me. Awesome.
