Mackenzie Jackson: Hello everyone. Welcome back to another episode of The Secure Disclosure podcast. I'm so thrilled to be here with you, Tanya. You're one of my heroes. It's the second time I've done a podcast with you, so I'm a little bit starstruck, but we're going to get through it anyway. I wanted to talk to you a little bit about a project primarily that you're really heavily involved in. I think every developer probably knows about the OWASP Top 10. Just in case, in your words, what is that resource that comes out? How is it decided?
Tanya Janca: So, the OWASP Top 10, if you have not heard of it, is an awareness document. It is the most popular project that OWASP does internationally, and it is a list of the top 10 risks or things that can go wrong for web applications, but we have expanded it so it applies to most software. And how does it happen? Well, we ask people like your company and many companies and pen testers and all sorts of organizations if they will share their data with us about the types of vulnerabilities and problems that they're having. And generally, people that respond are really awesome pen testing companies and vendors. But what I wish people would respond with is postmortems.
Mackenzie Jackson: Right.
Tanya Janca: I'd love to see that. No one, give us your breach data. Apparently, no one wants to do that. They give it to Verizon, but not to us.
Mackenzie Jackson: Yes, yeah, because people are so sheltered about it, too. I mean, it's happened. If we can learn from it and project it out there so that people can benefit from it, it feels like that should be it, but I also understand the hesitance.
Tanya Janca: So, some of the things that are on the list, most of them are supported by data, but some of them are a little higher because the community voted on various things and gave us feedback repeatedly. And if enough of the community speaks, we try to listen. And so, some things are higher than we have data to support. It would be on the 10, but maybe instead of being number six, it's number three because the community's like, "No, that is literally happening all the time. I know you're not seeing it in the scanners data. We don't care. That's happening to us all day long."
Mackenzie Jackson: One of the things that I think the OWASP Top 10 highlights a lot is that whilst things change, a lot stay the same.
Tanya Janca: I know.
Mackenzie Jackson: So, OWASP Top 10 number one, Broken Access Control, it's been number one at least since 2017. I don't know before that. Why do you think, you're a secure coding expert and it's what you've kind of spent most of your career doing. Why do you think we struggle with the same concepts? Is it because it's so difficult? Is it because we just ignore it? The training's not there? Why can't we seem to get this done?
Tanya Janca: Okay, I have a few thoughts on this. So, one, we do not teach any of this in school.
Mackenzie Jackson: Right.
Tanya Janca: So, you go to university, you go to college, you take a boot camp, we just don't teach it. Two, security training's expensive. No, but it's true that it is. It's not kind of democratized yet, etc. And that's why I wrote books to try to share.
Mackenzie Jackson: Yeah.
Tanya Janca: Three, access control is complex. Unfortunately, it is a hard problem, and unlike authentication where you can go and buy a product that perfectly does this for you, you can just do first-level authorization that way. But, once you get into a custom application, by definition, it is a snowflake. Each one is unique. It's very hard to buy a product that will perfectly deploy that security control for you. And so, you have to do the thing yourself. And I remember when I was briefly a pen tester drawing the grid of like, here's every single type of user, here's every single type of feature, here's all the different resources. Can a person see this? And I was like, "Oh my god, it's so boring."
Mackenzie Jackson: [laughter]
Tanya Janca: That's why we pay them the big bucks. It is for their patience.
Mackenzie Jackson: Yeah.
Tanya Janca: They know a lot of stuff. They're smart. There are many good things about pen testers, but it's their patience that I pay them the big bucks for.
Mackenzie Jackson: Yeah. And then I remember actually last conversation we had, you talked about your pen testing time. And one of the reasons why you left, and perhaps correct me if I'm wrong, is because you were sick of seeing the same stuff. Going back, because you'd do a pen test, and then you could almost just give them the same pen test.
Tanya Janca: Yes.
Mackenzie Jackson: Again, you know?
Tanya Janca: This is a huge thing across the entire industry. They'll come back year after year with the exact same issues. And around the five-year mark, and I did not last that long because I figured out AppSec was a job I could do, and that was meant for me. But basically, they'll see the same problems over and over again. And at the five-year mark, a lot of pen testers become very disillusioned, and either become security researchers where they're looking at very unique, cool, super interesting bugs, or they go do something else because they're just like, "I can't take it anymore. Why won't you fix the bugs I told you about?" Because it's so frustrating. They're doing all this hard work for what?
Mackenzie Jackson: Yeah. And that is frustrating. Something I regular listeners are going to be sick of me saying this, because I say it so often. But, one of the things that amazes me about security is that SQL injection was coined in 1998. We technologically solved that with prepared statements and prioritization in 1998, right? It's still like 15% of the vulnerabilities that we find out there or somewhere to that point and I can't quite wrap my head around it.
Tanya Janca: There's injection. There's less SQL injection and more all sorts of other injection. There's everything injection. Can it interfere? We can inject. I would say that again, it's a lack of education. I also think that the tool set we have needs to mature a bit. I don't mean to be rude because I know you make tools, but I feel like right now software developers are like, "I'm going to build this cool awesome app. I'm going to plan out what I'm going to build." Sometimes called design. And then they start coding or vibe coding or code whatever is happening, right? And then the outside tools come over later and they're like, "Did you know how wrong you are? Let me make you a list of all the ways you're wrong and give you a whole bunch of homework and things to fix." And I'm hoping that as AI changes, blah, blah, I know everyone's talking about that, but that we can find ways to make the code that we're building from the beginning be more secure. And that's how I started doing training is people kept saying like, "Will you know this? Will you come talk to the devs and explain this to them?" And I was like, "Yeah, of course I'll do that." But it's changing. It's really changing because previously it would be like, "Let me teach you the OWASP Top 10. Let me teach you what's wrong and why you're bad," right? And now it's like how can I review code and spot problems instead of writing code because we're not writing as much code now. We're reviewing code all the time. We're checking code. We're prompting better. And so now it's like, "Can I spot the OWASP Top 10? Can I make sure I'm giving prompts such that I avoid those? How do I look for these? How do I threat model?" I feel like the way we're doing things is changing now and I'm hoping the tool set keeps maturing with us as it has been as developers' jobs change.
Mackenzie Jackson: Yeah. I mean, it is it is funny like the AI thing. We're at RSA. We're at basically Security AI Con at this point.
Tanya Janca: Every talk has to mention AI or they just throw you out.
Mackenzie Jackson: Yeah, yeah, yeah. And like, you know, like we have a booth here. We talk about AI. Like how does Matrix themed for that reason, you know, like but it's interesting that you think it's going to, do you think that fundamentally that AI will change the security tools? Because the system that we have, and I understand why it's wrong, is that, you know, we can scan code, but we have limited context of it. Then we have our dependencies. We have all these things that we can do. But I mean, they're never going to be in that context 100% secure, and they're always going to provide information at too late. Do you think that AI will help us to get there?
Tanya Janca: Oh, yeah, for sure. So, not to plug my talk for tomorrow, but tomorrow I'm releasing a secure coding AI prompt framework for free that's a summary essentially of two thirds of my book as prompts.
Mackenzie Jackson: Right.
Tanya Janca: And so, the idea is you have a skill in code or in your memory and chat or whichever one you're doing. So, every single time you generate code, it has a bunch of requirements. It states its security assumptions, etc. And so, from the beginning you're generating significantly better code. Then there's a code review prompt. Then there's an I'm doing authentication prompt. Then there's an I'm doing this prompt. And you just fill in the blanks so that kind of holds your hand to get significantly better code. Because while every single person is using AI, not everyone's using it well.
Mackenzie Jackson: Yeah.
Tanya Janca: Right? And there are things you can do to reduce the number of hallucinations you have. There are things you can do to have it alert you as to if there are issues. Because it knows there's issues. It knows.
Mackenzie Jackson: It's amazing that if it gives you an issue, then you tell it it gives you an issue, it can't solve it. But then you, you know, you wonder like sometimes, yeah. We should put that in there. There was a funny experiment that I did where this was a year and a half ago now, so things might have changed, but I made 100 little apps with the three foundational models. And I used and they had to do the whole thing with a prompt and then we would scan them for security issues. By adding at the end of the prompt and make it secure, it reduced the number of vulnerabilities from around about 40% to just above 20%. So I was kind of like, why wouldn't it why wouldn't it do that anyway? Alas, let's get back to the a little bit of the Top 10 because we have just received a new OWASP Top 10 end of last year 2025.
Tanya Janca: Yeah, but we did the release candidate in November and then we're just like panicking towards the end of the year and we're just like closing GitHub issues like no one's business over Christmas. And they're like, "Boom, 31st we're done."
Mackenzie Jackson: Done, yeah, yeah, yeah, okay, not late.
Tanya Janca: The guys are like, "We cannot have it say 2026. We cannot have it say that, Tanya." I'm like, "Okay."
Mackenzie Jackson: Sounds like much of my university experience.
Tanya Janca: It would remind you of your project group work of university.
Mackenzie Jackson: Yes, yeah, yeah, yeah. I don't I'm not keen to go back to that, to be honest. That's quite stressful. But what were there because you can look at it and there's some things that shifted around, but there was also some new entries. Was there anything that kind of surprised you when you were looking at this as to you know, the shifts or maybe even what actually stayed to it?
Tanya Janca: So we changed using vulnerable and outdated components to insecure supply chain or supply chain failures.
Mackenzie Jackson: Yeah.
Tanya Janca: And we had all these arguments about it over Slack for weeks. We had calls about it and I'm like, "Listen, look at the really big breaches from the past several years. It is supply chain compromises."
Mackenzie Jackson: Yeah.
Tanya Janca: "It is not just libraries. There's all these different ways that they get into the supply chain and then we are still completely screwed. And if we don't talk about it, we're not going to have another Top 10 for around three years. And in the meantime, all of the bad guys, all the bad guys hurting us," right? I'm like, "We need to expand it to include the entire supply chain or we're going to be in trouble." And so I went on my social media and I asked everyone, do you feel supply chain is more than just libraries? And it was just hands down, everyone agreed. So, I'm like, "The community's spoken. Let's do this," right? So, that was good. And then we had two that tied for number 10. And one was mishandling of exceptional conditions, which won. And then the other one was a lack of application resilience. And here's the thing is that we argued.
Mackenzie Jackson: We like smart argue. Like, all right.
Tanya Janca: Oh my gosh. It sounds weird, but it's a pleasure to have technical arguments with incredibly brilliant humans are very experienced. It's a really a pleasure. I learned things from them, they learned. And we became closer friends. It's great. And essentially, I was like, "How about this? The fixes for lack of application resilience does not fix exceptional conditions. However, if you properly handle exceptional conditions, you have fixed almost all of lack of application resilience."
Mackenzie Jackson: Mhm.
Tanya Janca: "And so, what are we trying to do? We are creating an awareness document, and we're only allowed 10 things that are going to help software developers build better software. Which one's going to help people build better?" And they're like, "Damn it, Tanya." And I'm like, "But we're going to make an honorable mention list." And it was on the honorable mention list along with memory safety and vibe coding. And we don't have data for vibe coding, but everyone knows.
Mackenzie Jackson: Yeah.
Tanya Janca: And then we have a lot and a lot of stuff for memory safety, but memory safety isn't very webby.
Mackenzie Jackson: Yeah.
Tanya Janca: And the Top 10's supposed to be webby.
Mackenzie Jackson: Right.
Tanya Janca: So, we couldn't we couldn't really quite justify it being on there, but we're like, "Memory safety's still a complete disaster."
Mackenzie Jackson: I mean, especially that especially when now through vibe coding, you can start building in like memory unsafe languages. I don't know that that's the right technical term, but you know, C and those ones where you where you can do funky things with memory. And it's kind of like that kind of freaks me out a bit because there's very good reasons why you want to code you want to get down to that level.
Tanya Janca: Medical devices implanted in our bodies. I worked with a company last year and we built all this custom training around medical devices embedded in our bodies. How to threat model that, how to code that securely in C and C++. And it is and it is still a huge issue and a lot of security folks are only aware of security conferences and they're not aware of all the things the giant giant giant ecosystem that is the software developer community. But there's CPP Con. There's 50,000 software developers that descend upon Aurora every year.
Mackenzie Jackson: Yeah.
Tanya Janca: And I'm going to be the only security training there this year. And I want to have so many security talks there.
Mackenzie Jackson: Yeah.
Tanya Janca: I want us to hang out with them.
Mackenzie Jackson: Yeah.
Tanya Janca: Your company should come. We should all be there.
Mackenzie Jackson: Yeah, for sure.
Tanya Janca: Right? That's way more than RSA. All of them are going to use those tools. I'm just like, why aren't we all there?
Mackenzie Jackson: Yeah. Yeah, that's so interesting. And when you talk about hacking medical devices, the stakes go up so much, you know? We I had a gentleman Lin Nou on the show who has implants to help him hack. And it was kind of like he said he said like hacking me hacking me is like a possibility now, you know? Like you can you can get in there. One of the things that I'm kind of like always interested in is is how we can actually help developers. Now, the vendors are going to say use that tool, right? But what are in your experiences from this and maybe through your trainings? Like what are some really practical ways that developers can actually learn these secure coding practices? And maybe not just understand them, but actually implement them and and want to implement them.
Tanya Janca: Did you see my talk earlier today and that's why you brought this up? Because that's literally the whole thing.
Mackenzie Jackson: I may have stalked you a little bit before for preparation for this.
Tanya Janca: Recently, and by recently I mean like the past five years, I've been kind of obsessed with behavioral economics. There's this idea of like why we make the decisions we make, how can we support people in making better decisions? So I don't mean social engineering, which is a bit more of manipulation, where behavioral economics is like, "How can I show these options in a way such that I steer you to a better option, but I don't coerce you? I never pressure you. I never take away your power or your choices, but I present everything in a new way such that it's very clear this is the way you should go." So, as an example, in Canada, you have to opt in to be an organ donor. But, lots of countries you don't have to opt in, you have to opt out. Guess how many more organ donors they have? A lot. If you don't want to be one, no problem. You like, you know, click a button and you're out, right? But, we would have a lot more organ donors. We would have a lot more, you know, for instance, like saving for retirement. What if we opted everyone in instead of and made them opt out? We would get better things. And so, I started doing this years ago because I was like, "Oh, well, what if I nudge people instead of nagging them? What if I make a secure default where they have to undo it and it's like a pain in the butt to do it my way? What if I remove friction unless it's like I need you to pay attention here. This is super important." So, I'm only going to give you an alert if Tanya's freaking out. So, yes, I could break the CI on anything, but instead, I'm going to teach you why you shouldn't have those bugs. I'm going to give you IDE tools up until then, but I'm going to break on injection. And I'm going to break on these other things that scare the pants off me, but I'm not going to break on all these other things because if you've decided to go that far and you've been educated by me, you know the policy, you know there's a reason.
Mackenzie Jackson: Yeah.
Tanya Janca: Right? And also, you are responsible. Your team and you are technically responsible for the risk of your application. I taught you. I gave you the tools. I did you the thing. I made it visible to you. You chose to go forward. That is your business, right?
Mackenzie Jackson: Yeah.
Tanya Janca: I am going to give you the disapproving glare that your mom gave you when you were a teenager. Let's be clear. There'll be no pizza for you, but do you know what I mean though? But, give them the friction. Like, stop them when it's an emergency. Like, sometimes I'll work with companies and they're like, "We're going to break the build all day long." And I'm like, "You're going to have no friends." And they're just going to turn that stuff off behind your back.
Mackenzie Jackson: Exactly, yeah.
Tanya Janca: A software developer is a hacker.
Mackenzie Jackson: Yeah.
Tanya Janca: They don't call it an exploit. They're like, "Oh, I made a workaround, Tanya." And I'm like, "Oh god, please show me." Show me your workaround.
Mackenzie Jackson: Absolutely. I think such a good example of that, too, was like AWS had S3 buckets. The default was that they were public. And then like, we had endless breaches about S3 buckets being public. Then they made it so the default was private. Now you have to forcefully make it public. Yeah. And it still actually happens. They still get publicly. I don't know why now, but the amount, right? The amount is not.
Tanya Janca: Exactly.
Mackenzie Jackson: Yeah.
Tanya Janca: And so in the talk, and then I have like a blog series that followed that just deep dives and because like I get obsessed with things, right? And when you're curating yourself, it's like, well, I've decided that I'm going to write a blog post today, and that's my job. So there. And so you can be as nerdy as you want to.
Mackenzie Jackson: Goals.
Tanya Janca: Right? Right? And so I feel like there's just so many security faults that we don't even think about that we could set up. I also I'm seeing like a lot of people hate on vendors, but without any tools, my job would be hell. Our jobs would suck. Can we always do better? Yes. We should always be improving. We should always be trying to do better and do a better job. But like, I like that in the IDE we can plug in tools now, and we can get warnings and little red squigglies and other things. I want to improve. I want to do better. I'd love to have there be friction only when it matters more. Do you know what I mean? Or I get to choose the level of friction, right? Like, so for instance, I'm sending this to my boss in an email, so I want you to check all my grammar. Or it's like I'm sending it to my mom, and it's just like just like, you know, if there's something terrible, tell me, right? Like, can we tighten things up and make things looser? Could the suggestions be I don't know. I feel like there's like a lot of things we could do, and I feel like we really need to be very careful with the developers' attention span because they're expected to be front and back and UX this that all the other AI ML and also they're all AppSec engineers now and that's not reasonable. And so I feel like whatever we can do to make all our guardrails invisible whenever possible and we just create friction where it's like I need you to look at this. They'll be more respect and they'll turn it off a lot like because they don't have to do what we say. They don't have to.
Mackenzie Jackson: Yeah, and going to that friction point, I mean like the most friction you can add is breaking a build. But you can add friction points along the way which you know the squiggly lines, the things that they that they get in there. What what is kind of like a great tool for educating developers on on on secure coding practices? And this could be like resources like like the OWASP Top 10. I feel like that that's such a fantastic job of I think it's probably the number one resource to to kind of look at. But are there other kind of methods and things that companies can implement secure code trainings are there areas like what what like functional things can you do?
Tanya Janca: So I give trainings, so obviously number one hire me. If you're not going to hire me that is cool. But let's say you're going to design your own training. I like to try to do this thing called bad, better, best, and you can steal this idea if you want to design training. So what we used to do is we would just show them like hacking and bad things. That did not teach them what to do.
Mackenzie Jackson: Right.
Tanya Janca: We're not trying to build pen testers. That did not work. But what if we explain to like this is input validation this is why it's important. And then we show them them some code and then either you know there isn't that security control or it's poorly implemented. We show them the most common mistake that's made. And we're like, "Hey, what's going on here? Is this bad?" Because they're going to be reviewing code all the time from the AI.
Mackenzie Jackson: Yeah.
Tanya Janca: Right? And we need them to be able to see this. And then it's like, "Okay, so let's look at something better. Let's fix this together. So let's add that security control we just talked about together." And then let's check it, right? And then once you've done a few lessons, then I call this the best, but what you call it whatever you want, call it defense in depth. It's like, "Okay, so now let's take that thing that we've been slowly building together, and we're going to show you like 12 layers of defense deep of how this could be completely bulletproof. And we're going to go through and we're going to talk about each one, and you don't need 12 every time." So, it's like, "What's your threat model? How many of these does your company need? If it's a medical device that's in someone's body, they could die."
Mackenzie Jackson: Yeah.
Tanya Janca: I want 12.
Mackenzie Jackson: Yeah.
Tanya Janca: I don't know if you ever heard about that app called Yo that teenagers had, and you would download it, and you would add your friends, and the only thing it did is you could send them you would press the button, and it would send the person the word Yo.
Mackenzie Jackson: I love it.
Tanya Janca: And they could send back Yo, and that's all and it guess what? It was very insecure. You could basically like take over someone else's phone with a Yo. It was such a disaster, but like but the but the Yo app doesn't need to have the same it doesn't need 12 layers of defenses. Maybe it could have one or two.
Mackenzie Jackson: Yeah.
Tanya Janca: Right? Because the threat model and the risk is is like we don't want remote code execution, please. But I'm like, "Maybe we don't need 12, maybe we need three." And then talking about like when you need what so that they know because every security team's like, "I want everything." Like, "You don't need everything, and everything costs a lot, and we're not going to spend $10 million securing something that costs $100,000 unless lives are at stake or national security, right?" And so I feel like we're having more conversations. A thing that so, I know I talk about security champions programs a lot. It's a topic that I really like, and I bet you like it, too. But I feel like I'm security people talk about security champions programs, but what about a community of practice? Like if the software developers have one where you work. So, a community of practice is where they all meet each month, and they teach each other. What if you presented at that, and you came to their event, and you brought pizza or whatever it is that you did, right? And so, maybe you don't run an entire huge program yourself, but you attend, you learn with them, you get to know them, and every like, you know, fifth one you get to present something because you've earned that trust and you've earned that time. Because when I was a dev, I had a community of practice in the government for three years, and it was very powerful. And at first it was my team, then it was all the teams, then we started having people from other departments, and then we're actually streaming it across the Canadian government by the end. And it was just this thing that I did off the side of my desk because I was like, I want my devs to be up to speed. And I feel like any organization would super benefit from having something like that. And so with the security team partnered and became part of it. So, regular repetition, things that are up to date. Like yes, maybe once a year you blow them away and you have a trainer and everyone's excited and you have this great time, but reinforcement regularly.
Mackenzie Jackson: Right.
Tanya Janca: Or inviting them to the OWASP chapter, or having, you know, a subscription to these baby lessons or whatever. Like there's all these different ways. Like buying them books, like having a technical library.
Mackenzie Jackson: Yeah, for sure. For sure. Yeah, yeah, yeah. And there are some great books out there. In fact, two of them you wrote.
Tanya Janca: Yes, you should buy the Alice and Bob books. Me and my mom agree, they're very good.
Mackenzie Jackson: I can talk here for ages, but there is a topic that I wanted to ask you about whilst you're here. And you've already mentioned it, we kind of danced a little bit around it, but that's the vibe coding, right? The AI assisted coding. The cat's out the bag. It doesn't matter what you think about it, right? We're not putting that back in. It's it's here to stay.
Tanya Janca: It's going to occur.
Mackenzie Jackson: Yeah. And there's like my favorite joke when I'm presenting is that I have a stat. It's like GitHub says that 97% of developers are using AI tools. Here we go. What the, but just to wrap, so GitHub has a stat that says that 97% of developers are using AI tool. And I might joke that 97 developers admit to it, 3% are still lying. Because I feel like that. But the question that I have that I want to get down to is do you think this is a friend or foe for security in the short term and in the long term, right? Because the two can potentially be different.
Tanya Janca: So, I think just raw vibe coding is bad.
Mackenzie Jackson: Right.
Tanya Janca: I think that just telling the AI "Make me this." And then pasting it in and committing it is not good.
Mackenzie Jackson: Right. Yeah.
Tanya Janca: I think that the AIs are not making extremely high-quality, very secure code yet. It is greatly improving every single day. So, I do that bad, better, best thing. So, I used to just ask it to do whatever, and when 100% of the time it was missing the security control. I never had to change it at all. For the past year now, I'm like, "Oh, Claude, look at you. You did input validation. Good dog." Sometimes it actually did a good job. Sometimes it was like, "Mhm, I got to tighten it up," right? But sometimes I have to remove the control. Now I'm like, "Good good dog." Right? And I'm excited that's getting better and better. But it's still not great.
Mackenzie Jackson: Yeah.
Tanya Janca: By any means. And but here's the thing is if everyone's vibe coding, if we can build an architecture around the tool, like we the security team, that's what I mean. If we could build an architecture around it, so let's say we install some privacy guardrails. Let's say we have generation server, and we have this is how we do it here. So, this is how we do authentication. This is how we do authorization. This is how we do. So, we have things that have been pen tested that are really good that we know are good.
Mackenzie Jackson: Yeah.
Tanya Janca: Right? And then we have a bunch of prompts. And so, we have a prompt in the memory that runs 100% of the time that does X. And then we're we teach them, "Okay, so if you're going to do these types of things, these are the prompts. These are the security requirements." And it includes things from our policy or from our secure coding guideline or a requirements document. So, like every time we make an API, this is the gateway we use because you're using a gateway and you're not putting random APIs on the internet without one. They're just like, "These APIs aren't public." I'm like, "I am talking to them right now. I assure you they are public. It doesn't matter that you put a line on your architecture document. I'm saying hello right now." And we don't want that. Anyway, and so, you teach them to to use that each time. And then then you have, let's say, like an MCP server that then calls some more traditional AppSec tools that does like that last check, right? If we do all and we teach them why this matters, we make it the default, so they have to undo it and avoid it intentionally, right? So, then we're generating significantly more secure code from the beginning, and then we're giving it a bunch of checks. I think we'll get way better code. I think that then because the developers vibe coding, they have more time.
Mackenzie Jackson: Yeah.
Tanya Janca: To do this this work and to check these things.
Mackenzie Jackson: For sure, and dive deep into it and to explore the curiosities of this thing. I had a very interesting talk with Casey Ellis from Bugcrowd. He's on the podcast, and after asking him, it's like I asked him a complicated question, like, does bug bounty exist in this AI world, right? And what was really interesting about his answers is like he took a completely different approach because some people will say, "Oh, it's always going to exist. It's never going to be good enough." He just said, "Well, I think the bugs that we find that the humans find are going to be completely different. I don't think that we're going to find the same like the same things. We'll be finding We'll be going in a totally different direction from it." And I think that was like a really interesting approach, you know? And I think security will kind of be the same. We might get finally get rid of SQL injection or command injection, or you know, maybe the the the vibes are so good that we figured out that untrusted data can't be trusted. But maybe we have a whole new area that we have to learn.
Tanya Janca: I think what's going to happen is we're not going to be able to wash, rinse, and repeat things anymore.
Mackenzie Jackson: Right.
Tanya Janca: So, I used to do these consultant calls where, you know, there there used to be this game show on on TV where you could phone a friend for the answer. So, you could phone a nerd, and I was what the AppSec nerd. And so, I just like crushed these big problems for large companies. Like, you know their these companies' names, and they had that as a subscription to the service, and I would just help companies all week. And I I really liked it. And then AI came out. And then I could just ask AI what Tanya says. Does that make sense?
Mackenzie Jackson: Yeah, yeah, yeah, yeah.
Tanya Janca: And so then each question I got would be extremely advanced. And very complex. So where someone would ask me like, "Oh, we have this champions program and you know, how should we how should we format it?" Well, ChatGPT's read my blogs. So it would do that for them already and tell them all the things Tanya's, everything Tanya's written now is inside of those.
Mackenzie Jackson: Yeah.
Tanya Janca: Right? And they need to pay some royalties, but anyway. So as a result, like each call I started getting would be so complex they would be like, "You need hours of work ahead of time." And to me I was like, "Well, I'm not doing these anymore because it's not worth it for the money for how much where I'm essentially doing an entire consulting gig for a one-hour payment."
Mackenzie Jackson: Yeah.
Tanya Janca: And when it was one hour, that was good pay, but now it's not good pay.
Mackenzie Jackson: Yeah.
Tanya Janca: And so same with the bug bounties. You used to be able to be like really good at IDOR.
Mackenzie Jackson: Yeah.
Tanya Janca: And then you could just apply IDOR everywhere. But now what we can do is we can teach very carefully teach and train this IDOR pattern to the AI.
Mackenzie Jackson: Yeah.
Tanya Janca: And it will do it. And so what is going to happen is that only truly exceptional people are going to excel. So truly exceptional creative bug hunters are going to be able to make this amazing new pattern or vulnerability. And then they're going to be able to sell it across many programs. And and that is a thing that has been happening, right? But then they'll be able to train the AI to do that and find that for them. But someone that's just like Joe Schmoe that's like, "I know SQL map and I'm going to run it on it." They're not going to make any They're not going to be there anymore. I feel like the beginner junior pen tester, like my previous self, wouldn't have found a job.
Mackenzie Jackson: That's that's the exact question that I wanted to ask. Like, how does this work for entry-level now? I think this is kind of a conversation beyond tech and beyond everything that AI is at at a the entry level.
Tanya Janca: Intermediate, how's that work? Yeah. How do non-exceptional people function in this new? I don't know the answer. There are a bunch of people that give secure coding training that will charge you one tenth of what I charge and they will have the AI make the thing. They will not know the answers to your questions.
Mackenzie Jackson: Yeah.
Tanya Janca: They were not a professional entertainer for 17 years before they started doing this, right? So, you're not going to have like a super fun time with them. They're not the expert that wrote the books. You're getting a different level of quality, but some people only have $2,000 a day for training for their whole team maybe and so maybe that's what you're going to get, right? And so, we're going to have like a bit of the race to the bottom on certain things, I think, which is like what Uber did to taxis.
Mackenzie Jackson: Yeah.
Tanya Janca: Right? I think there's going to be a bit of that. I think there's also going to be the like general like exceptional things will bubble to the top and those people will create things, invent things, they'll have brands that people recognize and trust and there'll be like a certain amount of that and I really fear for the average person and I think that governments need to lobby.
Mackenzie Jackson: Yeah. Yeah, there has to be a solution, but we're not going to solve everything on this podcast, but it's so true what you're saying about the the the the trainings because I do a lot of presentations, right? I can get up there and I can sound like an expert on anything. If it's 30 minutes. If you give me a workshop, I will be so stressed because it's the questions that and that's often where the value is. The presentation is kind of like introducing a subject and then it's meeting the people afterwards.
Tanya Janca: Oh, every training, there's a there's a giant security issue that is brought up that the security team didn't know about and I always warn them. I'm like, "I need you to play it cool. Not stop They're throwing dirt dirt yelling, this needs to be a safe place." As I'm warning developers, like, so you say we're not supposed to do this. Well, what if?
Mackenzie Jackson: Yeah.
Tanya Janca: We're definitely doing that everywhere.
Mackenzie Jackson: Exactly, right? Yeah, yeah, yeah. No, that's so that's that's so true. And that's where the value is.
Tanya Janca: That is one of the big value out points. We can get an AI avatar to do that now.
Mackenzie Jackson: You know.
Tanya Janca: I actually went to a conference. So, a bunch of my friends review calls for papers and there's literally a thousand, two thousand, three thousand percent more submissions now.
Mackenzie Jackson: Yeah.
Tanya Janca: That are all written by AI. And so, I went to this conference in Europe and in Europe, I don't know if you've done this, but they'll have like a a a bowl and then they'll have like green, yellow, and red things and you put in red if you think it sucks, you put in yellow or you put in green it's awesome. And basically, I I had done my talk, I'd gone to my room to put my hotel or my laptop away because it's heavy and I was coming back at the end of the next talk and this person came out and the bowl had like a lot of reds in it and like the session wasn't over and people were coming out and I I was like, "Oh, it's there's still like 10, 15 minutes in the session. I'm surprised." And the person took the the red bowl and poured it in and used a four-letter word and I was like, "What happened?" And they're like, "That person the AI had written their talk. It's full of hallucinations. It's factually incorrect." During like we start like people just started interrupting the speaker and was like, "This, that. Like you don't know what you're talking." And the person had no idea what they were talking about. And so now like the CFP will see something and they'll think it looks really, really good. But the AI wrote it and they're and some of them have told me, "Well, we'll use this thing to see if it's written by AI." And I'm like, "Uh-oh." because I do my research and then I have the AI help me with like language. Like I'll I'll write an abstract it'll be 500 characters, but then another conference is like you only have 300 characters. And so then I'll get the AI to help me rewrite it so it's 300 characters because I don't want to spend 12 hours doing that so it's perfect.
Mackenzie Jackson: Exactly.
Tanya Janca: And then you'll be like flagged AI wrote that. I'm like, "No, that is my research. Like damn it."
Mackenzie Jackson: It's been a pleasure just talking to you. But I would I would got to tell we have to we have to finish up at some point. But I have a fun game that I always like to play. It's called Would You Rather? I'm going to give you two very crappy options and you need to pick one. Okay, so the first one, we're going to go back to the vibe coding thing. Would you rather vibe code an app, and when I mean vibe code, I'm talking about you you can only adjust code via a prompt.
Tanya Janca: Okay.
Mackenzie Jackson: Or you have to write the application character by character as Sam Altman put it recently. Which would you rather?
Tanya Janca: Oh, I would vibe code it.
Mackenzie Jackson: Yeah.
Tanya Janca: I'm only allowed prompts.
Mackenzie Jackson: You're only allowed prompts.
Tanya Janca: I mean, you can do a lot with the prompt.
Mackenzie Jackson: I feel like I need to restructure this for the next time because I think like someone like you, you could just prompt it to change.
Tanya Janca: Yeah.
Mackenzie Jackson: The the thing. But if you have I change this line to this.
Tanya Janca: Yeah, but if you if you if you had to like do the entire thing without knowing any code, I think then maybe it changes. I got a harder one now then. Okay. Would you rather have all your secrets, your API keys, stored in a .env file in your Git repository. It's private, but it's in there and it's in the history.
Mackenzie Jackson: Is it It's a It's private, but is it like for only CX Herbal Consulting where there's just like four of us that work there or is it like I work at Enterprise and it's private?
Tanya Janca: Enterprise.
Mackenzie Jackson: Okay.
Tanya Janca: We're going Enterprise because, you know, like at least just say, you know.
Mackenzie Jackson: Potential insider threat.
Tanya Janca: Yeah. Or your application has no firewall. So, you can't block DDoS attacks, you can't kind of filter out bad traffic or bad requests or things.
Mackenzie Jackson: But I can code it any way I want.
Tanya Janca: I can code it any way I want.
Mackenzie Jackson: I do that.
Tanya Janca: Yeah.
Mackenzie Jackson: Oh, yeah, for sure.
Tanya Janca: So, you would choose the the the no firewall over the. So, you mean no web app firewall?
Mackenzie Jackson: Yeah.
Tanya Janca: Yeah, I generally don't recommend that. To me a WAF is like a Band-Aid. So, if you cut yourself, you would put a Band-Aid on until it heals. So, let's say you have a really old piece of crap app that's terrible. I put a Band-Aid on it. I put a WAF on it.
Mackenzie Jackson: Yeah.
Tanya Janca: Until we decommission it or maybe we refactor the whole thing and then do a good job. I feel that a WAF is not instead of good code.
Mackenzie Jackson: Right. Okay. No, I feel like that comes from someone that's confident about their coding abilities. I think that's a good question there, Richard. I have one more. This one's the worst one, in my opinion. It goes back to the kind of supply chain attack. Okay. Would you rather had all your dependencies locked in their current versions forever? So, if the new vulnerability appears, it's locked there. Or, you auto pull the latest version every time. So, if malware gets injected, if if there's breaking changes?
Tanya Janca: I would rather auto pull because, although supply chain attacks are becoming more common, they're still not ridiculously common. And you can still, like, Metasploit a lot of things. And the current version is going to have some Metasploit modules. Like, it's going to have like some common known attacks. Especially like as time progresses, I Yeah, I'd rather auto pull.
Mackenzie Jackson: Okay. All right. Very good. Very good answers. Thank you so much for joining me. I really appreciate it. I was I was thrilled when you agreed to do it.
Tanya Janca: Oh, thank you so much for having me.
Mackenzie Jackson: I can have you back on the show again sometime.
Tanya Janca: Me too. Absolutely.
Mackenzie Jackson: Awesome. Thanks, Tanya.
