Mackenzie Jackson: So, Casey, thanks so much for joining me here on this episode of The Secure Disclosure.
Casey Ellis: Great seeing you, man. Yeah, it's good to chat.
Mackenzie Jackson: I first just kind of want to start with maybe a little bit of background for people that don't know you and your kind of journey.
Casey Ellis: Sure.
Mackenzie Jackson: Obviously, you're one of the founders of Bugcrowd.
Casey Ellis: Yep.
Mackenzie Jackson: But maybe even like before that, how did this whole journey in security actually kind of start?
Casey Ellis: Yeah, okay, cool. I mean, I was born at a young age, so there's that. That's my typical kind of backstory intro joke. But, you know, grew up hacking stuff. So I'm a child of the early '80s. I kind of grew up, you know, started school analog, ended digital, went through that whole kind of period, right? My dad was a science teacher. He'd bring home tech from school. My mom just fostered this creativity that they saw in me early on and tried to create every opportunity they could for me to tinker and play with stuff, as well as kind of wrangling the fact that, okay, this kid obviously thinks like a criminal but doesn't want to be one. What are we going to do with all that, right? So, that was kind of the early start. You know, fast forward to the end of high school, I basically pretty much went straight into a network engineering role. This is like 2001, and then realized, you know, I'd been hacking stuff, like got onto IRC, BBS, in that era of educational learning kind of thing. And yeah, started hacking stuff in the context of doing IT work and realized that people actually can get value from this. If I can like it's fun, it's we kind of stuff I love to do at that point in time. And I could actually teach folk how they could make their systems more secure. So, that was where it all got started. Did the tech side for about six years, and then basically, it was actually when I got married. My wife sat me down at one point and she said, "Hey, like you people good too. You're just someone who computer's good. You should go try sales and see how you feel about that." So, I did. Felt like a dirty white hat sellout for a little while in there but realized that if you solve a problem, it doesn't matter if that solution isn't connected to where the problem exists and that is all sales and marketing. So, yeah, fast forward all of that. I ended up, you know, breaking bad, becoming an entrepreneur, and you know, that led to Bugcrowd and all the things that have happened since.
Mackenzie Jackson: And what led you to go into Bugcrowd?
Casey Ellis: Yeah, I think I mean, there was a couple of things. One of the big things was just having grown up in the hacker community through that era, right? We didn't, everyone who kind of joins now, they sort of take it as a bit of a given that we're understood, we've got a seat at the table, all other stuff. That's only been true for maybe, you know, 8 to 10 years.
Mackenzie Jackson: Yeah.
Casey Ellis: When I was coming up in there, we weren't understood. People assumed we were a criminal. Yeah, we kind of liked that in some ways but it limited our ability to actually do things that helped, if that makes sense. I just kind of wanted to do something about that problem.
Mackenzie Jackson: Right.
Casey Ellis: So, that was a big piece of it. The other side of it is like just from a solution kind of entrepreneur standpoint, was running a pen test company prior to starting Bugcrowd and looking at, looking at kind of the way that we do those things and thinking, "Okay, if our job is to outsmart this crowd of adversaries that have lots of different skill sets, lots of different motivations, and an incentive to succeed, and we're trying to compete against that with someone being paid by the hour, it doesn't matter how smart that person is, they're going to fail because the math is wrong."
Mackenzie Jackson: Right.
Casey Ellis: Right? Yeah. Yeah. Meanwhile, there's this community that's sitting there unplugged from the problem, and it was, yeah, pretty much a series of conversations in Australia, and kind of a light bulb moment where I'm like, "Oh, let's like put everything in together and see if that works, then see if we can get the market to actually accept the idea."
Mackenzie Jackson: Yeah.
Casey Ellis: Of like, firstly, hackers and then crowdsourcing and bounty and VRP and VDP and all of that, and then kind of grow from there. So, that was 2012.
Mackenzie Jackson: Being in that space, you know, obviously having that hacker mindset, what was available to hackers kind of like before that? Before the Bugcrowd or even just before the bug bounty movement?
Casey Ellis: Yeah, so, and it's important to call out like Bugcrowd didn't invent bug bounty or VRP. Like that was prior art. You know, we were the first to come out and basically say, "Let's put a platform in between the community and all the problems that need to be solved and try to accelerate it." But yeah, prior to that, I think it was PayPal, Google, eBay. There was like a list of tech companies that were running, you know, VRPs, which is what we called them back at that point in time. That was available, so if you wanted to hunt and get paid and all that kind of stuff, that was an option. You had, you had the MSRC off the back of the work that Katie and Kimberly did and there they they'd really ramped up their vulnerability intake program, and yeah, they weren't paying at that point in time, but they were doing a lot around, you know, honoring the researcher, which was really cool. So, there was a bunch of different things that were kind of available, but there was no momentum behind it, and it was really very much an underground community kind of doing the thing at that point. So, we actually mostly learn from each other. Like that's probably the more succinct answer to your question.
Mackenzie Jackson: Yeah. Because what the platforms did is made it a real thing.
Casey Ellis: Yeah.
Mackenzie Jackson: You know, like it wasn't just someone that was like in between breaking stuff on the internet for fun.
Casey Ellis: Yeah.
Mackenzie Jackson: Would get paid.
Casey Ellis: Yeah, they just have to advocate for themselves.
Mackenzie Jackson: Not a yeah, guy and all this other stuff.
Casey Ellis: Yeah.
Mackenzie Jackson: And then also like with the vulnerability disclosure programs, it's kind of like there was no transparency around like, you know, that only passes on. So, that.
Casey Ellis: That's right.
Mackenzie Jackson: Platforms are really important.
Casey Ellis: But yeah, I think, I think a big piece of what we managed to achieve in that period was legitimizing the overall idea of a hacker being like potentially a strategic asset.
Mackenzie Jackson: Yeah.
Casey Ellis: For an organization as opposed to just this random threat on the internet that you've got to figure out how you're going to deal with it every single time.
Mackenzie Jackson: Yeah. Definitely. As soon as you brought up that, that, that not being perfect, I mean, bug bounty has been a pretty foundational part of security programs.
Casey Ellis: Yeah.
Mackenzie Jackson: Because you have your tools and you have your pen tests. But, and they have their place. Like, you know, I'm from a vendor and I've also been involved in pen tests.
Casey Ellis: Yeah.
Mackenzie Jackson: Love them both. However, it, it doesn't give you that wide coverage, that continuous coverage from it. So, a really important part of it, but it is, there's been some challenges kind of I guess more recently in, in, in dealing with this.
Casey Ellis: Yeah, and what I would say is that it's not actually that recent. The like, the, the, the reactions to AI slop, you know, there's just this overall kind of increase in submissions like across the board.
Mackenzie Jackson: Mhm.
Casey Ellis: Over the past, you know, six months to a year, in particular, feels very much like it did back in 2014, 2015 when folks went from one person doing the work to multiple people running the work, right? Oh crap, we're not ready for this. We haven't actually prepared. We're not, we haven't thought through vulnerability management. We haven't thought through triage, community management, any of that stuff. Yeah. To the degree that we need to for what's coming. And there was a period of adjustment there. So, it's like this is a I think a lot more dramatic because AI has kind of put the ability to find, submit, you know, do all those different things in the hands of way more people. And back then it was kind of just going from not crowdsourcing to crowdsourcing. So, it was a bit of a step function shift in volume change, but the reaction I think is kind of similar.
Mackenzie Jackson: Similar to that. So, what is like, we've had Daniel Steinberg on the show and he was talking about that they shut down the curl bug bounty program. Now, one of the things I do want to point out about that interview, too, is that Daniel's not at all against.
Casey Ellis: No.
Mackenzie Jackson: You know, bug bounty programs and all of that. And I think a lot of the people, there's a lot of news headlines out there with different organizations trying to adjust in different ways. I don't think any of them are really against.
Casey Ellis: No, I agree.
Mackenzie Jackson: Bug bounty. It's just the fact that it's becoming harder and harder to manage.
Casey Ellis: Yeah, I agree with that, too.
Mackenzie Jackson: The, the programs.
Casey Ellis: Yeah, so I think, and this is the thing that I've actually said a bunch of times over the years that often surprises people when they hear me say it, but I'll say it again because I think the, the stuff that's happened, that happened with Daniel, with Curl, and the things that are happening just in general, kind of validate the point. I believe that every organization should have a VDP, and in fact, every organization already has a VDP whether they like it or not. They're just maybe not getting the report. Because if you're on the internet, you can assume that you've screwed something up and someone outside your organization will find it. The end. Like, yeah, like for me that's like an issue of internet physics, you know? It's like spurring a vendor or any of that other stuff. To me that's just kind of how it works. But then there's option of adding incentive on top of that to encourage people to do that. I don't think that every organization should do that. And you can see the distinction because when people say bug bounty these days, it's like, is that crowdsourced security? Is it a pen test or a bug bounty platform? Is it a VDP? Is it a public bug bounty program? It's all kind of gotten lumped into the one term of art because that's honestly the term of art that we used back in the day to to kind of get this idea across the table in the first place.
Mackenzie Jackson: Yeah.
Casey Ellis: So, I think there's like, what are we actually talking about here? That's, that's a pretty important bit of separating. But yeah, like I said, I don't think every organization, I think, you know, the Curl program shouldn't have been a bounty in the first place because if you got like volunteer maintainers.
Mackenzie Jackson: Yeah.
Casey Ellis: It's like, you can only, you know, and good on them for for making it as long as they did without kind of throughput. But, you know, not every organization's ready for that. Not every organization's necessarily in a position where they can handle that and that's just, that just is what it is.
Mackenzie Jackson: So, and what what makes that distinction is and like if you're an organization, when is the time to move from kind of having a vulnerability disclosure program and it having a paid incentive on it?
Casey Ellis: Yeah, I think that's a really good question. Because like the other, you know, you talk about the different ways that we kind of carve up this crowdsourcing idea now and there's like AI being thrown on top of all of it as well. Yeah, the private version of it I think every organization can get some sort of value from that because they can control it, they can throttle it. But, when you're doing this with the open internet, it's like the metaphor I use is, you know, it's kind of like walking outside when there's a thunderstorm coming and telling it not to hit your house with lightning. Yeah, that's not how this works.
Mackenzie Jackson: Right.
Casey Ellis: Yeah, a VDP is like putting up a lightning rod expecting that like a bug bounty program is like shooting rockets with wire on them up into the cloud. And if you're not ready to do that, then maybe you kind of shouldn't.
Mackenzie Jackson: Yeah.
Casey Ellis: I think organizations that are ready for a public bug bounty program, it really goes to the maturity of your vulnerability management process internally. Yeah, do you have a proper understanding of what risk actually means to your organization? How do you prioritize? Do you have the ability to do the community management?
Mackenzie Jackson: Mhm.
Casey Ellis: Pieces on top? Yeah, there's, there's a variety of different things. I think the, the outlier there is if you've got a competitive vulnerability acquisition market for the product that you make. So, if you've got actual offensive procurement going on that you want to compete with that, that's another reason that's probably not a bad idea.
Mackenzie Jackson: That's such an interesting kind of thought though. So, here your your competitor in this space is like someone selling your exploit.
Casey Ellis: That's part of my backstory as well coming into this in terms of how I think about the economics of like to me it's not about like, is this a pen tester or a blah blah blah blah blah. It's like to me the real question is what is this bug actually worth?
Mackenzie Jackson: Yeah.
Casey Ellis: And that, you know, the answer to that is partly like, to whom, right? So, yeah.
Mackenzie Jackson: So, I mean part of this whole discussion it's, it's great to kind of to, to get some background on this because it's not actually the, the, the AI slop reporting. It's a new, it's a new issue because it's AI, but it's also it, it just, it's just highlighting an old issue. It's increased an old issue.
Casey Ellis: I think that's right. I think we're doing like, we're doing stupid things faster with more energy, right? We, you know, Bugcrowd was working with with the foundational models, you know, four or five years ago was kind of when that kicked off. And then at one point I had a demo of an offensive tooling set that got created at a hackathon at one of the providers, right? Yes, it's like four years ago. So, before all of the crazy make around stuff that you see going on now. Like all of this is coming fast, right? And the thing that that made me realize is that in security in general we kind of rely on this concept of the OODA loop, you know, observe, orient, decide, act, right? And there's like cycle times involved in that, like, you know, deploy, defend, like attack, learn, fix, blah blah blah. What AI does or was going to do and is now doing is like tightening that loop up so small that we can't actually fit inside it.
Mackenzie Jackson: To. Right.
Casey Ellis: To take action, right? So, we've got to, you know, I think in some ways kind of rethink the game.
Mackenzie Jackson: What does that rethinking of the game look like? So, we've got this here. And and something that stood out, stood out for me in what Daniel was going through is a part of it was that they always got crappy reports.
Casey Ellis: Yeah.
Mackenzie Jackson: But then actually, but now they're very, very convincing.
Casey Ellis: Yeah.
Mackenzie Jackson: You know, and.
Casey Ellis: Yeah, yeah, 100%.
Mackenzie Jackson: And that's and that's part of the problem, too, because now you could look at something and know pretty quickly.
Casey Ellis: Yeah. And again, that's always been true. Like I remember vulnerability reports, you know, 2013, 2014 with people like photoshopping fake.
Mackenzie Jackson: Yeah.
Casey Ellis: Like excess returns and and stuff like that. Like the idea of, you know, like we're effectively if you're running a bug bounty program, you're effectively creating a game for people that are wired up to break games.
Mackenzie Jackson: Yeah, okay. Yeah, yeah. So, can I, this is kind of a bug in the process?
Casey Ellis: Yeah. And that's just part of the deal. Like it is. It's one of the reasons why, you know, it's been so, so interesting kind of building out the platform cyber because you get to see all these like weird edge cases that pop up, you know, in the conversation itself. I think you, like the challenge with AI is as you said, it makes it really easy just to spray and pray.
Mackenzie Jackson: Yeah.
Casey Ellis: You know, like the whole idea of like big bounty.
Mackenzie Jackson: Oh, yeah.
Casey Ellis: Found this thing, give me money, and then I'll tell you what it is. Like.
Mackenzie Jackson: Yeah.
Casey Ellis: They're all kind of versions of the same thing. And to me there's two solutions to that or two categories of solution. One is basically community driven. Yeah, if you see, if you're involved in this community and you see your peers doing that, like they're actually harming you.
Mackenzie Jackson: Mhm.
Casey Ellis: In the process because everyone who's participating kind of gets tarred with the same brush, right?
Mackenzie Jackson: Right, right.
Casey Ellis: There was a season where, yeah, bounty hunters would call each other out if there was like extortion or big bounty or any of that kind of stuff and it was sort of self-policing. I'm right. That's not as true at the moment, I think. But I do think like elements of that really help.
Mackenzie Jackson: Yeah.
Casey Ellis: And the other side, of course, is technical controls. Like how do we, you know, how do we improve the ability to automatically triage? You know, think those things through. I don't necessarily think, I think platforms can be really helpful with the 80%.
Mackenzie Jackson: Mhm.
Casey Ellis: But it's going to matter with the 20.
Mackenzie Jackson: Yeah.
Casey Ellis: On the receiving side. Right? Because it's that's where the edge cases are and that's where you can kind of commit fraud like that and sometimes be successful.
Mackenzie Jackson: Yeah. Yeah. Yeah. Yeah, for sure. Because you, you know, yeah, you can just spray and pray and at some point it's just so overwhelming your bug bounty program maybe it's just like.
Casey Ellis: Yeah. Like the asymmetry of it's always been the challenging part.
Mackenzie Jackson: Yeah.
Casey Ellis: You know, like we, like Bugcrowd built out a triage team before we built the platform.
Mackenzie Jackson: Right.
Casey Ellis: Because we knew that like this, this, you know, we're dealing literally in unintended consequences as a service. So, the fact that it's weird.
Mackenzie Jackson: Let's talk about like how do we, like what do you think kind of the solution to these could be? Like where, where does it go? Because it's obvious that something needs to to change. I think like one of the, we've got the area of AI pen testing, which I think is interesting. I don't think it's ever going to replace the human element.
Casey Ellis: It'll replace parts of it, but yeah, I'll, I'll let you finish.
Mackenzie Jackson: No, yeah. But so the, the solution that a lot of people have been floating and also from from vendors as well is that we can automatically test bug reports and yeah, kind of get that AI in the triage process. Is that the solution? Is the solution? Other companies have kind of said, well, we're just not going to accept reports from certain tier of people.
Casey Ellis: I think those are like, you talked about the technical and the community category. Yeah. They're like, in terms of like, you know, automated triage and reproduction where it's possible. The thing, the thing about the triage pipeline, and this is like as someone who has seen what that looks like between like the entire internet and like 3,000 different companies, right? So, there's a lot of, we've seen most of the edge cases that you could think of, but also operate on the assumption there's stuff that we haven't seen yet just because of the nature of what we're doing. Yeah. So, like to me the problem's never solving it to 100% because you just can't. Right? Like a bug's not meant to be there in the first place. So, like to, to operate on the assumption that you can, you know, fully predict it, that's like, cool, we can all just pack up and go home now. Yeah, I think the triage piece is really, really, you know, potentially useful especially when there's code available for that. You know, dust can get weird sometimes as edge case types of attacks that are hard to reproduce automatically, but there's a lot that aren't. And if you can deal with that and qualify things in or out, awesome. You're golden. I think the community piece, you know, in terms of like there should be, you know, to some degree like a whole of not necessarily a whole of shame, but this idea of like you're a known bad actor. Right. In this, in terms of like we've told you, you're not just like misinformed and super enthusiastic and kind of annoying and needing an education, like you are actively ignoring that and making our lives hell. Like, cool, put them in the dev doll for that purpose. Like I think that's a really good solution. I think doing that in a way that's visible to others so that they can understand what's valuable and what's not is pretty, pretty useful as well. I think the other thing is just with bounty itself, right? If you do have an incentive, it's not that you're disincentivizing the things that you can find through automation or the things that you don't want to hear, it's that you're just incentivizing everything else more. Right. Right. It's like no, we're not going to pay for AI slop level bugs because I actually do think that AI is going to get better at, you know, sparkling QA. Yeah, right. As opposed to like hacking in that sense. And it should. Like if, if, if we've got a greater ability to find those types of issues so we can deal with them and then we should produce the cost of doing that to as close to zero as we can. That just makes sense.
Mackenzie Jackson: Yeah.
Casey Ellis: But you don't want to pay a thousand bucks every time that's happening.
Mackenzie Jackson: No.
Casey Ellis: If you've got some other stuff that you're really concerned about that you'd rather focus people on.
Mackenzie Jackson: Yeah.
Casey Ellis: Do you know what I mean?
Mackenzie Jackson: Yeah, I know. I totally, I totally understand what, what, what you mean. And then the, the part that like worries me too about all of this is from that community side where bug bounty and, and in these areas they've pushed the boundaries and if we made it harder for the entry for the new gen to come in to this.
Casey Ellis: 100%.
Mackenzie Jackson: I like that kind of worries me a little bit.
Casey Ellis: Yeah.
Mackenzie Jackson: And when I say is it like I love AI, like I love the category of AI pen testing. I love that we can now find vulnerabilities that scanners couldn't.
Casey Ellis: Like I do.
Mackenzie Jackson: Yeah, it broken access control. Things that are kind of not obvious for a scanner. But it's, it's not in my opinion going to replace the, the human element of really just trying to break something in very weird and creative way.
Casey Ellis: I think that's right. I, I think the, you know, and we've seen like a revision of this that we've seen in the past like early, again I feel like old man telling a story right now but whatever. Early early on a lot of hunters were really successful by looking where everyone else wasn't.
Mackenzie Jackson: Yeah.
Casey Ellis: Whole idea of like, here's your, you know, www dot domain. Everyone's going to be looking at that. I'm going to find things that are related to this company that are so far-flung on the internet that no one's going to think to look there. And like people making a million bucks, you know, hand over fist in that season because that was pre-ASM.
Mackenzie Jackson: Right.
Casey Ellis: As a market.
Mackenzie Jackson: Yeah.
Casey Ellis: So like anyone who worked in, in security or in industry knew that like there's a whole bunch of that people have left out on the internet and forgotten about. Like we all knew that but that problem wasn't obvious. And then the bounty hunters came along and kind of set the whole thing on fire.
Mackenzie Jackson: Yeah.
Casey Ellis: And now we've got ASM.
Mackenzie Jackson: Yeah.
Casey Ellis: The category, right? So like, cool, good outcome. You know, you could argue that they kind of cannibalized their own opportunity.
Mackenzie Jackson: In doing that.
Casey Ellis: Yeah.
Mackenzie Jackson: Like, there's an element like that's sort of part of why we're meant to be doing all of this in the first place, right? We don't want to be vulnerable.
Casey Ellis: Yeah, I know. That's the entire point.
Mackenzie Jackson: Yeah.
Casey Ellis: So, there's this like, I think there's this continuous kind of refreshing of low-hanging fruit. And, you know, kind of to what I think you were saying like that creates the opportunity for people to upskill and actually apply their creativity. Like, cool, you've got a bunch of lame bugs, but can you chain them together to get a better impact?
Mackenzie Jackson: Yeah.
Casey Ellis: Or, like, can you learn from that and start to learn new classes of vulnerability that are more difficult for AI or scanners to find and that kind of thing.
Mackenzie Jackson: Yeah, and and I think like and speaking of AI, it's like for the first time in a very long time we've got a new class of vulnerability, which is the whole prompt injection.
Casey Ellis: Yeah.
Mackenzie Jackson: Thing. So, all of a sudden it's kind of like.
Casey Ellis: I mean, we, so yeah, there's that.
Mackenzie Jackson: Like.
Casey Ellis: You think about this. Like, we're not even touching still, we're not even touching hardware.
Mackenzie Jackson: Yeah.
Casey Ellis: Like, I think when when I think about the overall kind of threat landscape, like again, like why are we doing this all in the first place? It's not just to build cool things. It's to actually make the internet safer.
Mackenzie Jackson: Yeah.
Casey Ellis: Most of the conversations focused on like the top five turtles in a stack of like 50.
Mackenzie Jackson: Yeah, right.
Casey Ellis: Like, China is in the bottom 20.
Mackenzie Jackson: Yeah, yeah, yeah.
Casey Ellis: There's, there's a lot to work on that I actually think needs attention is probably having the oxygen sucked out of the room by a lot of this kind of AI app sec.
Mackenzie Jackson: Yeah.
Casey Ellis: Conversations going on. So, I actually don't think that's a bad thing. If that causes people to go look elsewhere in terms of what they get skilled at as hackers, that's probably a good thing.
Mackenzie Jackson: There's this, there's such an interesting point because I think, you know, when you when you look at it and you try and go to the to the future but we look at kind of what we're finding now, what we're doing now.
Casey Ellis: Yeah.
Mackenzie Jackson: And we think that that's going to stay the same. So.
Casey Ellis: Yeah.
Mackenzie Jackson: You know, in five years time, you, we've got no more bug hunters because there's nothing left to find.
Casey Ellis: That's right. And this is where I get because I mean part of my job, you know, part of how I see my job in the founding role and general is just like try to see around corners, right? So, I'm like always thinking about how does this stuff evolve over time, not just what's happening right now. And so, when I get asked the question like, is AI going to replace pen testing like research specifically? I'm like, "No, because even if we were able to solve every security problem that we have today with with AI, which we're not even scratching the paint off that."
Mackenzie Jackson: Yeah. Yeah.
Casey Ellis: Theoretically, if you could, what are the bad guys going to do? Yeah. They're just going to pack up and go home?
Mackenzie Jackson: Right. Yeah. Yeah. Yeah.
Casey Ellis: I don't think they are.
Mackenzie Jackson: No. No.
Casey Ellis: And they've got human intent. They've got human creativity. Like, yeah, it's their job to operate in the edges that we leave behind. And they're going to always be pursuing that.
Mackenzie Jackson: Yeah.
Casey Ellis: Therefore.
Mackenzie Jackson: So, yeah. Yeah, that's it's so it's, yeah. Yeah. Yeah. That, that where you talk like totally right. I think.
Casey Ellis: Like philosophically, but, you know, like thinking about like, I got, what, what does that mean we can do next? It's the kind of stuff I was just talking about.
Mackenzie Jackson: Yeah, and like whenever I kind of think about this, too, is something that I, I say a lot is it's a little bit facetious, but, you know, I say that, you know, SQL injection the, the term was coined 1998 or around about that. We solved it that same year with prepared statements, basically. You know, like it's, it's technically a solved problem, right? You know, it still makes up like 50% correct, which is the best kind of correct.
Casey Ellis: Yeah. Yeah. Yeah.
Mackenzie Jackson: It still makes up like so much vulnerability. So, you know, we can solve all these things from a technical point of view.
Casey Ellis: Yeah. We can get better at finding them.
Mackenzie Jackson: And they will still exist.
Casey Ellis: Yeah.
Mackenzie Jackson: It's just kind of like as a rule of nature.
Casey Ellis: Yeah.
Mackenzie Jackson: There's a long tail that needs to be cleaned up. You know, we've got like quantum coming over the hill at some point in the future. We've got, you know, AI getting slapped into robotics and mechanical. There's a lot of like cool frontier tech stuff that people can kind of dig into and work on. And that's always going to be novel and that's new. And in the meantime, like, you know, half of the, the hardware, um, vulnerabilities to get hosed by flux typhoon or like format strings from the '80s.
Mackenzie Jackson: Right.
Casey Ellis: Yeah, really. It'd be like you said, to your point, like there's a lot of solved problems that aren't.
Mackenzie Jackson: Yeah.
Casey Ellis: Solved at all. And yeah, I think those create a lot of opportunities for hackers to go and specialize, figure out how they can be a part of contributing to the solution.
Mackenzie Jackson: Yeah. Yeah, for sure. Recently like IBM got crashed momentarily because Cloud could now support COBOL. And then I had no idea that there was that much COBOL left in the world. Yeah.
Casey Ellis: I mean, I, I gave, I gave a keynote at an Indian conference in 2020 and like literally what I told people was, learn ASP.NET, Java, and COBOL.
Mackenzie Jackson: Right. Yeah. Yeah. Yeah.
Casey Ellis: It's like it's boring. It's not cool. Like these are not the cool languages to go off and learn right now. But if you go off and look at what's actually powering the United States, those two, and then you look at what's under the hood, it's COBOL. Like it's everywhere. And like the thing with that is that the people that know how to do that stuff, they're either retiring or passing away, literally.
Mackenzie Jackson: Yeah.
Casey Ellis: Like the fact that AI understands COBOL now, of course it does. It's been around forever.
Mackenzie Jackson: Yeah.
Casey Ellis: But it still requires like discretion of the human to actually wield those tools to make those things work and you understanding of the system to be able to do that well.
Mackenzie Jackson: Yeah.
Casey Ellis: Yeah, it's just stuff like that.
Mackenzie Jackson: I want to kind of start to to wrap things up, but I'm, I'm very curious kind of what, what your position on where we're going in the future. We've touched, we've touched on this, but like where do we end up in, in, in the bug world? How does it kind of fundamentally change over the next five, five years?
Casey Ellis: There's a lot in that one, right? I think, you know, kind of what I said before, what I would love to see happen and what I think will happen is just a question of whether it's orderly or chaotic.
Mackenzie Jackson: That we get there.
Casey Ellis: Is this idea of vulnerability disclosure being just a thing that happens?
Mackenzie Jackson: Yeah, I'm sorry for, you know, if you're building a company, you recognize that people outside have the ability to identify risks that you don't know about and you've made that safe for them to tell you legally. Plus you've got, you know, to the point of the slop and all those different things. You don't need to actually triage that and then like get to the signal and the noise. I think, you know, all of the conversation around bouncing and all that other stuff is sort of almost hiding like the, like the axiomatic truth of the fact that that's sort of what we're actually talking about.
Mackenzie Jackson: Right. Yeah.
Casey Ellis: Yeah, what is pen testing? It's assurance. It's no, it's like to err is human and other people are going to figure that out and then they want to help you. The end.
Mackenzie Jackson: Yeah.
Casey Ellis: So it's like to me that's, you know, physics is pulling in that direction. I think pen testing and, and just assurance in general, it'll be interesting to see how it plays out, but I think there's a correction kind of in progress now with AI putting kind of doubts pressure on on pen test.
Mackenzie Jackson: Mhm. Yeah.
Casey Ellis: In the market. I think it was frankly a little bit inflated to begin with. I thought that was true when I started Bugcrowd as well. I was going to do the same thing with the crowd. Improve effectiveness, right? And, you know, like the, the, the hidden thing behind all the slop stuff and and that story is that there's way more vulnerabilities out there than we care to actually talk about. And if you're in security, you know that, but people outside don't realize that. It's like the, the fact that the internet still works is a minor miracle.
Mackenzie Jackson: I can't believe in the amount of times how close we've come to it breaking like the XC utilus example when like the, the compromises have happened in like NPM at the moment.
Casey Ellis: And when you go down the rabbit hole, you're like, holy crap. Okay, this is wild. So, I think that's going to become more obvious and like the sense of value is is, you know, potentially going to align itself better like will become more focused on being able to fix things quickly, being able to like build them in a way that's secure in the first place, which is a lot of what's happening right now, which is yeah, there's still a lazy trash to clean up, but that's, you know, we'll see how that plays out. And then honestly, I think, you know, the idea of like, why are we doing this all in the first place? Like what are the threat actors up to? Who are we actually defending against here as aside from, you know, corporate reality, integrity, business availability? Like they're talking about an actual motivated threat actor the role ostensibly here to defend against. Yeah. What do they want? Like how do we impose cost? How do we disrupt their economics? Or at least make our own hurt reflect theirs better. Like yeah, it's very high polluting incentive, but I do think the, the vulnerability marketplace is going to get kind of interesting as things get spicier. Yeah. You talk about like nation on nation stuff, you talk about, you know, letters of mark in like Congress and the White House right now, different countries saying, yep, countries that companies can hack back. When they deploy IP written laws and things. You talk about the idea of like non-cooperative defense. So if there's a whole bunch of systems that are vulnerable or compromised and there's threat associated with that, we can actually break into them ourselves and fix them on behalf of the user. Like there's a whole bunch of, I'm not necessarily advocating it. Don't go off and do any of this right now, but when you look at kind of the raw ingredients of what we're working with, I can't see how some of those things don't end up becoming normal.
Mackenzie Jackson: For sure. Over the next five years, you know what I mean? So it's, yeah.
Casey Ellis: Yeah, yeah, the hack back, the right to hack back is always kind of been something and you can see.
Mackenzie Jackson: I don't like it. I don't like it, but I don't see how we don't end up there.
Casey Ellis: Yeah, yeah, yeah, yeah, yeah, for sure. Well, look, we've come to the end, but we always like to play a little game on the show called Would You Rather? I give you two options, both of them are going to suck and you have to pick one.
Mackenzie Jackson: Fantastic.
Casey Ellis: Okay, so would you rather store all of your secrets, your API keys, credentials in a .env file inside a private Git repository? So they're in your Git. They're there, it's in all your history. Or you've got no firewall on your running application?
Mackenzie Jackson: No firewall.
Casey Ellis: You're going to pick no firewall? Oh, that was a very quick answer.
Mackenzie Jackson: It's because I know where the bodies are buried.
Casey Ellis: I always think about it like whenever you look at like a big breach, like a secret's always involved somewhere, right? Yeah, yeah, yeah. All right, let's do another one. For your dependencies, if you auto update to the latest version every time.
Mackenzie Jackson: Right.
Casey Ellis: So there's kind of risk of break, risk of malware, or you pin your versions now and you can never touch them again?
Casey Ellis: Ooh, that's a hard one. Um, never again.
Mackenzie Jackson: Never again?
Casey Ellis: Update.
Mackenzie Jackson: Yeah, I'd go with that. I'd pin this. Yeah, this is at, okay, so this is, what, what's your root thinking behind that?
Casey Ellis: I think the, the internet's collectively had a our moment around the whole supply chain thing. And it's far from a solved problem right now, but I think there's a lot of attention going into it. And in the meantime, like actual vulnerabilities will probably continue to exist. Yeah, I, I that would be a bet on the fact that we actually get better at not allowing malware into the supply chain at some point in the not so distant future. I'd just be super vigilant in the meantime.
Mackenzie Jackson: Yeah. Yeah, yeah. I like it. I like it. We'll do one, one more. So, would you rather, I guess if we're, the pro, should I take from this is if you were starting Bugcrowd entirely again, right? Write the entire thing as Sam Altman said, character by character, like, like without, with, with code or vibe code the whole platform?
Casey Ellis: What would you rather?
Mackenzie Jackson: Um, I would, I would not, I mean, I hate the term vibe coding.
Casey Ellis: Right, sure. Let's use AI-assisted coding.
Mackenzie Jackson: I'd have to use AI-assisted coding.
Casey Ellis: Yeah, yeah.
Mackenzie Jackson: Yeah, for sure. Yeah, yeah. I think that's kind of like the, the general approach. Since I've been doing this podcast, that question has changed.
Casey Ellis: I can imagine.
Mackenzie Jackson: So, it was 100% at the start that people were saying, "I would never, I would, if I had to use AI, I would not trust it."
Casey Ellis: Yeah.
Mackenzie Jackson: It's flipped 100% to the other side.
Casey Ellis: Yeah, which is interesting that you could do. That doesn't surprise me at all.
Mackenzie Jackson: Yeah.
Casey Ellis: And especially like with a security context, right? We can't control things below that. Going slow, like the risk of not doing a thing at all is 100%. So, like what are you going to do?
Mackenzie Jackson: Well, yeah. The other part of that is seeing how the, the frontier labs are doing it. Like they're using so much of their own stuff to write their own platforms. Yeah. And to me, that's like they're off in the future somewhere, but like that's probably roughly where we end up.
Casey Ellis: Yeah.
Mackenzie Jackson: So, I, I'm kind of using that as a bit of a leading indicator.
Casey Ellis: Yeah, for sure. And we love to, we love to bash on like when like AWS fired a whole bunch of people for AI, and then they had like an outage, and we all love to connect these thoughts. But, I also kind of think about it as like we're at a breaking change right now, isn't it? Where things are going to break a little bit, but as long as it doesn't break, you know, it's, it's too, too bad. Like there's going to be turbulence.
Mackenzie Jackson: Not to double click and start a, you know, brand new podcast on that one too much, but like I did it, you can do it well. Yeah, this is the thing. This is why I like object a little like, are you talking about really live coding or actually using AI from an engineering standpoint? Because like AI assisted engineering with like thoughtful design, you know, good thought put into like maintainability, updates, all those different things. Like AI is just a force multiplier to people that are already really smart and good at doing this stuff well. Yeah. Like why wouldn't you use that?
Casey Ellis: But, I think it comes down to the fact that like what you're talking about is do I put a prompt in lovable or replit? Or do I or or am I using Cursor and kind of understanding what it's doing and getting it to make suggestions for me? You know, like there's a, there's a very big difference between the two of those.
Mackenzie Jackson: So, that was the case. Yeah, I'd still probably find a way to use AI to.
Casey Ellis: Well, fair enough. Well, guys, it's been awesome. Thanks so much for having you here, and I all the best going forward. Good to chat.
