Mackenzie Jackson: Hi everyone, welcome back to The Secure Disclosure. I'm so thrilled to be here with Noora. Thank you so much for joining me today.
Noora Ahmed-Moshe: Thank you for having me. It's lovely to be here.
Mackenzie Jackson: We were actually just discussing offline before we started this, about what you bring to the table. You've been in tech for 20 years.
Noora Ahmed-Moshe: Yeah.
Mackenzie Jackson: And you didn't call yourself a technical person, so how did that happen? How did you first get into tech?
Noora Ahmed-Moshe: That's a great question. Actually, my passion has always been in how we as humans shape technology, but then technology shapes us in return. So, that interaction between humans and technology. I studied media and cultural studies, so a lot of human behavior and communication. Then I did my master's in digital media, where I did have to learn to code. I'm the world's worst coder, but I think it's good to have that level of understanding of how coding works in order to work in tech as a non-technical person. So, my passion's always been around how we utilize technology. And being in cyber on the human side of things, my interests lie a lot in behavioral science: how we can actually encourage people and get people to behave securely to reduce cyber risk. And it's a topic that never stops giving.
Mackenzie Jackson: Humans are so unpredictable, for sure. And when you look at these big technical breaches that happen, they may use very sophisticated malware, or very advanced payloads. At some point, someone probably got phished, or at some point someone leaked a credential. There's a human element to even the most advanced threats that we face.
Noora Ahmed-Moshe: It's still the most common way they get in.
Mackenzie Jackson: Yeah.
Noora Ahmed-Moshe: It is. Depending on what stats you look at, 60% or thereabouts of attacks start through the human layer, whether it's email phishing, whether it's voice phishing. And there are just constant small things that are being optimized on the attacker side. So, we hear a lot about deepfakes and cinematic, kind of crazy futuristic scenarios, especially if you follow mainstream media, and where AI is taking social engineering. But the truth of the matter is that it's happening all the time, everywhere, in the edges of where people work, and just reaching people in those little sneaky ways that then can normally open up the path for the attacker to get in and gain that lateral movement that will enable them to in the worst cases cause a lot of damage.
Mackenzie Jackson: For sure. You've mentioned AI here, and that's something I really want to talk to you about because you're here at RSA, and you're talking about, you've been delivering talks about shadow AI and shadow IT. It's interesting. The AI element is crazy because it's happened so quickly. And at first, everyone loved to hate it. Now everyone is kind of figuring out, oh, we have to live with it. And it's still very challenging because it's just kind of increased our attack surface. So, one of the topics that you're talking about is shadow AI or shadow IT in general as well. Can we start with what that is so that we can kind of know what you're referring to when you talk about shadow IT, shadow AI?
Noora Ahmed-Moshe: I suppose in its simplicity, it's employees in organizations using IT tools that are not sanctioned or approved or signed up to by the company. So, any unofficial tools. And I think it's been, for as long as we've had IT, it's been somewhat of a concern because employees have always been able to find ways around company policies. But it's much harder in the past, especially if you have to install software on your computer and your IT can block it, for example. But there's always been ways that people can find ways around it. But I think now with the advent of AI's mass adoption, it's a completely different ballgame. So, it's kind of like the same issue of shadow IT in essence: employees in organizations using tools to do work that are not given to them by the company. But the scope of what these tools can do with the data that you put in, and the plethora of tools available, and the amount of tools that you don't have to deploy or download in the same way because a lot of them are browser-based. A lot of tools you can access actually on your personal devices, but you can still put company data in there. So, employees can use these same tools for their personal uses as well as work usage. So that's kind of, in essence, what the issue is without getting too deep into what people are doing there.
Mackenzie Jackson: For sure. And I relate to this a lot because when AI first started coming out, it kind of felt like cheating.
Noora Ahmed-Moshe: Mhm.
Mackenzie Jackson: There was a lot of not wanting to admit that you're trying AI, especially at the start. Now I think the conversation has changed completely. But the AI tools bring such risks when they're not implemented properly. Because they have so much access to things, or they can, with agents and other areas. What are some of the risks that you're seeing in having these shadow AI areas? If you did have an organization and let's say they didn't have a clear direction of what tools you can and cannot use, what's the risk of everyone having lots of AI that you have no idea about, or tools, or just other IT as well?
Noora Ahmed-Moshe: The one big risk is obviously that employees are putting sensitive company data into tools. They don't know what's being done with it, or where it might go. Whether the tools themselves are safe, even if the tool might be quite secure, or a reputable company tool, it could still be on an unpaid version, for example, using that sensitive company information to train models. But in the worst case, it could be a tool that could be breached more easily. We just don't know. So, we'll have obviously the mainstream providers that have enterprise plans, and a lot of companies have their enterprise accounts in Copilot and ChatGPT and Claude and whatever they may choose. But a lot of people, like what you said about it felt like cheating back in the day. And now when we talk about back in the day, we're talking about a shorter and shorter time span, aren't we, because things move so fast.
Mackenzie Jackson: That's true.
Noora Ahmed-Moshe: But some of the risks obviously come from, I think in some studies they've found that more than half of employees are using unapproved tools, AI tools at work. And more than half of those are putting sensitive company information into those tools that are not company managed. So, the thing about the risks is that we actually, I don't think we've really seen the risks materialize yet because it's so new. And as you mentioned, agents. So, when there are more agents also making decisions and taking actions and with input company data and company stuff in there. And we start to lose track of maybe who's made a decision and where and why, and then what the impact was. I think it sort of still remains to be seen what's going to happen. But actually, in my session yesterday I asked this question and I asked people to share if they'd had breaches happen in their organizations. And one participant talked about something that happened to their supplier. So, it wasn't from inside this person's organization, but their supplier had had a phone call or, Teams or, online meeting about a legal litigation matter with their lawyers. And apparently this AI tool had written a summary of the whole conversation and sent it to the opponent. And the opponent had gotten in touch with this company and said, you know, they'd found a way to settle. And as a result, the company lost a million dollars. And obviously because this was a person telling me the story whose supplier it was, they were now very suspicious of the security of this supplier. So, it kind of illustrated that feeling that you have total loss of control now in terms of who's using your data. And we've always had a concern around third-party risk when we think about social engineering and those types of breaches. And it's kind of all of this escalates as the tools get more intelligent because they have the ability to do more stuff. Like, I work in this area and I think about these things day in, day out. And it hadn't occurred to me that this specific scenario could happen where the note-taking tool sent this legal information to the opponent, but obviously it tracks that this type of thing is going to happen. So, what other things can happen?
Mackenzie Jackson: Yeah, exactly, with that data. For sure. And especially when the data is being restored somewhere, it can be retrieved. You have prompt injection. You have this whole new class of vulnerability that we don't understand now.
Noora Ahmed-Moshe: Yes.
Mackenzie Jackson: I want to talk about kind of the response to some of this. But there was, when AI first came out, there was a knee-jerk reaction from a lot of companies. And maybe it's understandable, where they just decided to outright ban a whole bunch of tools. Samsung famously banned OpenAI. Is banning a good approach for this? How do you kind of start to solve this? If you were a company who wanted to understand your tools, what do you kind of do? Is banning right? Is direction right? What's right?
Noora Ahmed-Moshe: So, I think banning is only going to result in workarounds. I think that's the first thing we need to understand. Like, of course, in enterprises and all companies and organizations that take their cybersecurity seriously, you do have to ban access to anything that you think is directly harmful for the company. So, you do have to get the lay of the land and you do need to use and stay on top of any technical controls where you can detect as much as possible of this usage. But much like with any kind of threats that can come in through the human layer, the same way like in emails, for example, we should always use the best possible email filtering solutions. But they're still because obviously the attackers are always optimizing their methods, there's always going to be ways that they get in. And then, we're on to the human layer and on to the human agency and the employee and the human behavior that then will make or break or make the difference. And I think like before we think about the solution, it's really important to recognize why people are doing these things. Why they might be using unsanctioned AI tools. So, I believe in most people being very good and wanting to do the right thing. And through my experience of talking to so many security leaders, so many people working in different roles and organizations about cybersecurity, because I really want to try and get to the bottom of understanding what drives people's decisions and why they might or might not behave securely. We, of course, have a small percentage of malicious insiders and people doing bad things on purpose. But I think that's a whole other problem. And I don't think it's related to why most people use shadow AI tools at work. Firstly, I think there's currently, if we step outside of our cybersecurity RSA bubble, there's a general sense of panic around a big mass of employees that AI's going to come and take your job and you're going to lose your job and you're going to become irrelevant because that's a lot of the media discussion, isn't it?
Mackenzie Jackson: Yeah, and to be fair, even though I'm in this, I will publicly say, "Oh, no, I don't believe that," it still freaks me out. I still don't know if you're going to be sitting here with my agent, as in my
Noora Ahmed-Moshe: Depending on your role, if you're doing coding or if you're doing whatever it is you're doing, somebody will tell you that there's this other thing I found on the internet, or that my friend told me about, or my friend developed this thing and gave me access, and it's going to make you more productive. And so, I think that's the real core reason is that we do, we do need to utilize new technologies. And I think as humans, if you weren't using email and you just preferred to write letters on a typewriter and send them to people, you would have been left behind modern ways of work. And that's just the nature of technological progress. We've got to constantly somehow stay on top of it. And I think that if you try to ban AI tools in your organization, I don't see how really you can succeed as a company anymore. Because you're going to very, very quickly, your productivity and the speed of innovation and the speed of change and the speed of execution is being enabled by these tools. And so, you have to find a way to leverage it cleverly and safely and securely. And you know, if you ban everything apart from one thing and that one thing works for everyone, you might be in a position where, "Okay, you're lucky and now all your employees, nobody's tempted to use other tools." The real issue is whether the people making the decisions on what tools can be used actually really understand how the humans are working and what things they're trying to get done. So, if you've got leadership, for example, that doesn't want employees to use AI tools freely, maybe they don't understand this reality of business these days, that you will be left behind if you don't innovate. That's just a fact. I don't think you can argue with that. If you, let's say, you have this kind of old-school method in your IT department that you're going to look at a tool and you're going to take a POC and you're going to test it, and then three months later, hopefully sooner, you're going to tell everyone now you have access. Did you look into whether that tool can help the pain points that people have? Because if they don't get a solution to those problems, they will 100% go and find something else. And if that's even that URL is blocked, they can still try and find a way to use it on their personal phone or whatever. So, you have to be able to direct people to safe solutions that still enable them to get their work done. And I think this is the kind of big friction often around the wider themes of good organizational culture generally translates to good security culture. And the baseline of when you have good security culture, it's normally in organizations where the security team and the security function has higher leadership trust and approval and a mandate to kind of work. They, the security cybersecurity IT is a trusted function in the eyes of the board or the senior leadership. So, the teams are enabled to have enough time and not be too stressed out to actually listen to the employees, understand what they're trying to execute, what are the problems that they're trying to solve, what are the tools that they like. When you really get to understanding what the bottlenecks are in somebody's workflow, then you can offer them solutions. And you can sometimes realize that this tool that we wanted everyone to have Copilot or whatever it is in your organization, and you're like, "Oh, no, this person who's thinks that's very, very old school and they've already found another tool." And what are they trying to achieve? So, but it takes time and it takes a lot of effort and security teams are generally unfortunately overworked and burned out in a lot of cases. And now with this shadow AI, the issue we're here to discuss, it's because the breadth of the problem is so wide and the risk can hide in so many places that it can create more stress and it can create more of a feeling of, "If we ban, we stop them accessing, don't do that stuff." You know.
Mackenzie Jackson: I think you're touching on something so important there. And I think part of the reason why shadow AI comes up, at least that's what I'm understanding from you talking, is because they're not implementing tools that actually affect the people. And going back to what I said about cheating, we can kind of move away from cheating, but you mentioned a tool that people, the productivity gains, people feel like they're going to constantly miss out. If I don't learn this new thing, I'm going to lose my job. I need to stay relevant. And then, there's an approved tool, but it doesn't quite fit you. And you're now freaking out. And then you look at someone else's desk and they're using the tool and you're going, "Oh, no, I'm going to freak out." And then you get this whole thing where people are sneakily trying to use tools because the people implementing the tools aren't understanding that. What do people need to look for when they're trying to discover pain points? Like, if you're an IT manager, you're rolling out tools, how do you find a fit for that?
Noora Ahmed-Moshe: I think it's a great question. I think you need to, when you watch people working or listen to them working. And of course, if we're talking about a large organization with lots of different departments, people working remotely, a global organization, you might not be physically in the same office. But I think still, you need to be able to understand enough about the ways of working in the business, in IT. In order to be able to provide the right tools, you do need to, I think, look for champions or team leads or any people within the organization that you can work with to understand what is happening. And it is a two-way process. So, I like to talk a lot about psychological safety, which is a massive factor in cybersecurity culture as well. Because we do know that when security teams are approachable and people feel like security communicates in a dialogue with them, tries to enable them to do their work and supports them. And if they mess up, they support them to get through it and don't punish them for mistakes. It's all of that same, it's an open dialogue. And again, as an IT manager, I think if you, sometimes I speak to CISOs, for example, about, "Oh, what's the culture like in your organization?" And I just spoke to someone yesterday who said, "Twice I went to a company that had really poor cybersecurity culture and then it took me two years to turn it around and then went to another place." And I think if you're in that sort of position as a CISO or a leader that is both board and higher management facing, as well as facing the IT teams that do the actual work, it's so important to get the leadership on board to give enough resources for security teams to have enough time to communicate with people because if they're firefighting all the time, then they don't have that bandwidth. The hardest thing about the human side of security is the unpredictability of humans. Because we can know something, "Don't click on that, don't input information." The same way we all know you shouldn't smoke and you shouldn't have five cheesecakes a day, everybody knows those things. We still do, we still act against the advice that we know on numerous occasions, many times a day we can cut on a red light or whatever it is. We need to know the risks and we need to be told. Somebody needs to tell you the risks in order for you to even have that information because if you don't know, you can't act accordingly.
Mackenzie Jackson: This has been a fascinating conversation and it's a shame that we have to wrap it up. But what I loved so much then is that it's been quite a philosophical conversation, but what you just did then is you outlined about five or six bullet points of what to do, which are very actionable items of what we actually need to do. So, I don't think that's a kind of a great way to wrap things up. It's a human problem. It has elements, but there are concrete steps that you can do to implement culture, trust, and we're not going to get full visibility, but we don't have to accept that.
Noora Ahmed-Moshe: Exactly. And it's a continuous process. It's a never-ending task.
Mackenzie Jackson: Yeah. All right, thanks so much for coming on the show. It's been fascinating and I really hope to have you back, and maybe our AI agents can have a chat next time. But thanks so much. It's been brilliant.
Noora Ahmed-Moshe: It's been a pleasure.
