How AI Became the Ultimate Cybersecurity Blind Spot: Understanding the Microsoft 365 Copilot Vulnerability

Aaron Crow (0:1.238): AI is going to hack everything. That's what's starting out, right? So welcome to another episode of Protect It All. I'm your host, Aaron Crowe, where we dive into all things cybersecurity, IT, OT, a little bit of everything in between. Most recently, we've seen a release of a CVE, a vulnerability, when your AI assistant kind of turns into a potential backdoor for attackers. So what happened?

Aaron Crow (0:31.178): So the recent CVE focuses on Microsoft. Big surprise, it's not an attack against Microsoft. It's just that they're probably the most adopted where it's integrated into your systems the most, just because everybody uses the Microsoft 365 platform. And people are a little concerned for good reason in allowing their systems to be integrated with third party.

Aaron Crow (1:0.108): chat GPT and Claude and Grok and all these others, right? But co-pilot living in your entity and your enterprise and your system makes you feel a little bit more comfortable. And for the right reasons, it is, it has less attack paths, right? But what is the CVE? Let's talk about it and kind of all the details around it. So this eco leak, a zero click vulnerability, it really exposed a critical flaw.

Aaron Crow (1:27.470): in the Microsoft Office 365 co-pilot implementation. you know, it's not just about a lot of the vulnerabilities that we see, especially coming across email in the past have been an attachment or a link or something like that. Like you have to click on something, right? This is different. An attacker could actually send a benign looking email with embedded malicious stuff in the background.

Aaron Crow (1:55.882): And then when you ask, if you have like a copilot routine or it's automatically summarizing your inbox, when it processes that, that infected email behind the scenes using its retrieval, augmented generation pipeline thing that, that, that Microsoft uses, it takes sensitive data from your email, from your files and chats and embeds in the images or links, and then quietly X fills them using Microsoft approved URLs like Teams.

Aaron Crow (2:25.396): and SharePoint. No clicks, no alerts, no chance to stop it. So, you know, this is really an AI blind spot. I don't know why. Blind spot may be a little harsh. Blind spot implies that nobody could see this coming. I think this is something that people have warned about. But to the point, it is a blind spot because there are supposedly protections against these things. Microsoft safeguards were bypassed. Co-pilot.

Aaron Crow (2:54.220): jailbreak filters, you know, fa- excuse me, jail- they're jailbreak filters. They failed to detect, you know, the distinguished prompts. The image links and markdown formatted evaded the traditional link, you know, redaction of, you know, a short link or bit.ly or something like that that it's going to look and find, right? And then the internal external data were blended together. So breaking the fundamental principle of, you know, trusting the boundaries. So

Aaron Crow (3:21.326): You know, it's just a technical mess. It's a design failure and how we secure agents. And this is going to be more and more prevalent as we start using AI. This is not a we shouldn't use AI, right? This is going back to, and I talk about it a lot here, you know, coming out of Idaho National Labs, you know, cyber informed engineering, we need to be designing all of our systems using cyber as, as, as part of

Aaron Crow (3:49.474): the requirements in designing these systems. And I'm not saying that Microsoft didn't and I'm not saying that, you know, if you've implemented these things that you didn't, but it's not always top of mind when we're implementing these capabilities. So it's very easy to look at the desire for the outcome. I want to be able to enable my people to be faster, to not have to read through hundreds or thousands of emails. I mean, my inbox is just full of emails I'll rarely ever get to.

Aaron Crow (4:17.294): And Google does a good job of summarizing that stuff even without AI. Same thing with Microsoft, but when you add that AI thing, you can really find the needles in the haystack of these are the things that you should look at. Again, not saying that we shouldn't use those. This is just a good example of first attack that really is intentional, focusing on how do I get around

Aaron Crow (4:45.454): traditional phishing software, traditional things that would be detected by, you know, an avarice and endpoint protection and, you know, DLP device, you know, data loss prevention, all of those types of tools that are there looking for very specific attack vectors. And when it didn't trigger any of those, so it was able to slither around all of those and use AI, this tool, this benefit against itself and against all of your other protections that you have in that space. So

Aaron Crow (5:15.156): it's really what can be exposed, what is the fallout of this? Some people will swing the pendulum all the way. This is why we can't do AI. We're not doing anything and connecting to critical systems or any of our internal systems. And that may not be the wrong answer. Depending on your industry that you're in, you may not be able to do this. Like in OT, many times we are not choosing to do AI automated systems like this because of things like this, like the unknown of the unknowns. I don't know what I don't know.

Aaron Crow (5:44.376): So I'm not comfortable enough. It's so immature. I'm not going to connect it to critical stuff because I just don't feel comfortable that I know what the outcome is and what the risks are. So until I can really understand what those risks are, I'm not going to deploy that in a super critical system, right? What was exposed in this type of thing? Copilot has access in theory to everything. Now, obviously your configuration could be different, but...

Aaron Crow (6:10.754): You know, it has access to your email. has messages, access to your team's messages. It has access to SharePoint and your OneDrive, you know, internal context for each individual user, who they are, what their role is, what their title is, you know, who their boss is, who their subordinates are, you know, where they're physically located. Like it's a lot of enterprise data that it should, we don't want to get out. And there's, other implications that could come.

Aaron Crow (6:37.848): from this data being released and, and, and, you know, having access to this, just imagine everything in SharePoint, everything in OneDrive, everything in your email, having access, you know, if it's the intern, maybe it's not as big of a deal, but if it's the CEO, the founder, the CTO, know, VP of sales, like there's lots of people that have, you know, a critical information that they don't want or should not have released, not even counting if you're in an environment that has any regulatory, if there's any PCI information in there.

Aaron Crow (7:6.506): If you're in a regulatory entity like a NERC SIP or something like that, somehow that data got out from a system that it shouldn't have, that can be a big impact beyond just the fact that bad actors and attackers can then use that data to pivot because they know more information and maybe they have an attack path now that they didn't know about before. So what did Microsoft do? What did they not do? To their credit, Microsoft moved really fast.

Aaron Crow (7:33.822): They hardened the prompt injection detection. They tightened the isolation of the user context. And they implemented stricter content security policies. They claim no known in the wild exploitation, which is good. But the fact that it ever happened is a red flag that every AI integrated enterprise tool we need to be considering. Like what is the...

Aaron Crow (8:1.460): absolute worst thing that can happen. Like we need to be having those conversations and I'm sure people have, but I know we just switched over to Microsoft 365 and co-pilot in this kind of new branding that we've gone through and we're asking a lot of those questions and from your CRM to your third party tools, whether it's HubSpot or Salesforce or how do all those things together? Because honestly,

Aaron Crow (8:28.770): That's the goal for us. All businesses is to work more efficiently. So if I can have a product that can integrate a lot of those things, I can automate things. can, you know, sprinkling in some AI to make me make decisions faster, bubble up, summarize all of those are great things to do. And again, I'm not suggesting that we don't do them. I'm just suggesting this is an example of why we have to do it correctly and why we have to be careful what we enable. We can't just, you know, admin default all that. That can be a problem.

Aaron Crow (8:59.655): So what does this mean for the rest of us? EchoLeak is a wake-up call. Here's what leaders and CISOs, what you guys should be focused on to do now. So audit your AI access. What can CoPilot see? What access does it have to run against

Aaron Crow (9:21.804): your internal systems. If you do have, you know, obviously PCI or other type things that it doesn't need access to, or that could be causing a problem. Obviously you need to limit that, right? It's, it's, it's just, just right permissions, just like everything like we used to do, you know, when we had file servers and we had folders and we delegated authority down to, you know, a specific team and we had, you know, finance different than engineering, different than sales. And, you know, it's, it's, it's simple.

Aaron Crow (9:50.080): implementations of security posture that we've been doing for decades. We'd seem to make sure when we're deploying AI or any of these automated type tools that we're considering those simple policies and procedures that we've been doing for years and make sure that we have that. And then audit it, right? Make sure that Bob doesn't have admin rights and can't adjust that and then expand that beyond what it should be. know, monitor for, you know, prompt injection vectors. There's all sorts of tools.

Aaron Crow (10:19.650): that can monitor your AI side, but the SOC team needs to really understand what happened and what to look for. Now that we know this, there's a patch out for it. Make sure that we are training our SOC and our tools to look for those types of things. Already mentioned it, but to double down on it, make sure that you have separate trusted and untrusted data in AI pipelines. There should be no blending of internal, external context inputs

Aaron Crow (10:49.304): Like you really need to segment those spaces and make sure you don't have data overlap. You're not getting things that, know, Aaron has access to because I'm authenticated. Now I have access to public information and private information. And then my AI, because it's basically, I gave it permissions. Now it can bridge those two gaps. Now it has access to everything, right? So in the NERC SIP world, the way that you look at, know, segmentation and the way that it's implemented,

Aaron Crow (11:17.898): It's a single shared cyber asset that can impact more than 1500 megawatts, right? So that's power utility talk, but really what it's saying is, there a simple, like if I configure this device and even though this device in and of itself, and maybe it's just an HMI, but if this thing can have connections to both sides, to two sides that can impact something bigger, I want to be really careful that I have to lock it down or I need to segment it. I should have two different devices, right? And that gets a little bit more complex and a little bit more difficult, but

Aaron Crow (11:46.850): this is the problem that we have when we bridge those environments, when we're bridging public and private, when we're bridging, we're plugging AI, or again, it's not just AI, it's not the big bad AI thing, it's really just all of these tools as we're bringing them on, we need to be careful with what access and what data access it has. This is just a way that it bypassed a lot of the protections that we were saying, yeah, you can have access, but I'm going to do DLP and I'm gonna do all those things, but it found a way through those things. So just having access,

Aaron Crow (12:15.330): didn't matter that it those other tools that potentially blocked them, but this is a way that it found a way around, right? So AI isn't just an assistant. It's really a new attack surface. Just like with anything, when we bring new things and new capabilities, new stuff into the enterprise, you are also bringing in vulnerabilities and potential risks.

Aaron Crow (12:43.030): Every time we implement new technology, it is also going to bring in new risk. And in the beginning, we don't always know what those things are. We don't truly understand what that risk environment looks like and what risks we're really bringing in until we start seeing some things like this pick off. And then we know how to lock down. What is best practices? There aren't great best practices in the space because we are truly bleeding edge in a lot of these spaces and we're all trying to figure it out. So again, just

Aaron Crow (13:12.584): Look at what you have, make sure you're considering what to do in the in the spaces and what's the right thing. Make sure you're having the conversation. You're you have somebody that is challenging every configuration, every decision and making sure you're looking at it from that. What is the worst thing that can happen and how do we protect against that? Right. You know, it really you know, this this echo leak really proves it. Your own A.I. may actually be the way in.

Aaron Crow (13:41.342): You know, don't want to talk about Terminator, Judgment Day, but you know, that's what it is, right? AI is able to think faster and think differently than a human and find different ways. And now obviously human is the one that sent this thing, but AI is a way to work around problems that we put in because we don't necessarily think, well, a person didn't do that and couldn't do that, right? So let's not wait for the next one.

Aaron Crow (14:5.218): Let's make sure that we are again, auditing our environments, we're reviewing our copilot deployments or any whatever your AI is. If you have your own, this isn't just a copilot problem folks. This is going to be a problem for any AI. So if you have your own in-house LLMs on Olamma running on your own servers, this can still be a problem. This is still an attack path that can come in. So you need to make sure that you're looking at all of these things, even if you're using local, even if you're using one-off custom in-house, all those types of things, right?

Aaron Crow (14:34.656): audit those environments, make sure that you're looking and understanding what those environments have access to, who has access, what environments, what data they can see, make sure that you have and you're looking at this type of attack and this type of, you know, exfil of data, which is really what it was. It was getting data out of these environments in a play in a way that we weren't the environment wasn't you would not have been able to detect with traditional detection capabilities, right? And then really educate your teams beyond just how to configure it, but also

Aaron Crow (15:3.734): what the risks are, making sure that they're not connecting to things that don't need to be. If you look in the government world, that's why you have two different computers sometimes where this one is on a secure network and this one's on an insecure network or a public network. And you don't want physically those things there. So there's no way that I can get data from this machine to that machine. So I have two different. So if you have those environments and you want AI in both of them, you may have to set up different AI systems to

Aaron Crow (15:33.346): to access and give you the capabilities you're looking for because one in the middle can bridge both of those things and that can become a problem. The attackers aren't asking for permissions. They don't follow the handbook. They didn't do the cybersecurity training. They don't care if you send them a bad email or give them a horrible performance review. They're trying to get in any way they can. So they don't need your permission. They're going to try to get in any and every way that you can, every they can.

Aaron Crow (16:3.070): Anyways, just wanted to really dive into this one. This one's a new one that came out, figured it would be a good one to jump on and talk through. Love to hear anybody that's seen anything like this, thought about it, have you solved these types of things. Please make sure that you like and subscribe to this, leave some comments. Definitely listen and love all those, love hearing from everyone that listens. And always looking for guests to come on and talk about interesting and cool things like this. So if you've attacked this or...

Aaron Crow (16:30.284): Better yet, if you've approached this and found a way that you're implementing it in a better, more secure way, love to have you on the podcast to talk about it. AI is definitely on everybody's mind in all spaces from IT to OT, enterprise and everything in between. So it'd be really good to dive into this topic. AI seems to be a very common conversation that I'm having here lately. And for good reason, it is really the next thing that's coming and it's coming as fast, faster than I think we're even ready for.

Aaron Crow (16:57.718): Anyways, thanks. Thanks for time, everybody. Thanks for listening. Until next week, you know, like I said, just reach out, get on the podcast, me suggestions for folks, topics, etc. So thanks a lot.

Transcript lightly edited for readability.