Raghu: Hello, everyone. Welcome back to another episode of The Segment. I'm your host, Raghu Nandakumara, and what makes me especially excited for today's episode is that today's guest is a person that I accosted at a speakeasy at Black Hat just before he was about to go on stage and host the first-ever CISO comedy roast.

Ashish: Woo-hoo.

Raghu: But hey, absolutely, woo. But beyond the laughs, we got talking about something that's been on both of our minds. The current rush around AI feels like something we've seen before. We watched cloud adoption move faster than security could keep up, and now organizations are racing to adopt AI, and it raises a question that neither of us could shake.

Raghu: Have we actually learned anything from the last technological shift, or are we about to make exactly the same mistakes, just with better marketing this time? So there are very few people better placed to answer that because he hasn't just lived through both these transitions, he's interviewed several hundred of the people who did.

Raghu: Ashish Rajan is the CISO of TechRiot. He's the host of the incredible Cloud Security Podcast and the AI Security Podcast. And if that wasn't enough, he's the author of his new book, AI Security Engineering. Ashish, welcome to The Segment.

Ashish: Wow. Thank you so much for that, Raghu, and I, I appreciate the intro, man. It almost made me cry.

Raghu: Don't cry just yet. Don't...

Ashish: I thought you would have gone down the British path, what we were talking offline, that, you know, the country of the disappointment. It's like he's done so much in his life, but yet we are still disappointed in him. Why not some more? Someone else has done so much in their life, but he's only doing this?

Raghu: Well, hey, like me, you have Indian parents, so you know exactly what that's like, right?

Ashish: The disappointment is like in, inbred into us. My dad's voice is always behind my head is going, "Not good enough." It's like, I don't know why.

Raghu: Exactly. Exactly. And I'm just passing that on to my children now, by the way.

Ashish: I mean, that's what you're supposed to do, just pass it to the next generation. So then they go for therapy, and then we're like back and like, "Oh, it's all my-- it was my parents to begin with."

Raghu: Exactly. So here's the most important question of today, right? Dude, how the hell do you get your podcast to 8 million downloads? That's what I want to know.

Ashish: Well, to be fair, it's seven years of running it as well. So it's, it's, it's-- There, there is time, uh, under tension, for lack of a better one, to use a fitness anal-analogy. Um, the, uh, the idea, well, at least for both the podcasts so far, has been it's practitioner-driven first, and that's kind of what I feel has resonated with a lot of people.

Ashish: And I, I, I do want to caveat, right? I think didn't come from a media field. I was always a practitioner, spent 17 years being in cybersecurity, did-- started my career in IAM, switched over to security architecture, cloud security, built a SOC team, became a CISO, did all of that. And then, um, when I started the Cloud Security Podcast first, I was literally looking for someone to talk to about cloud security as an ecosystem, as a topic, as a, as a practitioner to another practitioner, it didn't have a vendor conversation.

Ashish: It was literally talking about, "Hey, what are you seeing? How have you tried solving something?" And it resonated with a lot of people. And a few years later, it's about three years earlier, I'm sorry, three years before, years ago, we started the AI Security Podcast, which is another, um, similar theme, practitioner-first conversation.

Ashish: What we were noticing was that everyone who was listening to our Cloud Security Podcast, everyone was asking us about AI security, going, "Hey," clearly it's not the same conversation as cloud security. It's bigger than that, and it has to be its own thing. That is why AI Security Podcast. But I think definitely find that the lens of being a practitioner first definitely helped. That's why we've had guests like the CISO of Visa, CISO of Standard Chartered Bank. We had the CISO of DeepMind, CISO of Anthropic as, uh, guests on the podcast who were kind enough to just, uh, spend their time with us, talk about what they're working on, what's top of mind for them. I think things like that I definitely find resonated a lot more because generally the media ecosystem-- Obviously now you have the podcast, so you have a practitioner lens. There are other podcasts as well, similar, who are trying to bring that practitioner lens. But I guess I want to claim that we were one of the first few, and maybe that's where we had the advantage of time under tension, that we were there for a lot longer.

Ashish: So we're grateful, uh, that a lot of, lot of these people still recommend us. And when we go into conversations, people already know about who we are,

Ashish: Yeah really helpful. So I definitely find that, uh, if I were to just do a TLDR, it's the practitioner lens that has helped us grow that quickly, just because we started with that lens and we've continued to be in that lens even today.

Raghu: I love that story, and I think just that, that, that practitioner-led focus allows you to be particularly sort of curious and inquisitive. Um, and it's always with that sort of how would I actually use this in practice, uh, or leverage this in practice mentality. So let, let me ask you a question again related to conversations that you've had on the show, right?

Raghu: And you've had so many, um, vendors on the show, right, across a whole, wide variety of product categories. If you don't mind, Has there been any particular category that you've kind of been listening to and even on the show thought, "Yeah, that category is not going to exist in like a year"?

Ashish: uh, I would say-- So it's funny, I think I've seen generations of different things and themes, I, for lack of a better word. First thing that comes to mind from an AI security perspective, I think shadow AI is something that will just disappear. Um, the k- and I'll, I'll give you some data points for this as well. How often do we talk about shadow IT today, right? It's almost like it happens in the background, you identify it, but it is not the thing that people, "Oh, I want to solve shadow IT." Yeah, unless you're starting a program from scratch, you're not really thinking about it. So maybe some of the newer AI native companies, they may be coming across that problem today, but for people who are established, like I end up doing strategic advisory for companies like Fortune 500, Global 2000, who's been running programs for a long time. I find that shadow AI is definitely something that would just become background noise after a while. Uh, it will either standardize because we'll have processes for it, and it would just become something that is not top of mind. That's the, thus at least in the AI space, I think that would definitely become a thing. The other thing that you would also find, uh, is the conversation around the... I, I won't say security vulnerability for AI, but more like a... know how at the moment we don't focus a lot on prompt injection, we don't focus a lot on AI incidents. I want to bring that, that I would like to see as more of, but today we don't talk about.

Ashish: So I think the, the yin to my yang is, or maybe I've gone into a different direction, but essentially the shadow AI part, part would die away. What would start getting bigger is conversation for what does an incident look like in AI. We don't know. Like, is it the same thing as me, uh, if I refer to the chatbot ins- sorry, if I refer to the OpenAI Hugging Face incident that happened. I am an agent asking on a quote-unquote message board, "Hey, does Raghu work for us as an organization?" "Can I get Raghu's password?" Two different things.

Raghu: Yeah, yeah.

Ashish: Getting the intent of that is super hard, so I think that would kind of amplify quite a bit and that would become a big one. That's from the AI side, and I can definitely double down a lot more on the AI side. But if I now add that same layer of what would die down and what would become more conversation in the world of cloud security, I would say for cloud security, the CSPM CNAPP thing is going to  become background noise. and it's already starting to happen where people would start expecting that to be part of your AI suite. not going to  be, "Hey, I'm not going to  go specifically for a CNAPP anymore." Because think about the transition that happened between, uh, the, the data center to cloud. What ended up happening was, how often do we talk about a, I need a virtual machine manager firewall thing, And I think was replaced by CNAPP and CSPM, and that, that's what's happening, happening now, where most of the workload that's going onto the cloud is an AI workload So the expectation now is going to be that CNAPP and CSPM is going to become...

Ashish: You'll probably start seeing it, the, that the category is now evolving to, yes, there will be security engineering, they would work with AI tools. They're not going to  just work with the CNAPP, because just working with the CNAPP is like being in 2000s. Not today. Because either you have used AI to improve your workflow, improve your engineering around it, um, but that's kind of where it, it's going to reduce quite a bit. Where it's going to increase is the fact that now I'm... Like a lot of the workshops that I've been running through AI Security Lab that we have is around, hey, how do I connect my AI system into a, into a cloud ecosystem? How do I triage a CVE that's being shared by Amazon, Google, Azure, whatever? How do I not spend my entire day trying to figure out which one of these is relevant for me, and which one of them we actively host in our AWS environment or Azure environment?

Ashish: Like I did a research using Grok Bot on this, I released it last week, but the whole idea was just that, where a lot of these AI workflows going to be top of mind for people. Maybe some of them would be replaced by products, but a lot, a large majority of it would just become like AI workflows and cloud security as a conversation would be very different, where it's just like, "Hey, did you run that workflow?"

Ashish: Instead of, "What happens in the background of that is that AWS call that we make to make this unique thing happen in a CLI?" That would disappear

Raghu: So I, actually, let, let, let's talk about that a bit more, because that, I think that's, that's a very interesting claim and, and I'm sure there are, um, essentially categories of vendors who will say, "No, no, no, cloud security is the most important thing," right? Because it's essentially the, it's the data center of now, so you need to secure it properly, et cetera, before you run any workloads on it. So going back to that, right, if cloud security is essentially going to become AI security, right, then who is securing and how is the plumbing being secured? Because that, that's the bit that I think that is, that I feel then becomes the gap. So how is the plumbing secured?

Ashish: Yeah, yeah. Fair. It's a fair question, and I think I would say I don't want to caveat by saying not everyone is doing this today, but the people who are AI forward, the way they operate, and I'm fortunate enough to work with a lot of them, uh, what the pattern has become is that you have cloud security engineers working through an, like an MCP or an API to whatever CNAPP provider.

Ashish: They're not really log... Like today, if I were... Actually, let me rephrase this. If I were to look at a workflow of how I work as a cloud security engineer today, is that I log into a CNAPP, see what the number of alerts are. Okay, I have 50,000 alerts. I'm going to pick the top 100 because they seem reasonable. I use my human judgment for it, and then I go through that and spend the entire week on it or a day on it, That's my, that's my normal day. And if I find something, I go, "Okay, who do I reach out to solve this problem?" Because I can't solve that problem myself. I need to find someone for it. So I, I go, go about the company. Oh, it's Raghu. need to find out, is Raghu in today? Oh, he's not in today. I wonder when he's back. Next week. So you're, now you're doing ticket management, for lack of a better word, for something that should have just been like a, "Hey, can I remediate this not?" But based on the context that I have. Now, fast-forward to how the AI forward companies are working today on this. still use a CNAPP, they still use CSPM, but the new infrastructure that is, is being built, is being built with the cloud cores of the world. Like the Terraform templates that you use are not being built by humans. They are being built by the coding a- because everyone's being forced to use more AI.

Ashish: And I... Maybe good, bad, however you see it, it is being used to produce all the code. Like, I don't need to remember the Terraform commands to create an AWS infrastructure anymore. Right where the mistakes used to be. And if, and, and no matter what a cloud security vendor would say, would remind and go, "Hey, do you remember the one, top three reasons why there's a cloud breach?" First one used to be the fact that there's some kind of a misconfiguration. Some- basically, someone got phished. I got access to Raghu's credentials, and I logged in and boom, I'm like

Ashish: Yeah, the second reason used to be that you either have a misconfigured resource like an S3 bucket or a...

Raghu: Classic.

Ashish: Right? The third reason is that there's genuinely a security vulnerability that is exposed to the internet. Outside of any of this, it's literally just a process flaw or pro- process basically has failed somewhere. But outside of that, top three reasons have not changed in the 20 years that we've been doing cloud security. So you almost realize, oh wait, so the, the first part, phishing part, clearly I can't control that because Bob in accounting is still going to click that link for whatever the thing is, right? That's a security awareness problem. Now, the third problem, which is the, uh, hey, I have a genuine vulnerability that's been exposed to the internet. That's like your exposure management people looking at the...

Raghu: Yeah.

Ashish: And a lot of people who are doing the, I think, uh, across Europe, the people have started doing MITRE tools readiness assessment. It's still the same exposure management, but let's just call it MITRE tools readiness assessment. But same thing. That's already been, uh, uh, uh, this already been looked at. The only thing remaining is the middle part, which is a misconfigured resource. Now, the misconfigured resource is actually being created by cloud code or something which is helping you code that. And let's just say the first few versions are not perfect, right? I'll give, give people that much, uh, rope and go, okay, first few versions because for whatever reason, I wanted to be really careful and whatever, I have not been satisfied with it. But over time, you would start having playbooks for how infrastructure is created, which is what always was the goal for cloud security teams. Hey, how do I build that paved road for organizations? If that is being created by code and it knows, because the thing with, uh, us individuals creating a Terraform template versus an AI-created template, I may feel like I want to put the password as the second line versus the 20th or 25th line today. I don't know. I just copy-paste it from Stack Overflow a very different way. An AI, if you give a template, it would always follow that template for creation Right? And obviously I'm not saying that the controls around it, like the... You still need a CNAPP, you still need a because I may still log in and just switch off, or maybe I may still log in and forcibly open an S3 bucket. Like, you can't prevent that human from doing that. But if you were to think of the basic majority foundation, you could get to a point where, majority of that Terraform template, AWS MI pr- provisioning, Azure provisioning, all of that is being created by AI creator tools, which means that the, the second bucket starts getting a lot slimmer than it was before. And which is why you would probably find that a lot of s- cloud security people have to change their conversation to becoming more AI friendly, uh, where it's either you are helping the companies reduce that tech debt that they have from all the years of cloud security vulnerabilities, misconfiguration that they haven't filled, that's one bucket. But the other bucket, which is like, "Hey, I don't want to misconfigure in the first place," that bucket is shrinking quite a bit. other bucket is still there, like I, I still have to wait for Raghu to come back from holiday and fix it.

Ashish: Yeah

Raghu: Yeah.

Ashish: That's not going away. But the other part which I needed a CNAPP for is now becoming more like, "Hey, I have an AI workflow. I want you to have an MCP to my CNAPP or some kind of AI connectivity. I don't want to log into you directly. I just want to go, 'Hey, agents, do this work,' and Bob's your uncle after that." The thinking there.

Raghu: Yeah no, that, that, that, that totally makes sense. So a follow-up question to that, right? Like the, um, one of sort of the, the famous statements when it came to cloud security was around shared responsibility model, right? We have that... We've had that s- that shoved down our throats for the best part of, like, 10, 12 years, right? No cloud security conversation starts without a, "Here's a shared responsibility model." Basically effectively saying, right, "You know what? You're still ultimately responsible for the security around your data," right? Like, "Whatever you manage, that's your problem, not ours."

Ashish: Yeah.

Raghu: but you have said, or you once said that shared responsibility means that no one is responsible.

Raghu: Yeah, how does that manifest in cloud? And I, because I know it, at that, at that point when you made that statement, it was specifically for cloud. And how is it further, um, exaggerated with AI, given that an AI stack consists of multiple components potentially managed, configured, deployed by different teams?

Ashish: Yeah. I mean, you can already ha- see it happen where, um... So the, the reason I made that statement some time ago, uh, and I still stand by it, is because it's a, uh, there's a problem called confused deputy that people know about. Essentially, the idea is that when I think Raghu's going to  do the job, but Raghu thinks that she's going to  do the job, none of us do the job. And I think that's the problem that, uh, for decades of talking about shared responsibility, people realize that there's a set of people who always think that, "Hey, AWS or Azure would do that would do that." And AWS and Azure keep harping to your point, shoving through our throat, "Hey, shared responsibility. Please take care of your own data. If you do not want this data to be exposed because of vulnerability that you have let go of, please don't come back to us. Legally, we are not responsible. We have shared responsibility to everyone." the, the funny thing is this has extended even further over the years because now they have the marketplace concept as well, where you have a third party that you can subscribe to using the Amazon platform. Which, uh, it, which pretty much means that I may have a license for Oracle, I may have a license for SQL Server, but I use that service through Amazon. Where is the shared responsibility there now? Where technically you're getting the service from Amazon, but Amazon is saying, "I'm not responsible because you're paying Oracle directly."

Raghu: Yeah.

Ashish: Who's responsible at that point in time? I, I'm like, because my data is going to go into Oracle, but Oracle is saying that, "Hey, it's cloud thing, so you guys should talk to the cloud people." But they're like, "No, no, no. You have my data. Cloud is saying shared responsibility." I think goes back just to, just what I said, it's a con- uh, it's a confused deputy problem as, uh, all over where y- once you start saying sh- even if you say shared responsibility, people just look at that, "Yeah, yeah, yeah, of course.

Ashish: I, I, I'll be responsible for my own stuff. But then when it comes to actually taking action, you don't think of that. You just go, "Well, I pay Amazon for this security." And then, or you look at your security and go, "Hey, security, tell me what, how do I do this?" And  you're like, "Well, first of all, this data should not go there." But you really want this data, like you really, really want this data to be there, so you would follow this path. But then the moment you talk about all the misconfiguration vulnerabilities, whatever, you, then you're told, "Oh, actually, I don't think it's a high, it's a low. Uh, you know, if we just... I was talking to Amazon, they said if you flick that button and send it through this firewall, it's a medium risk. It's not a high risk anymore. Can you, can we make it a medium?" And like, "Well, technically yes, but I don't think we should." But hey, you know what? As, as long as someone accepts the risk, don't care. Which is what the general... That is literally a normal day in most organizations for cloud.

Raghu: It's like saying, it's like saying, "I've got cyber insurance. Surely the risk is less now."

Ashish: That is that as well. Like I, I have cyber insurance, which is, uh, which is a whole nother conversation with AI as well. But, um, now extend the same conversation to AI. Initially, uh, I genuinely was talking to someone who said that, "Hey, my AI is safe from all this because I use my AI through Amazon Bedrock or Amazon or whatever," right? you're like- I don't know, it's like because you shared responsibility, whatever, you're like, oh, has taken the responsibility of making sure Bedrock is the, I don't know, whatever the right kind of gateway into whatever LLM that I want to use. the first thing that came to mind was blast radius, because technically going back to the cloud example that I gave earlier, that hey, by the way, everything that you have around that ecosystem, that is still your responsibility.

Ashish: So if your, uh, Bedrock is connected to an open public S3 bucket and all your data from Claude is saved there, there is nothing Amazon is doing there that can save you from that, right? Yeah, still the thing. Now, I gave the marketplace example. I'll give you another example for AI. Uh, um, Cl- sorry, Amazon now has direct Claude platform access, which means you don't need to use Bedrock to get... So earlier, when Amazon had not figured it out, you could only use it through Bedrock and Agent Code or whatever. you could technically just have a Claude platform directly, so you don't even have to have that Bedrock thing. So we have literally just said add another layer. You're paying directly to Claude. You're not even paying to Amazon. You're just using Amazon to, "Hey, please, Amazon, uh, because my workload is in Amazon, I want it to talk to Claude platform directly, uh, for my cloud resources." You've gotten rid of that layer yourself. let me, uh, ask and maybe, uh, put this for your audience as well. There is no shared responsibility for any of the LLM providers today. Like-- also, when you hear a breach in Anthropic or OpenAI, you would expect the whole world to shut down because everyone's using AI. And people think th- this has not happened, these OpenAI and Anthropic have reported breaches, we've continued to pay them. We've gone, "Please give me more. Just keep giving me more tokens to me." And you think that-- So you- the lines are even more blurrier, earlier I could say that, hey, if the Oracle had the vulnerability, I may be in this jeopardy thesis situation, but at least someone would take responsibility. But now with OpenAI, Anthropic, and any of the providers, which are paid providers, if they do have a breach, still ignore it and go, "That's okay. Let's go to the next one. Give me more tokens for next year." So I definitely find that shared responsibility, model has got- gotten even milky- well, murkier, uh, that it does not even make sense anyways today. it's required, yes, it is, but you don't have a governance layer for it because everyone's basically gone, "Hey, my, my-- That's not my job. I've put a document on the internet. What is your responsibility? My legal team agrees with it. Your legal team agreed with it. We signed a contract." it's, uh, after this point, legally, you cannot say anything. How often do you find companies actually putting a governance layer to make sure that shared responsibility is maintained? We don't with AI even more because we don't even know what we're building right now

Raghu: So given that, right, given that shared responsibility model is fundamentally flawed, right? Because-- Just because it, it kind of drives behavior that is suboptimal, right? Um, and what you just said about sort of like governance around, um, security, like security approaches for AI, security models for AI. So if you were doing this, um, essentially completely sort of greenfield, blank slate, right? How would you build up a security model that would work and be repeatable in the age of AI?

Ashish: Uh, and so I'm not greenfields. I'm basically-- I've had cloud, I've had on-premise. I'll use that example, uh, because most people today are on that category. Like if you were to think about, uh, if you were to draw a line on the left-hand side, you had everyone who was as a-- who existed as a company before AI, um, they're to me brownfield. Uh, and anyone on the right, which is AI native, they have their own problems. And so if I'd focus on the brownfield people, uh, or companies, I guess, uh, a lot of us already have the basic controls for identity, uh, networking. We also have controls for access, uh, cloud, CNAP, CSPM, use or whatever you want to call out, but I'll just call it fundamental controls, uh, fundamental controls. And, uh, let's just also call them compensating controls for the new age of AI as well.

Raghu: Yeah.

Ashish: Because the example that I just said earlier where if I had something on my cloud it, say Bedrock linked to an S3 bucket, which is public, uh, my fundamental controls would have still helped me because I would have picked up on the fact that there's an S3 bucket open to the internet, which just happens to connect to a Bedrock. That would have been raising alarms everywhere. Now- If I were to CISO today and trying to build a program for this, the way I would do this is I think the table stakes today have become the fact that my developers are using some kind of coding agent to develop, uh, code. Um, right? So if I were to just to-- if I figured out, okay, uh, A, that is going to  be a growing problem, but that's something that I need to pay attention to. The other growing problem is going to be around whatever this is producing, it's going to  become features in, um, in existing applications. So that's the two big thing that I'm caring about. But I clearly cannot look at 400 plus applications at a given point in time. So what I should in the beginning is I figure out, out of these 400, which are my publicly exposed, uh, crown jewels. That's where I'll start, because you can't boil the ocean. So I'll start there and identify which k- what kind of AI interaction do these com- do these applications have and start there. At that point in time, you need require, hey, whether it's a application that talks to an LLM API or is this an application that builds agents or is using agents, or is an agent itself, because each one of them has its own roadmap to g- to follow. Um, assuming you are able to maintain a, a AI inventory of that, even if we were to just do that, like for crown jewels, it is pretty awesome because most people are still trying to figure out, "Hey, do I need an expensive tool to, uh, do this?" But honestly, if in an organization you have enough humans to kind of do this manually as well if you want to. Uh, and b-- and most organizations are not producing the codes or releases as fast as Anthropic or any other people are. So it is still a couple of months or quarterly releases at max. If you're super fast as well, it's the... You have seen the rate of change come in, and you can manage that. very different when you go into like, uh, one of those AI Ford companies who have basically gotten most of their organization building code. Over there, you now you have weekly releases, no products coming out. That, that's a very different scale of problem. But brownfield would not have that problem today. They would... So if you identify crown jewels for what kind of AI interaction are they having? Are they talking to an API? Are they talking to, uh, agents? Are they building agents? Are they have third party? that a- inventory is, if you need to figure out a way to have a real-time information for that. is step one for most, uh, AI security programs that people are building. Now, the step two after that is what-- and I'm going to  ignore the fact of how AI is being adopted and all that, uh, which is a huge, huge conversation in itself. Um, but I'm assuming all of that due diligence has been done. There's a governance committee, and all of that has been already been done. You've landed on the crown, crown jewels to use AI after all that due diligence has been completed. So first thing, look at that. Once you have that, that i- that basically gives you the opportunity to identify where your gaps are today, whether it's in the fact that, hey, we have data that is being consumed by this, uh, crown jewel that is not segmented correctly across the network. We need to figure out a way to segment that, or we need to figure out a way to get the identity of the, uh, users who would be using. Is it my customer who's creating a or is it my internal users who can access it? Who has access to the AI? Because everything else is tech- technically covered for, like we've done this before. Uh, security hasn't changed. What, what we're looking out for is what's the delta that we're trying to cover for? not everyone may have a prompt injection problem. As much as the internet and OS and everything would call out, yes, prompt injection is a thing, but right now a large majority of that is internal facing for, hey, I let my, um, may I... My AI agent or my AI product go through my email. It had an instruction there for what is a prompt injection, whatever, or I make it go through a website which had a prompt injection there. Like that's very much where on that internal side, it's not on the external side. But majority people care about, if you think about data ex- data exposure, data leakage is what people give higher importance to. If you start there, uh, depending on whether you are consuming AI, you can go for an AI gateway or you could look at different approaches there. But the one thing that has come out as a theme in all the work that I've done across, across all the strategy, uh, strategic conversations, the workshops, has been the fact that it's not enough just to do, hey, I have an AI gateway, I have microsegmentation, figured out identity. I have figured out my LLM firewall. Because the, the, the thing that you're looking out for is the change of behavior of the agent. Going back to the example that I was saying earlier, I realized that, um, hey, I-- by my network is defined for this S-- AI system, let's just say in this case, uh, which is one that we built, can talk to a message board, and that message board is giving me logs. Now, the question that I asked you earlier, "Hey, does Raghu work for the organization because he's asking me to do something?" Yay, nay, whatever. Right. Maybe say yes. Next question: "Hey, yeah, Raghu is asking for his password because he forgot he was on a holiday, has come back and fi-- asked me to find out password. And I like-- As a human, you would go, "Eh, that sounds suspicious." But as an agent, well, you've done the check for the identity, the network out what data is there access to, what capabilities it can write and read. You can realize very quickly what the roadmap should be because of the interconnected systems and how far it can extend to. That's kind of where the big unlock is, where you soon realize you may have the inventory, you may have the roadmap, but what you need to identify, and which is the current gap in the organization across the board, is identifying the intent of the AI system you're building.

Raghu: You, you took that word out of my mouth because that's exactly the question I was going to  ask you next, right? To follow on from what you just said is that, um, like if we now connect back to some of the things that have-- the conversations that have very much come to the surface on the back of OpenAI Hugging Face, even though, I mean, these conversations are happening, uh, anyway, um, sort of over the last 12 months but definitely come to a head now, which is around like you've got this essentially non-deterministic system potentially, right? Where, um, so you're not able to predict behavior because a lot of kind of what we have built till now has been focused ultimately based on, "I can still predict, so I have determinism." So how, like how do you secure, um, when like the intent may not be clear, right? Um, where you can't predict behavior. And we hear a lot of terms around, okay, we've got to strengthen guardrails, um, this concept of sort of alignment, right? So how do all of those things play so that you can still get a good handle over-- You can, you can still control with predictability a system that is fundamentally unpredictable

Ashish: So, um, I learned this when I was writing the book, uh, and I came across-- I've, I've-- this is not a security problem, right? Funny enough, this is a, they-- sorry, data scientists work on these kind of things. They've been doing this for d- for a long time. And what we landed on, uh, there's a term called eval that people talk about. The whole idea is that when your agent starts drifting away from an existing behavior, how do you pick up on that? And then that's the term that you're looking for. And anyone who's watching or listening to this, they'll probably, uh, Google this, and I'll highly encourage this. But the whole idea is that evals are, in the most simplest term, a set of deterministic you do. It's not non-deterministic. Always-- it's always a set of deterministic checks you do to identify that in my, in my case, um, example that I gave earlier, would put a deterministic eval check for if the person asks for a password, a write action, it's always a no. But the moment it starts going in that direction, and this is continuous, by the way, it can't just be that, hey, and unfortunately, because we have been trained for the decades of IT that we have done, that we do a check once before an audit. Good. I come back six months later. that doesn't work in the AI case because that could literally happen the second you leave as well. So it has to be continuous. So what a lot of people are doing is a lot of people who have been AI-- very AI forward, they're building these deterministic evals. Now, some people have also said, "Hey, deterministic evals go sort only to an extent where what if it's a really long, um, phrase or what is a really complex message? How can you have deterministic controls for that?" So there is something called LLM as judge that people talk about, where you ask an LLM to judge the output or judge the, uh, input and output that you've been given as a prompt or an output that you got, uh, as a result of your prompt to identify what's the intent here. because it's hard. As a human, we can do that. We can look at something, hey, you should not be asking password, but, uh, uh, LLM doesn't really know, right? And that's where you add the third layer, which is the human in the loop part, where for certain actions which may have-- be of a certain intent, you have that human in the loop to go, okay, for actions like these, you need a human approval. And some people apply all three, some people apply just one or two, depending on what your organization application is. But that's kind of where we are getting to, and that's where I was saying that... Actually, I've, uh, made, uh, quite a few posts about this. It is not possible for a cybersecurity vendor to know that because how would you know the context internally? Like, yeah, it's like don't ask for password sounds obvious because a simple example, but it could be a complex example based on no organization is simple, let's just say of the brownfield that we saw called a simple organization. We have been-- us still have Windows 2000 lying around somewhere in the organization, right? What does that mean? And I think that's where that lands. I think so evals is one way people can figure out, uh, what the intent is, but the reason you don't see a product company talk about this because that has to be created by someone internally. And I'm not saying build a product, but you would need to build some kind of a harness that does an eval. Where I've been working on, so, uh, the AI security lab that we have, we-we've been researching, we plan to release an open source tool for this particular thing as well, where you have an eval framework, then you can extend that to your policies and whatever you want, because everyone would have an AI usage policy. Yeah, and I don't think it... At least for me today, I, I think there is something that like that's required, but the thing is, it's just not worth the time and money of a product company because that is a problem that cannot be scaled. Like I- if I make-- I'm going to use a few names. Let's just say I am g- a product company that is an eval product company. I'm going to work at Bank of America, but Bank of America applications are completely different to. And it's the entire eval set is need to be very different. You can give me the framework, but everything else around it like, well, I mean, J- Bob configured the same application in a different way. SAP is configured two different ways in Bank of America and JPMorgan Chase app, but just looks different. Does that make sense?

Raghu: Yeah, it do, it to-, it, it totally does. It totally does, right? Um, and so yeah, you, yeah, I, I, I absolutely get what you're saying and sort of the, the like, the, the concept of evals as, as a key part of how you, I guess, like assess an agent, right? And det- and as get confidence on the things it will do and it won't do

Ashish: Yeah. Yeah. And it w- it would have to be-- I think at this point in time, uh, it's funny, I was talking to someone yesterday about this, and I, I, I, I had this strong belief that, well, which I-- which is basically what I said, the evals have to be internal, because my belief system comes from the fact that, well, traditionally, cybersecurity com- products have not been given access to the internal backbone of the organization. Like, we have not been told that, "Hey, I'm going to  plug my cybersecurity product into my Slack, my Confluence, my SharePoint." That is asking way too much. I mean, every time a security person knocks on engineering or HR, they're, they're, they're basically going on tenterhooks at that point in time. What, what, why do you need this, need that? And are you doing an audit? Am I in trouble?" Like, that's the sentiment we have in a general conversation. This is not even AI conversation. But as I say that, I will remind people of the example where we were told as kids that, "Hey, you should not get into a stranger's car." But I take Uber all day, every day today. But the pers- I wa- I walk into an Uber, "Hey, is Ashish?" "Yeah, it's Ashish." I sit down, I go on an hour-long ride, half an hour ride with a stranger, no questions asked. our, our-- and I think there was another example where, oh, we never used to meet ma- meet friends with strangers. People add other people on LinkedIn all day, all. But-- and we still accept the connection. I have never met the person in real, real life, but I still will accept it and go, oh, wait, our trust level as humans has continued to evolve, and my thinking at least today is that the evals would be internal, but maybe tomorrow they'll b- model may change completely. And suddenly people are like: "Yeah, I'm just gon- you know what? I'm okay for security to look at everything in my organization will change tomorrow, but at least today as it stands, until that happens, for me, the belief is that, well, it's not going to go away.”

Raghu: Um, so, okay, so let's kind of close the loop on this part, right? Because, um, we know that an organization is going to  adopt a technology if they believe it's going to  give them an advantage, regardless of whether security is ready or not, right? And that, that's played out over historically over time on repeat, right? Cloud, containers, right now, uh, with AI, right? So as security professionals, what have we learnt from all of these changes? Um, and about how-- like what have we learnt about, well, we can't stop it, so how do we say yes without also kind of lying to ourselves that, ah, it'll be fine?

Ashish: Hmm Uh, this is the... So I, think people are trying to change the yes, but conversation, right? I think so earlier today, y- uh, I think AppSec is ecosystem tried driving this for DevSecOps, where the idea was that, no was the, hey, we should stop saying no, was the first campaign they came up with. Then it became like, oh, we should say yes, but if we do it this way. And I think the same applies over here where, uh, fortunately because the, to what you said about business feels there is an advantage, but no business out there wants to pay so much money for fines or anything else that they can stop existing as a business. So people understand that. And I think, uh, for the first time, a lot of security leaders that I've been working with, a lot of them have a say in, "Hey, we should do data security before we start implementing this." Like, let's get something which is doing a, um, micro-segmentation because relevant for the audience. Like, hey, we should do micro-segmentation of how data is shared and what the cloud infrastructure is, so that we can to an incident fairly quickly, or we can identify incidents fairly quickly, and we can separate them out so th- we can reduce the blast radius of it. Like things like that have become top of mind. So we're, we're still on that yes, but conversation, but I feel the, the yes and yes, but is easier today because, uh, uh, like say if I was trying to do DevSecOps or any kind of cloud changes, it didn't really impact the CFO or CEO. They didn't really care. They're like, "Well, I think the CTO is more important here, so let's just compact this for now and come back later." But today, because there is pressure from board, the CEO, CFO, CTO, CISO, they're all on the same page that we have to use AI, and they would all look at each other for what's the best way to do this, us to do this in a way that we don't stop being a business. Which also means that, hey, I don't want to get fined excessively, which, uh, for people in Europe, there is the NIS2, which is kind of now. There is similar things coming up in the US side as well. So there is, there are regulations coming in which are making people go, "Okay, wait, let...” You know what? I get that, but I don't want to jump in straight away. Uh, but what's the best way to do this? Let's talk security. Let's work with them." But that doesn't mean I'm slowing down my coding agent, uh, usage, because that is still all internal. I think where people are drawing their yes, but conversation more is on more on the external side of things. that in-- If you look at the typical risk topography of an organization, the internal risk is always is considered as a lower risk because I can always walk up to Raghu and tell him, "Hey, stop doing what you're doing." But I can't tell Ashish on the internet, "Hey, just stop poking us with this, whatever this random IP is. You, but then you come back with another IP." That makes sense. So that's why the external, uh, risk portion has always been higher. So for those kind of conversations, I definitely feel like yes, but is a lot more easier Yeah. Whereas the internal risk is still the same, where we want to drive coding, we want to drive the usage of AI. That's everyone token maxing until we basically burn down as a company. Like, you know, keep doing what you're doing, use more AI.

Raghu: Yeah

Ashish: There is obviously that conversation about cost to- of tokens is changing as well, but that's kind of where I find that evolution of yes, but has shifted where the yes, but was always for, A, everything, but I think the nuance there is now the yes, but is easier conversation for external facing, applications we have.

Raghu: That's-- I think that's really, uh, there's a, there's a really important point there and a, a differentiation you made that I think it's important to keep in mind because when we were adopting cloud technologies and containers, that was very much a technology-driven conversation, right? This is a better... This is a technology transformation that we just market as a business transformation to sort of get, oh, b- uh, because as soon as you say something business transformation, it's like everyone gets excited. But I think the change with AI, if we're, if we're looking to differentiate, is that AI at the-- or sorry, the adoption of AI is very much a business-led transformation, right? The, the, the, the t- because it's doesn't now become a technology conversation at all in many ways, right? And I think that's the fundamental difference, which is, and that's where you said for, is that suddenly you're not just having to convince the CTO, you're having to convince all the C staff.

Ashish: That's right. You can then move forward. Yeah, I think that's an important. And, uh, uh, I would want to give some credit to the Anthropic people, right? Because of the meteors fear they drove in people, least that's made this conversation easier because I think that has to contribute into the whole yes but conversation because my dad knows what cybersecurity is, and he's like 70 plus, it nothing to do with the internet apart from social media and what's happening in like the, uh, well, I guess the Indian politics, I guess. You almost feel like that has made the conversation such a easily accessible conversation with most people, so most board and C-level executives understand that. I think that's where CISOs have a lot more pull now, they are able to at least talk about, "Hey, this is how I would recommend it. So if we take this risk, I don't know if I'm confident enough that we'll be fine." And that definitely is being heard at every level

Raghu: I, I, I think absolutely right. I, I, because wh- what I've observed, because for so long now, now speaking as a security vendor, we've been saying, "Oh, you know what? Security needs to become a board level conversation." CEO needs to, needs to be talking about and dreaming about cybersecurity. Like deep down knowing that, look, at the end of the day, the CEO and the board are going to say, "Right, what's our operational risk?" Right? Uh, and, and they leave it at that, right? And cyber has just been put... But that you're right, like post-Methos it's like suddenly it's like this is a risk category all in and of itself.

Ashish: Yeah, and, and that, I think that's a good thing. Uh, to your point is that it- it's now bringing a level of focus that maybe cyber has not had previously.

Raghu: So, um, le- let's, let's kind of go back before we, before you wrap, um, let's go back to sort of where you and I met, right? At, uh, at Black Hat and you were hosting, uh, uh, a comedy roast. So firstly, um, are CISOs as funny as they think they are?

Ashish: Uh, I think we think we are funny, but I don't think the crowd thinks we are funny. So it's like I realized very quickly that I-- because I was trying to test my jokes out with non-security people, none of them laughed, And I'm going, "Wow, okay." But then I said the same jokes to security people, they all laughed. That means it's-- So I feel like we are a bit of a clique, for lack of a better word, where if the jokes are, um, I'm yet to see a CISO who's funny in a nor- in a, in a broader context, if that makes sense. I'm sure they're like funny in the CISO security jokes context, quite a few of them. But where it gets tricky and, uh, um, it's you truly feel that's a joke, that's a ha ha ha joke, but most of our jokes are, "Oh, um, it's funny because it's true." It's not a funny joke because I have observed it. It's a funny joke because it's true and I'm, I'm crying about this all day. So I, I feel like to answer your question, are funny, yes, but in a clique, not outside of... Like no one else

Raghu: Yeah. We have a lot of in-jokes, right? Yeah, That's pretty much it. So, a- another thing that you, you keep a bit of a scorecard, right? You make predictions and then you go back to them and say, "Hey, yeah, that was a great prediction," or that, that was... So give us a prediction that you made at the tail end of 2025, early 2026, that has already failed the test of time.

Ashish: Um, so fine enough. the first thing that I came up with, and maybe the book humbled me quite a bit as well, because the first version of the book for AI secure engineering did not have any agents in it. And I started writing the book. I got the from Wiley in Jan 2025, and I started the book. But after I started writing the book, Claude Code, by the way, came out in Feb of 2025 And you think it's not that far. It's, it's been literally a year and a few months that Claude Code came out, but imagine the impact it has had the board, right? And then, uh, around June-- So I was supposed to release my book around RSA, which is like the March kind of month, right? And around May or June is when, or maybe July, when agents became a thing. Suddenly within, between six months, February all the way to August, suddenly the whole world of AI has changed. It's no longer about I'm talking to an API, which is an LLM, so I need to manage my tokens like that. Suddenly it's like, oh, this can do things. It's-- Uh, then Open Claude came along, then it, uh, then it explored even more, and you're seeing like all this huge news being created. So initial banking for me was the fact that, hey, uh, I think AI secure-- AI, uh, as a, as a whole, doesn't need to be, uh, its own category. And so for the longest time, I was, I was still not sure if AI security engineering is the right, the right word for the book, because everyone that I spoke to, they're all like, "No, it, it'll be... It would integrate into existing functions of cloud sec, app sec, SOC," because you could see there are vendors who are doing AI SOC automation. There'll be a soon a vendor doing AI GRC automation, AI insert category automation, right? It didn't feel, in the beginning, uh, and this is early 2025, that it would become a thing. And I, I thought I held the belief till later, but once the agent became a thing, I think, uh, there's a, a person that I followed for a long time called Naval Ravikant. He's basically like, “Oh, yeah, yeah, yeah. Yeah, I know.” yeah, I-- M- he's basically-- I don't think he makes predictions, but he, he's interesting saying-- He, he's popular in the tech people, for lack of a better word. He uses a phrase called, uh, "Strong beliefs loosely held." And for me, I had a strong belief that, hey, AI security doesn't need to be its own thing. People would not build teams for it. And, um, I was corrected. People wanted it, it to be a separate thing. And by July, I was like, I was convinced. I'm like, okay, I was wrong, but I'm happy to be corrected. So that's why I made a post some time ago where I think I, I may... I think I didn't, I didn't have the previous screenshot, but I tried looking for who was hiring for AI security. There were like 10 jobs in June, July last year. Today, if you look at this, 100 plus jobs for that same thing. I-- It was, it-- In a way, it was a good and bad. Like, so it was good that my prediction was wrong, that AI security doesn't need to be its own dedicated space because... And maybe it comes from the bias that all of us security people always are like No, no, no. It like-- No, we said the same thing about cloud security. Why would someone want... It's a part of network security. Why would someone need a separate one? Then suddenly this ecosystem come explodes, and now security people. Now we are all checking our identity again. Am I an AI security person or am I a cloud security person? I don't know. So it would be really interesting what the next evolution is. So, uh, that was one belief that I had to change and, uh, that's why the book became "AI Security Engineering."

Raghu: Awesome. So the book is called "AI Security Engineering," published by Wiley. Who's it for? Who's it targeted at? Is it practitioner CISO?

Ashish: Uh, so it covers anyone who is building and securing AI at scale, which means your AI engineers are also the right audience for it, your AI security engineers are the right audience for it, but also the CISOs as well, because I also talk about how to drive a program in security, the kind of conversation we had earlier about this. Now, I, I do want to caveat, um, because AI is evolving very quickly, I had to draw a line at some point for, hey, I'm only going to cover this much in the book, because I could not predict there'd be Grok bots floating around, there'll be like Meta would have their own, uh, bots as well, and there'll just be like this Claude co-work and everything else would come out as well. So I had to draw a line somewhere and which is why, uh, as a, as an adjacent to it, we started something called AI Security Lab. The whole idea behind that is that that's the practical-- Like, so if you want to keep up with it, uh, aisecuritylab.io is the website that we are trying to maintain, and the idea is that it's the, it's the next up- next is the, it's a follow-up from the, where the book ends, all the how do you create Grok bots for security teams. It has all the research we're doing from an AI security perspective, because no one is doing the AI security research at the moment. Like the AI security research you do see are more from, hey, I want to find a vulnerability. But going back to where, where we started the conversation, we are coming more from a practitioner side. How does a CISO use AI to scale their organization? How do I integrate that into a cloud security workflow? How do I integrate into AppSec security architects? The list goes on. uh, the reality is that there's no point in a product company talking about it, and so, because that's not their domain, is why we have taken that baton and at least at Decryd, we are trying to push that forward. And, uh, AI Security Lab is, is that, is that fork on the ground, for lack of a better word, where we're going, okay, we're going to  start doing more research, releasing more workshops and stuff in that space where we, we want the entire community to rise, people who have access to the frontier models to be the ones who are just amazing at AI security.

Raghu: A- absolutely. So, um, and where can people get the book? Is it available now?

Ashish: Yeah, yeah. It's, uh, so it got released, it got launched at Black Hat. I was fortunate it went number one on Amazon US and...

Raghu: Wow. Well done

Ashish: ...and, uh, didn't make my dad proud, but hey, whatever.

Raghu: Of course. That, that, you know, that, that's, that, you know, d- don't try, don't try and win that battle

Ashish: Yeah. so we basically, uh, landed on, uh, any Amazon store you'll probably find it. I think it's in Barnes & Noble as well, so yeah. And stores, Kindle as well. Kindle, Audible as well. So search for "AI Security Engineering" of the book platforms, you should be able to find it.

Raghu: Amazing. And when am I getting my signed copy?

Ashish: Uh, dude, soon. It's already in the mail. Just like... My agent just picked up on the c- on this conversation and suddenly gone, "Da, da, da, da."

Raghu: Yeah. We'll, we'll meet for a coffee that costs less than 10 pound in London, and we'll do it. All right. Okay, a bit of a lightning fire round before we wrap, right? Um, okay. So you've, you've got to answer all of these questions in one sentence, um, and no preamble, et cetera. Right. Take a position, give it to us. First question, the cybersecurity truth most organizations don't want to hear?

Ashish: That basics have been the root of everyone's security problems, like basic hygiene, literally. Like if I could just stop a sh- uh, Bob from clicking that link, I would not have to r- uh, deal with email security, data security, any of that. Sorry for any Bob on, those listening to this.

Raghu: Okay, question two. Something the industry treats as accepted wisdom that you think is absolutely wrong

Ashish: Oh, that our, um, cloud providers have all the answers we need. I'll add frontier models as well. So frontier models and cloud providers have all the answers we need

Raghu: Yeah. And third question, the last time you were significantly wrong about something in this field, what was it?

Ashish: Oh, um, it's funny. I think I'm-- I've, I've, I've been lucky to have a lot of right. I'm trying to think of... The wrong was the AI security vertical that I was talking about earlier, I don't think that should have become a field, but became a field. Um, but I'm trying to think if there's anything else that stood out just absolutely wrong. Oh, um, I thought, this is my bias, I thought majority security people, um, would pick up AI, uh, really quickly. And I don't mean in a negative way, I mean more in the context of I've-- I found that it's a, it's a mental challenge to go through, uh, being a builder. I made a post about this as well, because AI is expecting you to build things, but security has always taught us to how will this thing break? And I mentally had to go through this journey for it's okay for me to build things without thinking about security first.

Raghu: Awesome.

Ashish: A reason why security is always number two, because you can't-- If you think about security first and start building, you never build the thing.

Raghu: Yeah.

Ashish: There's just so many things that scare us, so that I underestimated the journey that all of us security people have to go through. I personally went through that, and I'm still go- I think I'm still going through that today, where, uh, and I was totally wrong about it. I thought because we are all technical, we'll just pick up AI like that. But nah, it just the mental challenge of switching brains is, is not easy. Maybe it's I'm just old. Maybe that could be that as well.

Raghu: Yep. So, okay, so finally for the listeners, right? What is one thing that every listener can go and start putting into place that will reap them real benefits over the next 12 months?

Ashish: I think because we are recording this in 2026, I have to say, um, it's very likely you have the pressure from your bosses and bosses' bosses that you have to use AI. So I would say the, the future jobs for security would have AI integrated in them. And the example that I'll use is when a lot of us are applying for jobs, we didn't really call out Word, PowerPoint, Excel in our resume. And if you are looking for a job in the market or looking for a promotion, who are getting those are the ones who are comfortable with AI. You don't have to be a data scientist or an AI expert, but just being able to build a workflow from using AI, irrespective whether you're on the vendor side or on the practitioner side, that is going to  reap you benefits because everyone is lost in terms of how much of their jobs can be done with AI, and mostly because of lack of time, because not that we had free time before, and now you add AI to on top of it. So if you can become that person who spends your weekend trying to figure out, "Hey, how do I integrate AI into a workflow?" That promotion or that new job would, would definitely be yours.

Raghu: Awesome. Well, Ashish, thank you so much. Today has been the absolute proof that hookups in a Vegas bar can be productive. So th- thank you so much. We...

Ashish: ...kind of hookups to be precise.

Raghu: The right kind of hookups. No, no, no, no, no, no, we don't endorse any brands here, right?

Ashish: Yeah.

Raghu: So Ashish, again, thank you so much for your time. It's been so, such a fantastic conversation, really getting the perspective on AI and AI security from a real expert in this field. Um, encourage all of our listeners here, go and check out Ashish's truly amazing podcast. I am a huge fan of them. The Cloud Security Podcast, the AI Security Podcast, and of course, Ashish's already kind of, um, chart-topping book, right? AI Security Engineering. Go and grab it. I'm sure it's a fantastic, uh, read, and we'll put the links to all of those things in the show notes. Um, but again, Ashish, thank you.

Ashish: Yeah, thank you, Raghu. Thank you for having me. I really, really appreciate this

Raghu: Cheers