Raghu N  00:00

So welcome back to this episode of The Segment. Today's episode, we bring together two leaders who have spent decades shaping how the industry thinks about detection and response. In the blue corner, we have Dr. Anton Shivakin from Google Cloud, a long-time industry analyst and one of the original voices behind modern detection response. He's widely recognized for his work in seam log management for coining the term endpoint detection response, or more commonly known as EDR. Anton, welcome to The Segment.

Anton  00:32

Perfect, thanks. I'm happy to hear I'm in a blue corner, and I'm dreading to think that Erik is in red, so we'll see.

Raghu N  00:40

I'm glad you remembered what corner you were in, because I had already forgotten. So, in the red corner, we're joined by Erik Bloch, who has led security operations at companies like Salesforce, Cisco, and Atlassian, and now heads security at Illumio. Erik brings a practitioner's perspective on what actually works in the real world. Erik, welcome on to your first appearance on the segment.

Erik  01:00

I know I'm finally stoked to be here. It's only been like a year and a half, and you finally invited me on here.

Raghu N  01:06

You've gone through the qualification process, and we've eliminated everyone else.

Anton  01:09

I know, right? Who's left over? Okay, I guess we need Erik on.

Raghu N  01:13

So, together they've seen the evolution of detection response firsthand, and in this conversation, they unpack why, despite all the change, the outcomes still look the same. So, let's begin this. Well, firstly, I'm super honored to be in the midst of two industry legends, so they still let me go and speak, the serious folks, the grown-ups in the room. So, I appreciate that. Anton, over to you. First of all, as someone who has been commenting and observing and analyzing the evolution of detection response. Can you take us on a quick journey of essentially the evolution of detection response and tied to that the evolution of the SOC?

Anton  01:52

So it's, I'm happy that you didn't start with how everything's broken for the last 20 years or something, but I do want to, I do have kind of an informal history in my head, I feel like from my involvement it started in the in the early 2000s but of course when I started doing cybersecurity and when I started realizing the DNR detection response would be kind of a fun focus area, I read up all this original materials on the network IDS, I was familiar with the detection, as it was practiced in the kind of late 90s, but even though I wasn't part of that, at least not firsthand, so it certainly started from a bunch of network ideas, discussions, and deployment, and then people got really, really, really, really sour, and Gartner, of course, the well-known analyst at Gartner published IDS Is Dead paper in 2003 A lot of SIEM and SIM were born. Ultimately, and this would be important for the current evolution. A lot of SIEM and SIM, which later became SIM with E in it, was driven by the fact that IDS was really noisy and they needed some kind of attack to look at the alerts, and to make sense of them, to add context, to enrich, as we would say today. And, of course, the word correlate was born. Some of people who are much younger than Erik and me think that is some kind of a cool new technology, even though pretty much every SIEM vendor around 2000 to one, maybe 1999 I'm not entirely sure about 1999 was promising correlation as a way to make a deeper analysis of ideas, awards firewall log data, and whatever telemetry we had. So this was the part of the ancient history for most people. And then the right before we realized the problem is not being solved, compliance showed up. The Sarbanes-Oxley 2002 PCI became very popular mid 2000s and a lot of tech that was built for helping detection, like Sim was kind of maybe not repurposed, but kind of dragged into the compliance corner and left there for 10 years. So even today I see people who say, oh no, no, I don't want the SIM, give me an ADR, SIM is a compliance technology, and I'm like, yeah, but no sim predates compliance to a large extent, but there was a long period of association with, like, reports. Give me the status of how users are logging in for the last month. So this was, like, we're talking about 2000s and then early 2010s people started thinking, what should we do, and EDR was born, and I don't want to. Well, I guess I do want to take credit for inventing the term, but the point is that, of course, I didn't invent the technology. They were the inspirational tool at the time was Mandiant MIR, Mandiant Intelligent Response, I think it was called, and a lot of other vendors started. Carbon Black started very soon after that, and a few, a whole bunch of others, and then EDR was born as a separate thing, and the history of that colored a lot of 2010s because people thought, "Oh yeah, Sam, that's all, that's compliance, we'll just do EDR, and then Gartner made a really odd mistake of not calling the network detection NTR right away, there was a brief moment when Gartner. They called it NTA. Everybody laughed at them, and thought that's really stupid, but, but the point is that then they later they correct it, and NDR was born as well. I didn't do it, even though I was a big fan of not calling that technology NTA, and please don't ask me what NTA stands for, nobody remembers. And then detection was progressing down that path, and unfortunately, as we usually do in cybersecurity, we overdid it. And towards the end of 2010s I've met a lot of people who said, "I have a DR, I don't need anything else, and my reaction was, "Oh, so your every router is instrumented with EDR agents, and they'd be like, "Well, not the routers, oh yeah, every container as well. No, oh, what about your glorious, beautiful, secure, fortunate appliance? It's got a DR, right? Okay, Anton, what are you trying to tell me? At which point the conversation really started with, well, dude, you need to have logs, and they say what, like using sim, like, like the in the old days, and like, yeah, not in the like old days, the same modernized quite a bit. So this is kind of a rant, and Erik nodded, so hopefully I didn't say anything truly horrifying so far. And then there was a period of time when people started cranking out those something DR acronyms, CDR for cloud detection response is still kind of a thing, but also not a thing, who knows, and then people launched ADR for application detection response. There was a very brief detour when a couple of vendors that are saying maybe two three years ago, launched DDR for data detection response, and while I was kind of happy that my indirectly speaking, my legacy live zone, I realized that they're just renaming DLP, and then I thought, oh, so this is like DLP, but you hate the term, and they said, yeah, pretty much, and by the way, we, and everybody else, okay? I have some compassion for you, but please don't call it DDR, because it wouldn't really survive, and I think DDR didn't survive. Where are we now? Oh, yeah, there was also a brief detour when there was a brief, brief fascination with it with XDR. I really hated the term, and I really, really, really, really hated the term, but by the way, if you asked me how I feel about XDR, guess what, I really hate it, but and people say, Anton, you're only hating it because Forrester made it happen, you were from Gartner, and I said, I mean, yes, that has something to do with it, but it isn't the whole truth. The rest of the truth is acronyms who don't attach to something that you can clearly buy needs to die. If I go to the store and say, 'Here's dollars, give me a DR, I'll probably going to get an EDR. If I go to a store and say give me a sim, people would give me a sim, but if you go somewhere, waive your money and say give me XDR, you would be given four or five different things. Some of them would be like an improved EDR, some of them would be more like broken down sim, some of them would be something in between, some of them would be a combo of EDR and NDR, but the point is that we never agreed. We collectively, the industry never agreed about what this is, and to me this was a very clear sign that needs to go. Well, it needs to never have been born, but sorry, Forrester, but because you gave birth to it, it kind of needs to go, and I think Gartner finally published a piece, saying now that XDR is finally dead. Hi, Forrester. We are happy to report whatever they said before. I misquoted a tiny bit, but it's just one part of DNR space that I'm happy is gone. I need to have some kind of a pithy conclusion. I don't have it. Let me hand it to Erik.

Raghu N  09:00

So, what I'll say is this. I think across all of our guests on the segment, Anton, I think you win for the most energized, informative, and funny sort of opening. So I've had to sort of stop myself from laughing out loud. So all right, Erik, what have you got right? So let's, why don't you do this. So we've had sort of the history of the evolution of, let's say, kind of what makes up essentially the sort of that seam detection response, etc. Now if you give us the evolution from a practitioner's perspective as someone within the SOC or the detection response function, how has that role's responsibilities changed over that same time,

Erik  09:47

from the, you know, from the view of being in the trenches, you know, while a lot of the names and the faces have changed, you know, we've, we've changed technology, we've gone from, you know, on premise to the cloud, we went from. Data centers, to you know, in physical servers, the containers, or whatnot, that the technology has changed a lot. The names and the players have changed a lot, but from an actual in the trenches perspective, not a lot has changed at all. I mean, we're still talking about the same things we were 20 years ago, with, you know, having alert fatigue. There's too much noise going on, we can't, you know, find the bad things. We're still looking for the needle in the haystack, right? We haven't figured that part out yet, either. So, I mean, while a lot of things have changed, like, you know, if Anton has alluded to a lot of the core tooling we rely upon today is still very similar. Like, yes, the Sims have evolved and grown up. Yes, the endpoint detection systems and the endpoint protection software has grown up. Yes, we have more telemetry and more visibility into things, but it has actually changed the outcome that we saw 20 years ago versus today. Not really. Again, we're still trying to weed through all the mess to find the needles, and that the needles in the haystack are still somewhere between four and 7% of all the noise look through, but we haven't come up with a magical way to do this better again. That there's been tons of tools going out there, a lot of, you know, companies have been acquired, a lot of people have made a lot of money, but the people on the trenches are still, are still fighting the same battles we were 20 years ago today.

Raghu N  11:16

So, on that right, if, if those are the trenches of fighting, fighting the same battles that they, that they were 20 years ago, and not a lot has changed, and it's the same challenges, right? And the same sort of like the signal to noise ratio is still effectively the same, like what is then the value, and maybe both of you will provide a response on this, and I'll go with you first, Erik. Then, if that's the case, what is the real value that the SOC, the SOC function is providing to the wider cybersecurity organization?

Erik  11:52

Well, from my perspective, being in the trenches, it solves multiple purpose, not just kind of being the, you know, something bad happens, they respond to it, like the first line of first line of defense, the first line of response you have, but they also serve other purposes as well. I mean, from a compliance reasons, you have to have them in place, from customer reasons, for legal reasons, that they're supposed to be there, but again, they also kind of act as like your 911 function for your company as well. If something bad happens in the security space. Who you contact? Well, the SOCs is supposedly there 24/7 right? So it's kind of the tip of this spear for security in general for a lot of the larger organizations I've been having. When I've been in, you know, Alaska and Salesforce, Cisco, like that became your central landing spot for all things security. I know other organizations are running it different, some of them run it strictly as a kind of a tactical function, where they're just responding to alerts that are coming from the machines. Some of them use it, like at Salesforce. Again, there was a central landing space where people were reporting things that landed there, people were reporting phishing emails with land there. We had all the machine-based noise that would land there, and they became kind of like a kind of draw the analogy somewhere to like an IT help desk type function, but for security, where they were dealing with all human-based noise that was coming, all the machine-based noise that was coming in, and it was their job to kind of like sort it out. Can I fix this thing within whatever my SLA is, a half hour, an hour, or whatever it is? You're kind of deal with this, but escalate it. Does this belong to my team? Got this in some place else, and most of the time they send the work someplace else, because it doesn't belong to them. So they still serve a purpose, and this purpose is again kind of similar to what they did back in the day. It's evolved that their scope is expanded as they've gotten more access to tooling or whatnot, but from a strictly, you know, detection and response point of view, they're still kind of serving the same function as they were again 20 years ago, that first line response of triaging the noise that's coming out of all your machines, because again we haven't really found a better way to do that yet, meaning and for all the tools and all the technology that's been put out there, from the sim and again the XDR fantasy to SOAR to UBA, to now we can LLMs again. A lot of companies are coming and going, that the names and faces have changed, but the fundamental problems still seem to be the same ones we had a decade or two ago.

Raghu N  14:13

So let's come back to that fundamental problem in a second. Anton, like, how over time have just, I guess, following on from Erik's response is that how over time then have like the SOC function shown value to their organizations, kind of like what's the fundamental way that because we talk a lot about like is that is that cyber program delivering value, how does a SOC show and demonstrate that?

Anton  14:38

So I want to touch on one brief point that Erik made, largely kind of like a tangentially agreeing, and then we'll get to the value. So, Erik mentioned help desk for security. I think that the in my materials, in my, in my thinking, I sometimes use the SOC with NOC DNA, SOC, that's basically a carbon copy of 1970s AT&T network operation center. With big screens and the chairs, I think that help desk, I think it's more of a hybrid of help desk and NOC, because of the machine signals kind of go into the knock part and the human signals go into the help desk part, but it's the same DNA of humans waiting for things to happen, either by means of screens or by means of ticket queues or whatnot, and I think that that's the conventional classic traditional legacy, whatever term we use, is that thing, and let's leave this dangerous subject of a modern SOC aside for a few minutes, but the traditional SOC valued story was that type of picking on things happening possibly at an early stage, if something is just starting to break, they can quote unquote help detect it, or the machines would detect it. Humans would confirm this would escalate to IR, would escalate to people who actually know what to do, would escalate to IT teams. So, the this type of early warning function done by machines and people, but with a lot of work being done by people specifically, is quite central, and the value for this, again, to pick a random example, I first saw, like, a classic bank, large bank SOC, probably in the early 2000s I was working for a vendor, and it was like a vendor tour of how the customer does it in their SOC, and that was this type of client. What you would think is what I would think of the classic SOC, monitors, chairs. It was in-house SOC for a big bank, and the value at the time, if I'm vaguely remembering 20 years ago, was people thought, okay, we are pretty confident our team will pick something up early on before the damage is done to our company, before stuff really blows up, we would not learn it from CNN, we would learn it from the SOC, and the idea was that if they collect enough data, deploy the right tools, same at the time, ideas, whatever else, and have people carefully watch the signals and thoughtfully look at what comes in and triage and confirm and correlate manually, and using correlation rules, we will detect early enough. So the value was detect before damage is done.

Anton  17:12

Yeah, this was very clear. Is that that's why this bank spent whatever number of millions on tools, people, consultants to refine the process, whatever else, and that's it's not false, and for that bank it was very important, and that cost was justified, and I think they detected things early enough often enough, so that they continued to pay the bill for SOC, humans, machines, so that part is the classic value, and in 2002 it generally speaking could have worked. It has worked. It was a high cost, some value capability. High cost is very clear. Cost is fixed tools, humans fixed cost. You can't, it can't really go below a certain number, value of course, if nobody's attacking you, which is unlikely with a big bank, value is low if somebody's attacking you and you miss it, value is low if somebody is attacking you and you pick it, value is there. So this is like the my 3 AM answer to a classic SOC value is early detection. You have a lot of auxiliary value, of course. Oh, well, if there's just an anomaly somewhere, we analyze, we analyze the data periodically using some machines. We'll pick something like a hidden attacker, what later was started to could be called hunting. We would also help investigate. There would be other users, but this type of detect early was the main point, right?

Raghu N  18:39

Yeah, so both of you have mentioned, kind of in passing, right, lots of advances in tooling, and also the volume of tooling that organizations have invested in, but like in terms of sort of the base outcomes and the base problem hasn't really changed, is that just simply because of the nature of the problem, or is it because the tooling isn't providing the necessary coverage and gains that were expected? Anton,

Anton  19:05

I think that the idea was always that humans play a somewhat central role in this whole show, like I think that the original line, of course, back in that timeframe, late 90s, early 2000s, Bruce Schneier was at, I think, Counterpane, still, so the one of the first MDRs at the time, and then he coined the chart, he coined the phrase that has become very well known, that security is a process, not a product, and of course he was, he's correct, but he was also very self-serving to an MSSP provider, he was working for a managed security service provider, which is kind of outsource process outsourcing, so he wanted to say that it's not the boxes to tell you, but, but the process, and we happen to be, you know, helping with that, but the story is both self-serving, but also correct, it is boxes hasn't really stopped attacks, so the human side. Of SOC was always seen as absolutely non-negotiable, and only in the last two years from today people started talking about human, the SOC agents, robots, can we just buy a machine that does it? But it took 20 years of progress, and we can really heavily debate whether we are there now, and I think we won't debate it, because both Erik and me agree that we are not.

Erik  20:26

I mean, really to echo what Anton was saying. I mean, again, the players and names, a lot of things have changed, the outcomes seem they have not, you know, is that is that a because we're not deploying the right, or building, or deploying the right tools. Is it because there's other fundamental problems in place that we're not solving? Yes. You know, I look at a lot of the, a lot of the security startups that are out there today, and I kind of draw the analogy to the pharmaceutical industry. Well, they're building band aids, you know, the investors know they can invest in band aids, they know that the band aids will work for some time until they don't, and then they can invest in the next band aid, rather than going after kind of what are their fundamental problems that are called causing us to begin with, right? Because there are things we could do architecturally that would solve a lot of the problems that we respond to today. Do we do that? No, we buy more band aids to kind of cover up the architectural problems, you know, and we then respond to all those alerts too. So, I mean, there's systemic problems again with the tools that we're building, like, yeah, they're incremental, but we're getting a little better. Are they solving financial problems or some of the larger issues that are causing all this? We know they're not, and again, like we said today, with all the attacks that we're seeing happen, I mean, it's just, it's recycled news stories you've heard 1520 years ago, again, the names have changed, tools have changed, but the how things are happening are still very similar to how things were, you know, a couple decades ago,

Raghu N  21:59

so Erik, just in that response, right? I think you've, you emphasized, reemphasized, and reemphasized again the fact that really the fundamental architectural problems are not being solved for, right? Like Anton, firstly, do you agree with that, and secondly, if you do, why is that the case?

Anton  22:18

So, I, so I'm going to go with a side funny side angle, and then I'll agree with Erik. Sorry, I'll give you a spoiler. So, if I agree with Erik in 99% of the cases, then do I agree with him, or do I disagree with him? Like, can I find a counter example to this? Yes, can I find more than 123, but can I find there are 1000s of large companies on the planet, and there are millions of smaller companies, and can I, can I find something that exists that's an exception that exists at many of them? No, can I find something that's an exception, and I can waive it and say, here, look, this company over here is really doing very well in this, in this aspect, and it'd be like, sure, name one more, and I'll name one more, and Erik would say, name three more, and I'd be like, so that's the short, short, short finding side, but the more real side is yes, unfortunately, I was, because I was sometimes talking to people about, like, challenges, challenges in DNR, challenges in SOC, and I swear I've seen people make slides in 2020s that I made in early 2000 so these are the same slides done by different people who maybe forgot the lessons, and I looked at one slide that somebody in marketing put together and thought, I swear, this is my slide from 2002 and they said no, no, no, no, we just made it, and I'm like, I believe you, but I probably made exactly the same slide in 2002 and it had false positive, not enough automation, too much data, something, something, analysts not hand, not getting full context, and I'm like, this is when I was working for my first sim vendor, I had this slide, and I am mildly shocked that this slide was built in 2003 and you build a slide in 2023 and it's the same slide, so yes, and it's like I'm back in what Erik said, and of course somebody from the back desk stood up and said, Erik Anton, what about 2043 do you think it'll be the same? And then it was, I was just became depressed, and I wanted to not work for three days after that. So, let's avoid this, this angle. But the fun and challenging part to this is that there are exceptions, and when, when we sometimes argue with Erik, and this is an area where we argue sometimes, is about that I think that I do have a clear view of what's a modern SOC, and that unfortunately, and I can point at examples where it's done in a way that's actually very well, very working very well, but after this brief disagreement, I would immediately say that I really don't have many. Examples, and if you go look at the very classic Netflix blog from 2018 about how they do SOC-less detection, you'd look some of the stuff that we published from Google at Economic Security Operations on Asa, which is inspired by our lessons, and you'd look at maybe Palantir ADR - they had a couple of blogs a few years ago, they all hint at DNR function done differently, and despite some of this being from 2010s I just have not seen it spread from the really well-funded, really well, really engineering-led wave of magic wand, hire 800 engineers, not a real number, random number. I just made up and build something amazing that works very well. Yeah, and from that point I agree with Erik.

Raghu N  25:51

Yeah, so if that's the case, and if it's the case that history's, or at least kind of like the state of the nation continues to just repeat itself, why are there not more exceptions? Like, what's what? What's your perspective on why they're not more exceptions? Go on, Erik, you, you spoke first.

Erik  26:11

Yeah, I've kind of asked myself that quite a bit too. Whenever I roll into a new, a new shop, you know, where I'm taking over DNR, like I will do Atlas, enrolled in Salesforce, whatever. I asked myself, I go, why are we still doing things this way? I'm going to come in, since I'm running the show, I'm going to, I'm going to try to turn the car around and actually bring us to, you know, the kind of the nirvana of SOC, right? Modernize things, try to bring in the new best practices, try to make things so they work. It turns out that knowing that's really hard, because you're basically replacing everything, like you're tooling your processes, training things, but in order to do that, you have to get buy off from everybody above you, everybody below you, everybody besides you, that hey, we're going to reinvent this thing, that's also really hard, I mean, there are some shops that are in very forward thinking that have tons and tons of resources again, the exceptions to the rules, because when I see, like, the talk Anton gave at RSA about how they're running internally, I'm like, wow, that's a great model, right, that's Google, like they have the 1%.

Anton  27:16

Oh yeah, everybody else, like I wish I fear that we are talking about like a tiny fractions of a percent. 1% would be really not, really nice, by the way.

Erik  27:26

Yeah, I mean, it's the kind of haves and the have-nots, and everybody else, I mean, like, like even here, like we don't have the amount of money, funding, time, you know, to do this complete overhaul. So you try to make incremental recruitments where you can, or you try to fill gaps that you have with something better that you know will work better, but until the entire industry kind of shifts a different direction, which again, there's no incentives to do that right now. I mean, there's current tooling out there, as current companies, are we going to blow them all up and say, "nope, we're all going to go this direction? Right, that's hard. I mean, it's hard for companies to do internally, it'd be hard for industry to do in general, and so again, we just keep rinsing and repeating, adding band names on top, and now we're going to throw AI into the mix, which I know is going to be a spicy topic here, and hopefully that, you know, that magic fairy dust will fix all the problems, but I think, you know, I think we're both realistic here, that's not it, has its place, but it's not going to come in and fundamentally solve all these issues.

Anton  28:25

Yes, and we already saw examples of AI being added to the 1990 SOC, and the result is 1990 SOC with AI. It is not, not anything different, like not everything, like a certain, like a chain, and certain links go much faster, and maybe much better, but the whole chain is kind of largely still wherever it was. Sorry for interrupting, Erik.

Erik  28:45

No, it's to that point. I mean, we have, you know, SOCs that are vendors, like, I'm going to, you know, run 100 extra SOC or 1000 extra SOC, or whatever it is. It's like, okay, where are you actually applying that 1000 extra hundreds, whatever it is, right? And will that work with however my SOC is orchestrated today, and most of the time it's no, and then trying to figure out, like, is this actually working, being effective, because I mean the whole idea of alert fatigue is just is far more and more noise than anybody could ever look at, and so even if you could, you know, remove half of that noise, you know, by some natural AI fairy dust, the remaining half is still more than you could ever look at, and so there's, you know, the hope that you know AI go through all the things and you know find all the bad things for you in the big mess. What we're seeing that today is that's not really panning out, and for the ones that are trying to do that, it's extremely expensive. So we're kind of like, where can we apply AI in the SOC, where it makes an impact, and to in Tom's point, you don't have a 1990 SOC with AI, now you have a 2026 SOC with AI.

Anton  29:49

I mean, this is their right. I mean, I wish to, I wish I can say that things aren't as depressing as what Erik just said, but actually they're much more depressing, I. Because even that type of transformation light is not very common. I mean, we have seen this. We did a podcast a few months ago about a big bank in Europe that actually redid it. They kind of like got all the materials they put together, had some of people from Google who built DNR, Google fly there, sit with them for some number of weeks a month. I don't know exact details, and they did go through the stock transformation process, and the name is public, but I just wouldn't name it here. It's easy to find on the podcast, so but then now we have this beautiful example of a more, more like not tech company doing it, and we have this great example, and we have largely speaking one, so it proves it can be done. It proves that you can have the support from up, down, side, below, whatever regulators, and there are now more examples. There's an insurance company that's doing more of a rebuild of the SOC based on these principles, but again we are talking about 2026 based on ideas that were in the, in the, in the sort of in the ether for almost 10 years, and where people were shouting from the rooftops, at least from the Netflix original SOC-less Blog 2018 that they've done it this way, and hey, you guys can do it too, and it's taken whatever it's taken, so I feel like the progress is slow, and AI, weirdly enough, is making some people think that they don't need to rebuild, they just need to add AI. So, in that sense, AI has accelerated to tooling, but in some cases, and I mean, it's like a really odd take, and I frankly just made it up, but I think the things I've seen just converge in my mind that for some people AI has slowed down the transformation because they looked at some kind of an engineering-led detection response blueprint from Google and they say, hey, yeah, sure, Google needed to do it in mid 2010s using these principles, but hey, today we have AI, maybe we can not do this engineering-led transformation for detection response. Maybe we can't just do AI.

Erik  32:08

What becomes a big distraction?

Anton  32:11

Yes, and it becomes a distraction.

Erik  32:13

Yes, we're seeing now it's becoming a distraction, because most of those science projects are failing right at the same time. Like, I've been one of those people that's been out there publishing data and screaming from the rooftops how we need to change things, and here I am sitting in the same seat everybody else is, right? So, even those of us that have been go trying to promote this transformation, we're still stuck too.

Raghu N  32:35

So, I want to come on to one thing in a second around, because both of you have mentioned like the modern SOC and the and the AI SOC, so I want to come on to that, but there's one thing that again both of you have said, and to some extent, which is we continue to put band aids on things, right, and it's band aid after band aid after band aid, so like where does that end, because if, as you say, that like there is a very small set of exceptions of essentially organizations that are doing things well, but the majority are essentially either applying band aids and coupling those with sort of legacy processes that to me doesn't result in a good ending. So, what do you think is going to happen if we don't change the approach, if we don't change direction?

Anton  33:20

Can I start, and then hand off to Erik, and then finish. Yes, it's a little, maybe not very logical, but there was one thing that we didn't mention so far, and I think some of the hope in what you just asked, maybe in the area that some practitioners consider to be kind of hopeless, but I feel it's still kind of hopeful, namely that if you do have an MDR, a modern, well-done MDR provider that is built on those principles, and it's not - it's not an ad for any particular MDR, but, but I think the will, the wisdom to build a SOC for rent, or detection response for rent, you have more flexibility in doing from scratch in a modern way, you are not constrained by here's how our SOC was in the 80s and the 90s and 2000s and while the model of outsourcing DNR has challenges, the model itself has challenges. If you do outsource well or partner well, in my garden today, they hated the outsource term for MSSP MDR for many reasons, but if you partner well, it is possible that that type of modern SOC can be, can be rented from an MDR. And now I just want to have Erik pile snide remarks all over me, because I hinted at that, and I'm like bracing for impact and checking, of course, checking the clock.

Erik  34:38

No, I mean, I actually agree with what Anton's saying, here is that the ones that have the SOC functions that have matured, that have evolved, that have - they've been forced to figure this out - are the MSSPs, you know, the insure the CrowdStrike completes offering their SOC, you know, Sophos, any of these guys that are running these large. These large SOC functions, expel or whatnot, like, because they're being driven by profit, they're not being, they're not an internal resource, like they've had to figure out how do we do things quicker, faster, cheaper, right, at scale, because they're doing tons and tons of customers, so some of these MSSP functions, like they have very advanced SOCs, they've applied in the right places, they've sprinkled the fairy dust here and there, where they have it working well, they can produce very good outcomes at a reasonable price. At the same time, it's like on your side, like, what is the cost of that? Like, are they going to cover all your bases? Do you have to deploy their tooling into your environment? Like, are you locked into their tech stack, or whatnot, or they just taking all your signal, and so in order to get that, there is some work on your side, and again, depending on how much their technology, or how good they get their hooks into you, it may be really hard to leave later on when you hit that inflection point, where, okay, we're big enough now, where you know, or we've reached some stage where we need to bring this in house for whatever reason, and you're like, oh, that is hard. Only is it hard to get books out of us from our, from our vendor, but how do we replicate that? Because again, we haven't been doing it, so we don't have any muscles in that area. So, how do we do it ourselves? And they end up going back to the 1990s SOC model, because that's what everyone else is doing, right? I mean, they find themselves, yeah, and so it's kind of like the I must say, circular firing squad, but again, the MSSPs out there, some of them are really impressive of what they can do at the scale they go at, you know, with all that, well, all the all the noise they have to look at, and enterprise SOCs, you know, have not adapted to learn from that, because they don't have to. There's no motivation to. There's no, there's no internal forces. Again, they're not a profit center. They don't have to look at, you know, how much does it cost per alert and how much money are you making. There's no, there's no one pushing them to have to do this again. Hence, why, unless it's a one-off case, like the Bank of Europe, whatever, most of them will just continue doing what they're doing, because it's easy, or it's easier.

Anton  37:04

I feel like this is unnecessarily dark, and in many cases, what Erik pointed out, that was like that's really dark, but it's also real. Or I said, oh, actually things are much worse. So in this one case, I feel like things aren't as bad, because if you do partner well, you first you learn something, you learn how to operate with a modern MDR. You make smaller changes on your side, but still changes how to work with them, and ultimately, if you, for some reason, if they let you down, there are other modern MDRs to switch to. I feel like if you switch in from a modern MDR to a 1990s in-house SOC, that's incredibly painful, but like my advice would be don't do that, and the only appropriately dark point was that yes, some of this would be kind of obvious from the price. I still feel like this is where AI is not magic fairy dust, but it's real. Where certain well-architected platforms run by MDRs do increase the value while decreasing the price. Would they do replicate what humans do? There was one really strange example that I've heard from somebody, and it made me convinced that these modern MDR shops are a good deal for many, is that they were talking about how they observe almost like using video, video screen captures of their analysts, how humans do it, like how their best humans do it, and then these videos go into AI, and AI builds workflows and some ideas from video, literally from videos, screen videos, and I thought, oh, so as long as you curate the data, as long as these humans are really good, and as long as systems match, that's a pretty high cost, but also high value way to instruct robots to do it better in the next next wave, and of course there's a lot of caveats to this, like you need to curate, you need to then provide feedback reinforcement learning, blah blah blah, but a lot of stuff is really, I would trust MDR to do it. If my MDR is doing it, I would trust them, and if I, for some reason, they let me down, it's probably another modern MDR that does comparable things. So I am pretty positive on that line. It doesn't mean everybody go around and outsource, but it's more about there's probably a better way for some.

Raghu N  39:23

So, these videos that they're using to train them. I assume these are videos of what they're doing on screen versus kind of them having their coffee and sort of having a chit chat with their colleagues.

Anton  39:32

They, why would they were definitely screen videos? I don't think that they need to teach how humans take a break and play football or something.

Raghu N  39:39

So, let's move on, right, and I kind of, we've kind of touched on AI and bits and pieces, but not really discussed it in detail, and I want to kind of differentiate into three areas, one is kind of the impact of the SOC on essentially on their functions when monitoring kind of modern AI. AI infrastructure and AI apps, so let's start there. How has essentially the proliferation of AI, AI farms, AI applications, agentic AI, and the use of sort of just, just general sort of like chat type tools like ChatGPT, how has that impacted the role of the SOC, and just generally detection response.

Anton  40:24

Who, this is the bathtub of worms here, not a can of worms. This is a, this kind of a massive, you know, massive container of worms by gallon drum of worms.  

Raghu N  40:36

I'm going to pick on one of you two, if you got a door.

Anton  40:39

Yeah, we, I'm squirming a little bit, I kind of have pieces of an answer, but I probably don't have like a beautiful framework answer. Well, actually, maybe I do, so I'm fine. I'll venture in. So it does come up, and I spend my morning talking to somebody about largely this very topic, about like DNR for AI applications, and oddly enough, the very first thing that my kind of thing answer that I would give people is that if they double down on like the AI specific bits like model adversarial testing, how do I see what model is thinking? There was a startup at RSA trying to mind read the AI model. People who do that tend to make some kind of a really bad mistake on infrastructure side, like they leave s buckets open with all the AI data or something, and ultimately the AI app is owned through 2007 SQL injection or 1980s password fixed password or something, and then people who kind of double down on infrastructure and say we're going to deploy it into really secure environment, and they think it's all the same. These people get owned through prompt injection and something specific, so it's almost a hard must that you think about the model, about infrastructure, about the app, about the data. We had this sort of a four layer stack here that we use for some discussion: model application infrastructure data has four elements where you have to observe what AI is doing, and of course other security is applied to the four layers, and people who over pivot to one layer typically suffer to pay the consequences. This is not a detailed answer, how to monitor AI, but it's more of a very coarse-grained where I start thinking, like, do I need to do, do I need to monitor traffic flow into your API endpoints? Yes, of course. Do I need to make sure my authentication is done well? Yes. Do I need to also have some kind of in-model stuff observability? Yes, but it ends up being on the list, but it isn't the only thing, and it's probably isn't even the first thing. I mean, realistically, you probably would suffer from something in from the 90s, 2000s first, and then from AI-specific issues.

Erik  42:52

Yeah, I mean, it's it, it's one of those things for, for DNR or SOC teams, like, you know, how are we, you know, how are we reacting to all this, all this stuff out there, and it's, it's not the first time we've seen, you know, this big hype thing, or we had the AI hype around UBA, UBA, you know, what decade ago, whatever it was, and everyone was melting down about it. We've had other giant hype cycles too that have popped up, and from a pragmatic security point of view, like we kind of sit and wait, and kind of wait for the dust to settle, and you know, by then we figure out what use cases have they applied this to that's actually going to survive and continue to thrive, and which ones are going to die off, and then we can start kind of focusing on, okay, the ones that are going to work, okay, how do we secure those things, you know, from a particular use case point of view, you know, like today everyone's trying to, you know, throw spaghetti at the wall and see what sticks, and they're applying AI to everything, giving everybody access to all the things. You know, begin from a straight point of view, it's like, okay, there's no way we're going to be able to watch everything. So, what do you do? You become a little bit proactive, and you put some guardrails in place, and you're hoping you're staying within the guardrails, and of course they won't. We have Shadow IT, and just like we have Shadow AI today, and you're trying to root that out, except we try to root out, you know, Shadow IT the last decade. It's really hard problem at the same time, like, because this is so new, there's not a lot of tooling out that will solve this for you, I can't go buy an off-the-shelf product today that's going to secure all my AI usage for the company, or have observability, or whatnot. And so, again, we're trying to put guardrails in up front to try to meter things a little bit to make sure people don't go off-roading, so we don't get hit by again the 1980 exploit of a fixed password, or whatever, just some basic sanity checking controls in place, understanding that this is going to be the wild west for a minute, you probably, until the hype cycle kind of dies down, until the things that aren't going to stick die out, and the things that are going to stick start to take off, or we have kind of a more clear picture, it's like an area is a cloud, you know, that clouded all the things and secure. Was an afterthought. It kind of came along, catching up later on. We were very reactive at first, as things happened, we reacted to them, and finally we started getting out in front of them. And the AI is going to be the same thing, at least from a DNR security operations point of perspective, where we're going to be very reactive, and that's what we are today. Like, we have people coming and telling on themselves, you know, oh, I just download the new open browser for some AI company.  

Anton  45:24

I installment company credentials on it, which kind of happened to somebody a few days back, right? As far as I recall.

Erik  45:30

Then they come telling themselves, ask why it's not working, and so you know between that and some guardrails, you know, policies, what you can do from a pragmatic point of view. The rest of it's going to be us responding to when people do go off roading or you know crash the car or whatnot with the applications until the dust settles until we have kind of a clear sense of direction of what's going to actually stick and what's going to be the real use cases for this and then when you start wrapping your hands on you know how do we govern and secure those whatever's left right but yeah it's it's very active today it's very just clicking and putting in guardrails and trying to keep all the herd of cats going in the same direction, but it's the wild blast. Just say the early days in the internet, early data cloud security is going to be very reaction or very, very reactionable or reaction to these things until the dust settles,

Anton  46:19

and frankly, we still see this with cloud, admittedly, like we literally see. I mean, I probably told that story too many times, so if somebody's listening to this and they've heard me tell this, I'm going to be embarrassed, but I'll tell it anyway. Was I was talking to a CSO of some company, and he was complaining how this digital disruption and random rapid acceleration is so annoying, and I open my mouth to say, "Oh, yeah, dude, I have full compassion to you. I know AI is hard. Before I managed to breathe in and say that, he said, "Yeah, Anton, you know cloud is hard, and I realized that he is in the previous disruption, so he first encountered cloud last year, and he is really struggling, and this is very stressful for him and his company, and this is very new and disruptive, and it just isn't the wave that's hitting other people, it's the wave that hit other people 10 years ago, and it hit him now, and then we had a really fruitful chat about how he's going to be hit by two waves at the same time, cloud and AI, and it's going to be really fun, not for him, but this reminds me that a lot of this disruption stuff is is not going to wait for you, and while you're not going to die, you would go through a very stressful period of time if you don't, you know, prepare yourself until there are many breaches all over the place, and people who struggle with cloud now are going to start struggling with cloud and AI at any moment. If they refuse, then Shadow AI would come, and they would still struggle with AI, even if their companies aren't adopting, their employees are so AI struggles would start, even though cloud struggles haven't finished for many of them,

Raghu N  48:01

so I'd like to ask you about this, right, because I know you've kind of written about this as well, about sort of drawing parallels between sort of early mistakes with cloud securing AI, and because when I look at this, and I remember sort of being like at the start of the whole, the start of the cloud security discipline, right, when I look back I kind of I look at it and think actually one of the reasons we made cloud security such a big thing, and as if, like, it was this completely new discipline that no one ever thought of, was that we actually looked at the problem, I think, in a way that was wrong, because really, that when I look at cloud security compared to data set security, I think of all the same fundamentals, except for the fact that a misconfiguration in cloud is potentially you're exposed to the internet, right, and that's why kind of it's sort of the criticality of it sort of just goes up a notch. Are we making sort of the same, are we making some of the same mistakes when we think about AI security, in that we are trying to invent it as a completely new discipline rather than the application of the same fundamentals, just to a maybe slightly more complex problem.

Erik  49:07

I'd say yes. I mean, if you look at any, any technology advancement, and then the last three decades, right, that what's the what's the primary focus out the gate? How fast can we get it out there? How fast can we sell it? How fast can we make money? You know, how fast can we increase the profit margins? Oh, security. Oh, yeah, they're back there, right? They're in the back seat. They'll catch up in a couple years with us. And I draw the same analogy to AI, like it's a bum rush for AI, and we're going to have to play catch up, like we are, just like we have the cloud, just like we did with everything else. So that's that's the boat I'm in. You know, I'm waiting to play catch up, waiting for the dust to settle, and when it does, we'll be there to jump in and start actually wrapping our hands around whatever's left.

Anton  49:48

Yeah, I wish I had some kind of a contrarian take that says, Erik, you're completely wrong. In reality, it's X over here.

Erik  49:57

I wish you did. Please say...

Anton  49:59

I really. He, I really don't. I feel like this, a lot of this is just going over the same exact speed bumps. I have some project when we are working on the shared responsibility of a security AI, and it's a lot of lessons do come pretty much straight from what it was in the mid 2010s With cloud, we will have all sorts of, we are people are making all sorts of classic, basic mistakes with secure AI that they've learned after cloud, but then unlearned. So, unfortunately, I'm just going to double down on what Erik said, and it would be a little boring, but also kind of sad.

Raghu N  50:35

No, no, I appreciate that. I kind of realized I was like, god, I've got so many things to ask them, I've got to, I want to ask them about anthropic mythos, and what that means for the world that we live in now, but I think that's got to all be part of, like, our sort of, I guess, next podcast, do a follow-up version, but I do want to kind of, before we, before we leave, I want both of you to give a, a short succinct answer to this question, if there was one hard truth you need everyone in this industry to acknowledge, and then go and address, what would that be? Erik, you first.

Erik  51:11

I mean, from my point of view, this is something I've been ranting and raving about for a long time. It's like there are fundamental issues that cause a lot of the security problems we have today again that just are not being addressed for various reasons, hence why we're in this constant loop of band aid after band aid after band aid after band aid, where solving the fundamental problem would remove all that completely. Again, there's various reasons for why we don't do this in different areas, but if I had a magic wand, where I could go bing and fix all these things tomorrow, I'd go after all these fundamental problems and just solve them and wipe them out, and then I'd probably be out of a job, but then I would probably on a beach someplace too. But that's my quick take.

Raghu N  51:52

Save for a while. Nice, Anton.

Anton  51:54

I like this, and I do want to double down on this, but I kind of do have my own take, kind of more specific to DNR and SOC, I feel like the elements of a blueprint that made Google, Netflix, a few other companies in the tech space successful can be selectively ported to a more normal company, and while the journey behind it is still a journey, and it is not like magic AI just does it for you, but there are enough of the more portable lessons that you can apply. Again, Erik referenced our RSA presentation about how we use agents. Not everything would apply. Some of the slides hide really costly projects that underlied certain things that produced good value for us, but they were really costly, that you probably won't replicate, and you really should not replicate, but there are enough little boxes that you can't grab and carry into your environment, and they would improve things, and I feel like the world needs to be aware, and again, we tried it with ASO a few years ago, we're going to try it again with similar models in the future with agents, but that it's almost like there is a better way for DNR, and while you don't want to, you can't replicate what we've done, you can replicate some of the ideas and be better. That's what I would stick

Raghu N  53:11

to. Awesome. Well, firstly, Anton Erik, thank you so much for bringing your expertise, your humor, your storytelling abilities, the complete package for an awesome conversation. I'm just so disappointed that we only had like an hour for this. I had so much more to ask, so we have to have both of you on back again very soon to continue this. So, again, thank you so much.

Anton  53:35

Vulnerability management, let's do one management episode. We're going to cry much harder, much bigger tears, much louder cry.

Erik  53:44

Give us two hours next time, then we'll do it two weeks, probably.

Raghu N  53:54

Awesome. Thank you so much. Thank you. Thanks for tuning in to this week's episode of The Segment for even more information and Zero Trust resources, check out our website at illumio.com You can also connect with us on LinkedIn and Twitter at Illumio, and if you like today's conversation, you can find our other episodes wherever you get your podcasts. I'm your host Raghu Nandakumara, and we'll be back soon.

‍