Streamline threat detection and response across diverse environments with Microsoft Sentinel, your cloud-native SIEM solution. With features like Auxiliary logs for low-cost storage and proactive data optimization recommendations, you can efficiently manage high volumes of security data without compromising on threat intelligence. Leverage built-in AI and automation to uncover hidden threats and reduce investigation time from days to minutes.
Rob Lefferts, CVP for Security Solutions at Microsoft, joins Jeremy Chapman to show how to migrate from existing SIEM solutions with built-in migration tools, ensuring seamless access to your security logs while maintaining investigative integrity.
Utilize analytics and auxiliary logs.
Use Analytics logs for high-value data with 90-day retention and Auxiliary logs for low-cost, long-term storage of high-volume data. See your storage options with Microsoft Sentinel.
Leverage automation to streamline threat detection.
Simplify your SIEM migration.
Import your logs as JSON, automatically map schemas, and translate rules from systems like Splunk — all in minutes. Get started.
Watch our video here:
QUICK LINKS:
00:00 — Microsoft Sentinel, modern Cloud SIEM
01:12 — Unified security operations platform
02:55 — Prioritize security updates
04:27 — Storage options
05:11 — Optimize data coverage and usage
06:17 — Protect against long-term persistent attacks
07:58 — Automation using auxiliary logs
08:59 — Manual effort
10:10 — Automation
12:07 — Migration
13:31 — Wrap up
Link References
Get started at https://aka.ms/MicrosoftSentinel
Find samples for the Playbook Logic App and the Function app at https://aka.ms/AuxLogsTIapp
Unfamiliar with Microsoft Mechanics?
As Microsoft’s official video series for IT, you can watch and share valuable content and demos of current and upcoming tech from the people who build it at Microsoft.
- Subscribe to our YouTube: https://www.youtube.com/c/MicrosoftMechanicsSeries
- Talk with other IT Pros, join us on the Microsoft Tech Community: https://techcommunity.microsoft.com/t5/microsoft-mechanics-blog/bg-p/MicrosoftMechanicsBlog
- Watch or listen from anywhere, subscribe to our podcast: https://microsoftmechanics.libsyn.com/podcast
Keep getting this insider knowledge, join us on social:
- Follow us on Twitter: https://twitter.com/MSFTMechanics
- Share knowledge on LinkedIn: https://www.linkedin.com/company/microsoft-mechanics/
- Enjoy us on Instagram: https://www.instagram.com/msftmechanics/
- Loosen up with us on TikTok: https://www.tiktok.com/@msftmechanics
Video Transcript:
- The mechanisms you need to put in place to detect and respond to threats as an enterprise depend on how well you’re able to look across hundreds of thousands of devices, connected cloud services, on-prem systems, and identity providers at scale and correlate those signals happening across your organization to detect and pinpoint security events. And today, we’ll take a deeper look at how Microsoft Sentinel, our modern cloud SIEM, does just that. It’s designed to handle data ingestion at scale from your connected systems to identify security threats. And this now includes a new low-cost option for auxiliary logs to bring in previously unused high volume data from, for example, your firewall, which when combined with threat intelligence and automation, along with Security Copilot, can provide vital additional insights to discover and stop persistent attacks hidden in your network. And keep watching to the end because if you’re looking to migrate from your current SIEM solution, we’ll show you how we’ve made that simpler, too. And joining me today to go deeper on Microsoft Sentinel and our modern cloud-based SIEM is Security CVP Rob Lefferts. Welcome back to the show.
- Thanks, Jeremy. So happy to be back.
- Thanks so much for joining us today. You know, it’s been a while since we’ve actually taken a singular look at Microsoft Sentinel, so how should we think about this as part of our unified security operation solutions?
- Well, the SIEM is such a critical part of the SOC toolset. It’s like the central nervous system for all of the security data that comes in. In fact, let me show you what I mean. Here I have Microsoft Sentinel open with a view of all the connectors attached to this environment. Everything from multi-cloud services, firewalls, device sensors, identity and authentication systems, SaaS apps, and more. In fact, one huge advantage of Sentinel is the breadth of data it allows you to bring in from multiple cloud services and platforms, with hundreds of available connectors. Now, even though the data that comes in is often text-based and each unit of information might just be a few bytes in size, that all adds up and can quickly amount to terabytes of data. For example, if I drill into this connector alone, there are millions of events that have been recorded. Of course, in each data type and log that you see connected, there are often just a few properties available to thread all connected activities together. Like IP and host MAC addresses, URLs, processes, and usernames to name just a few. This information all needs to be mapped against intelligence and corresponding properties known about the latest threat actors and their tactics. And this is where Microsoft Sentinel shines. It detects threats by logging and finding the connections across all integrated systems, users, devices, services, and corresponding activities across your organization, and then helps you to investigate and visualize exactly what has happened. This is something that would be impossible to do manually at scale. You need a cloud native SIEM with built-in AI to do this.
- That’s right, and because threat actors, their tactics and, you know, the things they’re doing are constantly changing, it’s one of the reasons why I think everybody from an IT perspective, we’re always prioritizing getting those security updates and patches done as soon as possible, and really plugging those vulnerabilities and making sure that we log the related events.
- Yeah, it’s critical that you push those updates out when you’re closing known new vulnerabilities. Where a threat actor might breach your defenses and then try to get what they can before they’re detected, that’s like a quick grab and go theft. That said, another common tactic, and something we’re seeing much more, is well-funded state actors who will breach security systems and then hide in the network, waiting for the right time, that could be days or weeks, in what’s called a persistent threat, as they hide under the radar and slowly explore the network.
- Right, and it’s kind of like those movies we’ve all seen where you’ve got a government spy who might be sitting years, embedded into a country, and then waiting for that precise moment to attack.
- That’s right. And remember how I said that the SIEM makes sense of terabytes of lightly connected data? Well, these systems are designed to hold only so much information, often covering just a finite amount of time depending on the data type. So that persistent threat, our embedded spy, once they’ve breached our defenses, can often just wait until their initial entry and tracks have been deleted, which could be as little as a couple of weeks.
- That’s right, and the thing is here, you know, if they go undetected, there’s a good chance that that little hole they used initially, maybe weeks ago, to gain access, is still open and vulnerable. So, what are we doing about this?
- Well, it’s easy. It’s like science fiction. We’re giving you the ability to go further back in time, and without breaking the bank for storage. There are a couple of options, in fact, for storing the information logged from your systems. Analytics logs, which is where you spend the most time, and it brings in information from antivirus, authentications, threat intelligence and alerts. These stay active and in hot storage for 90 days. And a new option, auxiliary logs, for low cost, long-term storage, for high volume and low value data, like your firewall, which is available for up to 30 days.
- And selecting the right storage options, that’s going to help you ensure that you have all the data that you need, and now you can do that while keeping costs in check and without losing the ability to query.
- You can. And we go a step further in Microsoft Sentinel. In fact, we want to give you ways to save your money on things like storage so that you can prioritize your money for better protection, by giving you proactive recommendations to optimize both data coverage and usage to ensure you have what you need in place to detect and investigate threats. You can see how useful each data type is, and where you can save costs and make better use of data. For example, this is a business email compromise detection to find and contain phishing attacks. It has broad coverage for initial access and credential access, and there are others for adversary in the middle, credential harvest, human operated ransomware, and more.
- These are really great options, you know, for SecOps, and the extra data that you get with auxiliary logs is going to help you with those persistent attacks, like you mentioned, Rob, and because attackers won’t necessarily be able to outwait that lifecycle as they start to move laterally and trigger detections.
- That’s right. And it also makes it possible to trace these events all the way back to the initial breach. In fact, let me show you a long-term persistent attack. This is a multi-stage incident involving execution and lateral movement with ransomware on multiple endpoints. It’s really not good. In our case, thankfully, we can see that this attack was automatically disrupted, meaning that the attack has been contained. And in Microsoft Sentinel, as I dig in, down here in the tags, we see a couple of threat actors, Sangria Tempest and Storm 1113. This information was brought in using auxiliary logs together with automation that we built. These were high volume firewall logs that would normally not make it into analytics logs in order to control cost. And this is where auxiliary logs offers an affordable option, removing the barrier to bring this type of data in. And I’ll show you what we did in a moment. Now, I’ll introduce each threat actor in the activity log when we get there. Both are bad news, but when they work together, it’s even worse. Now let’s dig into the full details. The first thing I want to take a look at is the activity log. Here, we brought in more information directly from threat intelligence using lookups with host pair IP addresses. We can see that Sangria Tempest specializes in human operated ransomware attacks. We can see their tactics, techniques, and procedures, or TTPs. And if I scroll down, I can see Storm 1113 specializes in distributing malicious packages and payloads.
- So it sounds like Sangria Tempest, they might have built the back door and kind of the ransomware in the software itself. Then they worked with Storm 1113 to kind of get that package out and distribute it out to the people they wanted to attack.
- That sounds about right. It’s an effective collaboration, and not in a good way. So let’s dig into what the automation using auxiliary logs has uncovered. With all of these attempts, I can see that they’ve been trying to work their way into our environment several times, and also leaving several clues, with IPs and URLs, like these two here, from our threat intelligence mapping, with network patterns from our firewall logs. And here, we’ve used our auxiliary logs to find matching traffic from our firewall logs, which are usually too expensive and too vast to keep as analytics logs. I have one of these already set up for anomalous network patterns, and when I select it, you’ll see a summary query, and if I move over to Edit, I can edit the query text from here. You’ll also see that it’s set up to run every 20 minutes automatically. I can even preview the results right from here. That takes a second to run, and you see the details from the traffic logs. Now we have the information to start piecing everything together, which is kind of like finding a needle in a haystack, and we have found that needle. But before we move forward, let’s look at the effort that it would have taken. You’ll remember our suspicious URL called photoshop.adobe.shop. I’m going to copy that into my clipboard. And just to show you the connection, let me head over to Intel Explorer and Threat Intelligence. I’ll search for this URL, and we’ll see it has a high severity, was used by Storm 1113, and it resolves to this IP address, 173.255.204.62, that we saw before. Now I’ll copy that and go back into Intel Explorer and paste in the IP. And now I’ll go to the Resolutions tab and search for any matches. And I don’t see the Photoshop URL near the top of this list, but I do see another URL from our investigation, workable.uk.com. When I click into that and then go to the Summary tab, you might’ve expected Storm 1113 because we know that they’re linked to this IP, but it’s actually Sangria Tempest, our other threat actor, and they were using this IP address at the same time for around 10 days. So now we know for sure they are working together.
- And really, as you’ve shown, finding something like this manually would’ve taken a long time, so what do you do to solve for this?
- One word, automation. Otherwise you’re looking into potentially hundreds of URLs and IPs manually. So, back in Sentinel, I’m in the Automation blade and in the Active Playbooks tab, and we created a playbook called MDTI-Actor-Lookup. When I click into the Edit view, you’ll see that this is a logic app, and that we’ll pass through a function app to parse through our incident details and perform thousands of actions in a short period of time. You can see that it’s set up to look through IPs and URLs sourced from our auxiliary logs, and look them up against threat intelligence data, and when it finds matches, it adds the information to the activity log, then adds corresponding tags like we just saw. To do something that might have taken weeks now happens in just a few minutes. And we used Security Copilot here, too, with a prompt to summarize the writeups that I showed earlier about the matching threat actors, as well as update our incident with that information. Finally, to run all of this logic on a schedule, we built a function app to run this asynchronously on a schedule to collect the results of our logic app, and can process about 1,200 results in around 20 seconds. So it speeds things up exponentially. What would’ve taken days before, to do investigations and connect the right dots, has been reduced to just a couple of minutes by combining Copilot with our threat intelligence playbook. And if you want to test out this automation for yourself, you can find the samples for the playbook logic app and our function app at aka.ms/auxlogstiapp.
- Right, and like I said, the point of automation really is to make sure that things are a huge time saver that you program in, and also more accurate. And you can start with basic rules as you query your data, then use that data behind with more sophisticated logic. And this is really going to take the EDR solution to the next level, with more information and automation. So, anyone who’s watching right now and they’re using their own SIEM, how easy is it then to move from what they’ve got now to Microsoft Sentinel?
- You know, we’re getting this question a lot recently, and the good news is that we’ve made the migration simple to bring over your logs. Back in Sentinel, from the content hub, you just go to the SIEM Migration Wizard. It explains the prerequisites for analytic rule dependencies, data availability, and access requirements. Next, we give you the query you’ll need to run in order to migrate your content as JSON. I’ve got one of these ready to go, so I’ll drag in my JSON file. From there, the schema is automatically mapped, and you can see the mapping success column, and optionally, you can make changes to the mapping. Here, for example, we see a registry path without a match. This isn’t critical, so I’ll save changes and continue to rule configuration. Here too, we’ve translated the rules from your Splunk output to work with Sentinel. You can see how many are fully translated versus partially translated, and you can edit from here, or do that later in Sentinel. For now, I’ll just save changes. Then, I’m ready to review and migrate. And I’ll confirm. That takes a moment to run, but we’ll speed it up to save time. And once it’s complete, in Sentinel Analytics, you’ll see all of the new rules that we brought over from Splunk, and we just need to check and enable the rules, and that’s it.
- So now there are really no trade-offs in terms of your SIEM and the data that’s available to you in order to investigate attacks.
- And as we showed, you can leverage automation for detecting and responding to ongoing attacks. There’s easy access to insights, from extensive threat intelligence at your fingertips, GenAI is integrated into your automation flow with Security Copilot, all efficiently saving you more time and keeping you ahead of threats. And there’s a lot more to come, so watch this space.
- And I’m looking forward to seeing what’s next for Microsoft Sentinel, so for anyone who’s watching, what should they do to get started?
- Easy. You can learn more and get started at aka.ms/MicrosoftSentinel.
- It’s always fascinating having you on to really break down the latest threat tactics along with the detection response updates that we’re making, and we’ll keep bringing you updates around our XDR solutions, so be sure to subscribe to see them in action, and as always, thanks for joining us today, and goodbye for now.
