Setting the Stage
I have a severe case of OCD. For me, I can't stand doing anything in an inefficient way. For work, as you can imagine, it has its benefits, but there are some drawbacks. These drawbacks can affect work negatively but also can affect my personal life. My wife knows. It bothers me a lot, but even something as simple as the routine of getting out of bed and the steps I take from that point to getting my coffee and sitting down at my desk - if I find a better and more efficient way of doing it, I'll spend the whole morning fixing it. It can be debilitating.
Threat hunting is the proactive process of proactively searching for hidden cyber threats in a network. Unlike traditional security methods that rely on alerts or signatures, threat hunting involves actively looking for signs of compromise or malicious activity using various tools, techniques, and hypotheses. Threat hunting is an essential practice for enhancing the security posture and resilience of any organization. However, good threat hunting requires experienced personnel. Many Security Operation Centers (SOC) are short staffed and under skilled which leads to deprioritizing threat hunting. Copilot for Security helps immediately upskill staff by using natural language to query disparate tools in the SOC, which would otherwise require specific technical expertise.
Benefits of Threat Hunting
Threat hunting enhances cybersecurity by offering proactive defense against hidden threats, reducing the attack surface through vulnerability discovery, and improving incident response with insights into attack methods. It bolsters the overall security posture by increasing threat awareness and enabling adaptive defenses. Additionally, it aids in compliance and risk management, while also boosting stakeholder confidence through a demonstrated commitment to proactive and mature cybersecurity practices.
Threat hunting can provide many benefits for cybersecurity, such as:
- Proactive Defense: Identifies hidden threats that evade existing security solutions.
- Reduced Attack Surface: Uncovers vulnerabilities and misconfigurations for remediation.
- Improved Incident Response: Speeds up detection and response to incidents by understanding attack techniques and tactics.
- Enhanced Security Posture: Increases awareness of the threat landscape and adapts defenses accordingly.
- Compliance and Risk Management: Helps meet regulatory requirements by demonstrating proactive efforts to detect and mitigate threats.
- Stakeholder Confidence: Improving the trust of the stakeholders, customers, and partners, by demonstrating a proactive and mature approach to cybersecurity.
Challenges of Threat Hunting
Threat hunting faces challenges such as being resource-intensive, requiring costly skilled personnel and tools. The vast data to analyze can lead to overload, and the ever-evolving nature of threats necessitates constant strategy updates. A skill gap exists in the market, making it hard to find professionals for effective threat hunting. Integrating different security tools can be problematic, and measuring the effectiveness and ROI of threat hunting is difficult. Additionally, keeping pace with rapid technological advancements requires ongoing learning and adaptation.
Threat hunting is not without its challenges, such as:
- Resource Intensive: Requires skilled personnel and advanced tools, which can be costly.
- Data Overload: Massive amounts of data to monitor and analyze can be overwhelming.
- Evolving Threats: Constantly changing tactics, techniques, and procedures (TTPs) of attackers require continuous updates to threat hunting strategies.
- Skill Gap: Shortage of skilled professionals capable of conducting effective threat hunting.
- Integration Issues: Difficulty in integrating various security tools and platforms to streamline threat hunting processes.
- Measuring Effectiveness: Challenges in quantifying the success and ROI of threat hunting efforts.
- Keeping Up with Technology: Rapid advancements in technology require continuous learning and adaptation.
The Case for Efficiency in Security
Let's pause for a moment and ask the correct questions to establish the context. What is the reason for efficiency in security? Will efficiency weaken security or enhance it? Have you considered these questions? Are you as troubled about it as I am? And is there really a solution that can fulfill the promise of better security through improved efficiency without any compromise?
I think we can agree that we need to develop a modern response to security threats based on efficiency. As a security analyst or security team, when you are actively hunting or investigating threats in your environment – you are on the clock. Things must happen quickly. You must get to the point of remediation as soon as possible because one intrusion could lead to the next. One compromised account could lead to another – and like a pandemic, exposure can spread uncontrollably unless contained. You’ve probably heard the phrase “superspreader event” in relation to pandemics. If you don’t control an identified threat quickly, it can become a superspreader in a matter of minutes. And, based on the metrics for so many security teams, Mean Time to Acknowledge (MMTA) and Mean Time to Response (MTTR) are critical to evaluating incident management performance and to identify areas for improvement.
Based on surveys and reports, it’s been determined that the average company – listen to this – needs 162 hours to detect, triage, and contain a breach. To me, knowing what I know about Microsoft’s own tools and how efficiency is designed into them, that’s a ridiculous number. But it still rings true for a lot of organizations.
Let’s analyze this number more closely, so we can comprehend what it really means.
The average organization takes…
- 120 hours to detect an attack.
- 5 hours to triage.
- 6 hours to investigate.
- 31 hours to contain.
Total = 162 hours per incident!
Security Efficiency Goals
If you’ve worked in security for any length of time you have probably heard of the 1-10-60 Rule for cybersecurity, but if not…
About a decade ago, the 1-10-60 rule was proposed as a goal. Not originally a hard-and-fast, set-in-stone rule, but otherwise a goal that we should be able to attain sometime in the future when the tools and technologies have improved enough to get us there. Or, when technology has caught up with our intent.
The idea is that the most cyber-prepared organizations should aim to detect an intrusion in under a minute, perform a full investigation in under 10 minutes, and eradicate the adversary from the environment in under an hour in order to effectively combat sophisticated cyber threats.
Hence: 1-10-60
That’s quite a difference. 162 hours versus 1 hour and 11 minutes? If that’s our goal, we’ve been way off for far too long.
Making it Real
My approach to discussion topics like this one is always based on working with customers. I glean a lot of knowledge working with a large number of customers and based on – admittedly – my OCD, I want to solve their complaints, particularly when it comes to the topic of efficiency. It literally pains me when things aren’t done from an efficient standpoint and I MUST FIX THEM.
These (the following list) are some of the complaints I’ve heard and captured from working with our customers. This list was developed based on their knowledge of working within the confines of their legacy tools. Most organizations have security tools have been historically on-premises software and services and are a mixture of applications that don’t talk to each other.
The list:
- Preparedness and hunting are too time consuming.
- Continual Tuning and Evolution of SOC is difficult.
- Adding Investigative Context Requires Rare Skillsets and Manual Interaction with External APIs.
- Locating and Maintaining Trusted Threat Intelligence Sources.
Bottom line: Too much time spent on inefficient processes.
Making it Unreal but Relatable
You might think – hey, let’s just forget about this manual stuff. Let’s let it slide for a while. Or, like some customers have determined as I stated prior (incorrectly, I might add), let’s just stop doing it.
Here’s a great example of this preparedness exercise…
I love Sci-Fi. Michael Crichton is one of the best authors in this genre. He wrote famous stories like Jurassic Park, Westworld, Congo, Timeline, Andromeda Strain, and many more. But the story I like most from Crichton is Sphere. They made a movie out of it with big Hollywood actors like Dustin Hoffman, Sharon Stone, Samuel L. Jackson, and others, but of course the story - the book - is better than the movie.
Here’s how this breaks down…
The government hired a scientist (Dustin Hoffman played the character in the movie) to design a simulation and write a report. The simulation was meant to show what the government should do if we ever encountered aliens or alien technology. The scientist wrote the report and gave it to them - never thinking that the simulation would be useful. He didn't even believe in aliens himself. He thought he was simply making up a story and getting paid with taxpayer money. But guess what??? It turned out that the Navy was exploring the ocean depths and found alien technology. They wondered - OK what do we do? Oh, yeah...we have instructions for this. So, they dug out that old, hidden, dusty file from whatever neglected storeroom it was in and started following the steps that were written. Suddenly, those suggested as possible team members - based on their different skills - were pulled out of normal life, put on this team, and sent to the bottom of the ocean. Of course, if you remember, the story didn't end well. If this is your first time hearing this story, it's a great book - you should read it.
But here’s the lesson: They had a plan. They weren’t caught completely unaware. They – were – (sorta) prepared.
Our preparedness exercise for security is, thankfully, a bit simpler these days. Still important. And it is still critical to security operations. It’s necessary.
Applying Process
The first thing we do is develop a “theory” of what to hunt for. General vertical and targeted company threat intelligence typically influences the theory. It might also be driven from recent information from the information security community, partners, or vendors.
And, ultimately, based on our research and what we learn, we need to answer these questions:
- Does it exist? (Are we under active attack?)
- Where does it exist? (I need to locate the occurrence in my data – and I need to do it quickly.)
- Why does it exist? (Even today, our biggest security threat remains – sadly enough – end-user habits. We can win awards and kudos for securing our devices and software and services – but something a single end-user does can throw a wrench into all of that.)
- And, then ultimately, how – of course – will we react? (Very similar to our Sphere story, i.e, what approach will we take?)
Making it Real Again
Here’s a good real-world example of exactly what I’m talking about. Granted, it’s not malware, or ransomware – but it’s guidance from Microsoft on how to stay compliant against a potential threat.
A couple years back, Microsoft released guidance about a series of stage updates to solve an unsecure protocol. We could’ve released a massive update that would solve it immediately, but doing so wouldn’t give customers and partners enough time to get their software and services in a supportable, compatible state. Releasing the full update would break things and people would hate us. Despite what you might think – we care whether people like us or not.
But guess what? Immediately after this announcement a zero-day (called ZeroLogon) was reported that took advantage of this flaw. If the customer had installed the initial update from, they would be fine – but delaying updating the systems – which a lot of customers still do – would open their environment up to compromise.
So, as part of this preparedness exercise, using Microsoft as a trusted source, I used the guidance and indicators provided to work with customers to develop a way in Microsoft Sentinel to monitor for compliant and non-compliant systems.
//Choose which to track (compliance or non-compliance) and remove the comment
//Based on https://support.microsoft.com/en-us/help/4557222/how-to-manage-the-changes-in-netlogon-secure-channel-connections-assoc
SecurityEvent
| join Heartbeat on Computer
//| where EventID == "5829" //Tracking NetLogon Non-Compliance
//| where EventID == "5827" or EventID == "5828" //Tracking NetLogon Compliance
| distinct Computer, OSType, OSMajorVersion, Version
Copilot for Security as a Threat Hunting Tool (or let’s fix my OCD)
Our old methods involved sourcing knowledge for our theory, then generating the correct queries and running them, and then building the response. Our response could be no response at all because the theory (fortunately), didn’t pan out. But no response is still a response that takes active time.
One of the ways to overcome the challenges and improve the threat hunting process is to use Copilot for Security, an artificial intelligence platform that aims to change the way security is done. It addresses the issue of tool fragmentation in the SOC by providing a natural language interface that can reason across an infinite number of first and third party tools. For example, Copilot for Security can leverage data from Microsoft Defender Threat Intelligence (MDTI), Microsoft Sentinel, and ServiceNow, to name a few.
MDTI is a comprehensive platform designed to enhance cybersecurity operations. It streamlines processes such as triage, incident response, threat hunting, vulnerability management, and cyber threat intelligence analysis. MDTI helps security professionals by aggregating and enriching critical data sources, providing an innovative interface for correlating indicators of compromise (IOCs) with vulnerabilities and cyber threats.
The platform is built upon a vast repository of threat intelligence, which is derived from over 65 trillion signals and the expertise of more than 10,000 multidisciplinary security experts worldwide. The Microsoft Threat Intelligence Center (MSTIC) team is a group of experts, security researchers, analysts, and threat hunters at Microsoft. MSTIC tracks over 70 code-named government-sponsored threat groups, including Russian hackers (code-named Strontium), North Korean hackers (code-named Zinc), and Iranian hackers (code-named Holmium)1. Their mission is to stay ahead of threat actors and defend against cyberattacks.
This work and intelligence enable security teams to identify vulnerabilities more effectively and stay ahead of cyber threats. Additionally, Microsoft MDTI integrates with Microsoft Copilot for Security, allowing the use of natural language queries to summarize investigations and explore built-in threat intelligence.
Microsoft Security Copilot can significantly enhance the efficiency of threat hunting in several ways:
- Assists in building hunting theories through its ability to reason over MDTI. Copilot for Security provides expertise at the user’s fingertips to quickly build impactful hunting theories.
- Improves the speed and efficiency for security teams, allowing them to respond to threats faster. In a randomized control trial, using Copilot for Security improved security response time by up to 26%.
- Enhances the quality of responses through the connection of endpoints, cloud services, and external threat intelligence feeds, and enriching it with contextual and behavioral information. Security novices and analysts using Copilot for Security performed better and expressed more confidence in their work compared to those who didn't use it. A high percentage of participants noted that it helped improve the quality of their work.
- Reduces resistance or skepticism by providing clear and actionable insights into the value and impact of threat hunting, such as the number and severity of threats detected, and the risk reduction achieved.
- Offers considerable time savings in capturing and consolidating attack data, reducing the time spent on these tasks by 90%, as reported by a Principal Research Lead for Microsoft Defender Experts. 1
- Enables organizations to add threat hunting to their security processes by offering a user-friendly and intuitive interface that guides security analysts through the different stages of threat hunting, from defining hypotheses, to querying and analyzing data, to validating threats.
- Empowers security teams to perform proactive threat hunting by using artificial intelligence and automation to generate smart hypotheses, suggest relevant queries, prioritize and score threats, and recommend remediation actions.
- Alleviating tedious tasks and it empowers senior staff to focus on strategic priorities and strengthens the expertise of junior staff through step-by-step guidance.
- Quickly upskill junior staff on otherwise technically involved processes such as building KQL queries.
Copilot for Security can help organizations to transform their security posture from reactive to proactive, and to achieve higher levels of security maturity and resilience. Organizations can benefit from the advantages of threat hunting without the drawbacks, and gain more visibility, control, and confidence over their network and system security. Overall, Microsoft Copilot for Security represents a leap forward in the realm of cybersecurity, enabling teams to protect their systems with greater precision and at machine speed.
Bottom line: Copilot for Security is the next level in the ongoing story to resolve efficiency in security and help eliminate my OCD in this area.
TLDR
Threat hunting is the process of proactively searching for hidden cyber threats that evade traditional security solutions. It is a vital activity for any organization that wants to protect its data and assets from sophisticated attackers. However, threat hunting is also a challenging and time-consuming task that requires a high level of skill and expertise. Many security teams struggle with the problem of inefficiency in threat hunting, which can result in missed or delayed detection of threats, wasted resources, and increased risk.
Copilot for Security is the next level in the ongoing story to resolve efficiency in security. It is a solution that can help organizations overcome the challenges of threat hunting and achieve better security outcomes.
EXTRA: Learning to develop good prompts is also a very important aspect of building better efficiency using Copilot for Security. See: Get the most out of Microsoft Copilot for Security with good prompt engineering
1 Microsoft Copilot for Security provides immediate impact for the Microsoft Defender Experts team https://www.microsoft.com/en-us/security/blog/2024/02/08/microsoft-copilot-for-security-provides-imm...