Introduction
The modern enterprise, of any size, faces a complex and dangerous threat landscape. Compliance risk and security threats, both internal and external, have to be managed with a dizzying array of technologies, processes, and subject matter experts. The security industry and every major cloud service provider will tell you that end user phishing is the most common and most successful breach vector for any modern organization. Microsoft 365 Security, Compliance, Identity, and Management (M365 SCIM) provides an integrated, holistic solution to these existential risks, including phish prevention. In this blog, we want to share our best practices for creating an effective and actionable end user phishing training program.
Every organization should start with a robust technical solution to eliminate the vast majority of phishing attempts on your organization. Microsoft Defender for Office 365 is an excellent option. The organization should then focus on creating a robust and capable security operations and administration team that can react and respond to successful phishing attacks, which will inevitably happen. You should hire good security and compliance operators, and then equip them with the right tools and resources to manage those risks.
Finally, we believe every organization should have a data-driven phish training and behavior modification program in place to help their employees first understand how to correctly identify phishing attempts and to help them take the correct action when those threats are identified. Below, we lay out our recommendations for appropriate program goals, resources, simulation construction, training, and measuring success.
Goals of the Program
Phish training programs should have at least two main program goals:
- Measure and track the organization's susceptibility to phishing. This goal, while simple sounding, is quite complex to achieve. Most organizations use a click-through rate on phish simulation exercises across the organization to determine their overall susceptibility, and then adopt target click-through rates on simulations over time. For example, an org might run a simulation against all users at the beginning of the fiscal year, and measure that 21% of their users clicked the link of the credential harvesting phish simulation payload, and 15% actually gave up their credentials. They would then set their baseline compromise rate at 15% with a target to reduce it to 12% by the end of the fiscal year. This captures the extent to which the organization thinks users will do a 'bad' behavior in a phishing attempt. Good programs will go even further to measure whether the end user will do a 'good' behavior by reporting the phish message to security operations.
There are several problems with only using static click through rates to measure susceptibility. Phish clicking susceptibility depends, profoundly, on the quality of the payload being used. Attackers are known to use very cheap, generic payloads targeted at any user and can achieve high success rates. Specially crafted, high-targeted payloads directed towards high-value individuals are very difficult to detect, and have scary-high success rates. The best phish simulation solutions will leverage real-world payloads for their simulations, but program administrators are sophisticated enough to make an educated guess at whether any given simulation payload will have a higher or lower click-through rate based on its 'complexity' or 'difficulty'. This means that payload quality is the primary driver of your click through metric, and not actual end-user susceptibility.
- Click-through rates likely don't take into account the impact of any given individual being compromised, and don't take into account the coverage of your simulation metric. You might establish a cross-organization metric at annual, or semi-annual simulations, but a much smaller number of employees, who are targeted at much higher rates because of their perceived value to attackers, might be getting 10x the number of attempts. Large-scale, infrequent simulations miss out on the nuances of high risk populations to your overall susceptibility.
- There are many different kinds of phishing attacks, and if your organization is only measuring the click-through rates of credential harvesting attacks, you might find very different compromise rates for different types of phishing attacks such as malware downloads, illicit app consent grants, or drive-by phishing. Like payloads, the type of attack being leveraged can have a huge impact on the baseline click through rate.
- Phish susceptibility is driven by externalities such as attacker focus, frequency, end user behaviors required for their job function. It is important to remember that while click-through rate is a direct measure of end user behavior, that behavior isn't the only component of the overall phishing risk. If you assume that any user will eventually get tricked by a phishing attempt (which is a very safe assumption), the contours of that compromise is driven mostly by how badly the attacker wants to compromise that specific user, or whether that user's job requires them to click on a lot of links from external parties, or to open a lot of attachments sent from third parties. Your overall susceptibility is a function of both of these components.
- Educate users on how to identify and deal with phishing attempts. This goal is the more measurable and achievable one of the program. Phish simulations provide you with an opportunity to engage end users in two types of learning. First, the user is exposed to a very realistic execution of what a real-world phishing attack will look like from the end-user's perspective. Second, the user is connected to a differentiated training experience that explains how an attack works, and why it is important to prevent phishing. Most organizations have annual required corporate training topics, and so the key here is to balance the time burden of another training topic against the regular day job of its users. More on what the training experience should be like for end users below. Organizations should start with a strategy to differentiate the training in a way that connects directly to the users' knowledge and experience. Tracking completion rates, frequency, and productivity impact is a straightforward measure from there.
Resources and Dependencies
Phish training programs are a key element of any organizational strategy to address behavioral risk, but since the core mechanism behind these programs is interacting with your users in the same way that real world attackers are going to try to engage them, you should be very transparent and intentional in the creation of these programs. There are four key resources and dependencies you will need to fulfill to get your program up and running.
- Executive sponsorship. Starting at the highest levels of the organization, your CISO, CTO, and CEO should all have a clear understanding of the purpose and implementation of your program. Strategically, they should be bought into the idea that the best way to teach and learn is through authentic experiences, and in the case of phishing, that means actually trying to phish your users with real-world payloads. It is not uncommon for Boards of Directors and other oversight functions to require phish simulation programs for organizational members.
- Program owner. There are enough moving parts in creating and operating an effective phish program that having an individual responsible for the program is very common in the industry. Smaller organizations may have security admins operate the program overlapping with other duties, but if your org is mid-size or larger, we very often see dedicated people own and operate the program. Given the centrality of education to the mission, we've seen phish sim program owners align themselves with the security and compliance organization or human resources in equal measure. This person will be selecting payloads, determining simulation timing, selecting target groups, tracking metrics and results, and ensuring the end user experience is as educationally impactful as possible. Larger organizations may even employ a team of people to fulfill this mission.
- Tooling. While it is theoretically possible to perform a phishing exercise without an actual tool, or using open-source platforms, you'll need significant technical expertise and a lot of manual work to stitch together the entire picture. Our recommendation is to license and use a commercial phish simulation tool, that includes end user training, robust metrics, and a library of authentic payloads. Microsoft Defender for Office 365 has an excellent, integrated, no-config solution called Attack Simulation Training. For most of the technical requirements we lay out below, we'll assume that you are using the M365 Attack Simulation Training product.
- Security and Compliance stakeholder alignment. Your phish simulation program is doing something that most risk professionals worry about every day: exposing end users to attacker behavior. Users that don’t understand the purpose or context for these exercises can wreak havoc on normal security and compliance operations. They can cause big spikes in help desk escalations, and are fearful about why the organization is trying to catch them out by measuring their potential failure. Aligning your program goals, and mostly importantly it's execution, with your security operations is crucial for keeping everyone on the same page. Many organizations actually have their security operators perform the actual simulation creation. Ensuring your ops aren't overwhelmed with reports and escalations will maximize your results in the long run.
Once you have these four key resources and dependencies in place, you can now work through the actual execution and analysis of your program.
Stay tuned for Part 2 of this blog where we’ll cover Targeting, Frequency, Payloads, Training, Operationalization, and Measuring Success.
If you are interested in going deep to get strategies and insights about how to develop a successful security awareness training program, please join the discussion in this upcoming Security Awareness Virtual Summit on June 22nd, 2021, hosted by Terranova Security and sponsored by Microsoft. You can sign up to attend by clicking here.
.