More than 90% of targeted attacks start with email—and these threats are always evolving. Proofpoint Targeted Attack Protection (TAP) helps you stay ahead of attackers with an innovative approach that detects, analyzes and blocks advanced threats before they reach your inbox. This includes ransomware and other advanced email threats delivered through malicious attachments and URLs. And zero-day threats, polymorphic malware, weaponized documents and phishing attacks.
Azure Sentinel provides a practical way for you to connect your data sources. Azure Sentinel comes with several connectors for Microsoft solutions available out of the box. There are three major ways to ingest data into Azure Sentinel:
- Microsoft sources
- 3rd party sources
- Custom logs
Many security technologies provide a set of APIs through which events can be retrieved. The below figure is a high-level architecture on how to ingest data from any set of APIs to Azure Sentinel. APIs can be consumed by a custom logic app connector; it supports both a push and pull implementation which is made available in Azure Log Analytics. Azure Sentinel is built on top of Azure Log Analytics making data imported to Azure Log Analytics available to Azure Sentinel.
Figure 1: High Level Process to ingest data into Azure Sentinel via custom API connector
In a previous blogpost we’ve demonstrated how to send REST API data to Azure Sentinel. In this article we will walk through the process of how to ingest logs using a custom Logic Apps workflow to retrieve REST API data. However, this article can be utilized as a step by step guide to ingest logs from any data source that provides a set of APIs.
Please reference Proofpoint’s API Documentation which detail the various API endpoints made available which can be leveraged.
Step 1: Retrieve REST API data
The first step is to retrieve REST API data from Proofpoint’s TAP service. Currently, the following event types are exposed:
- Blocked or permitted clicks to threats recognized by URL Defense
- Blocked or delivered messages that contain threats recognized by URL Defense or Attachment Defense
Step 2: Build a Custom Logic App Data Collector
Once you have the logic apps designer opened, the first step is to add a recurrence trigger, in my case I will run my logic app every hour.
Note: The recurrence you specify must be coded in the SinceSeconds parameter of the Get request
Next, we will add a new step and select the HTTP block. In the method field, select GET to request data from Proofpoint.
Notes: Proofpoint requires Basic Authentication with an API key.
By specifying “Blocked” you will fetch events for clicks to malicious URL’s blocked in the specified time period. You can fetch other events by leveraging additional endpoints available on the Proofpoint TAP API Documentation.
To send the data to the Azure Sentinel workspace we first need to parse the data. To achieve this, we leverage a Parse JSON building block:
Notes:
A query may return results that are null or be interpreted as a number instead of an integer. Adjusting the auto-generated schema to include all the possible types a query could return for that field is safer and will avoid JSON parse errors.
If you select a different end point, you will need also to change the element from the JSON you pick for pushing to Azure Log Analytics .
At this point, we are ready to send the logs to the Azure Sentinel workspace. To send the data, we will add the Azure Log Analytics Data Collector building block . Below you will specify the connection name, Workspace ID, and Workspace key associated to the desired workspace you wish to send data to.
Once you specify those details, we will leverage the “Parse JSON” output as the input and will specify a Custom Log Name i.e. Proofpoint Data as shown below:
Step 3: See the data in Log Analytics/Azure Sentinel
Azure Sentinel is built on top of Log Analytics; hence, the data is available in both portals (Log Analytics workspace blade & Azure Sentinel). After the logic app runs, the data can be found in the Azure Sentinel workspace under the Custom Logs schema.
To get to the custom log data, we first need to click on the Logs tab under General
Note: Custom Log View of ProofPointData_CL (“_CL” is added to the name provided in the above “Send Data” block) We will append “_CL” for any custom logs ingested.
Now that we can see the data in Azure Sentinel, we now can build analytic rules, hunting queries, or associate it with any other data for correlation.
In the sample hunting query below, we are looking for the Top 10 Phishing sources:
ProofPointData_CL | where phishScore_d > 80 | summarize cnt=count () by fromAddress_s | top 10 by cnt
Wrapping up
In this blogpost I’ve shown you how seamless it is to retrieve Proofpoint TAP REST API by leveraging a custom logic app. This exact same process can be leveraged for other vendors who provide APIs to their data sources for easy integration into Azure Log Analytics/ Azure Sentinel.
Reviewers:
@Nicholas DiCola (SECURITY JEDI)
@Ofer Shezaf