Azure Activity Data Connector
Hi All, My organization is currently working to stand up Sentinel and we are implementing our data connectors. However, we are unable to enable the Azure Activity data connector. All policies are written correctly and should be sending to Sentinel, but it is saying not connected. Any recommendations?
Integration of Microsoft Sentinel & Microsoft TEAMS for integration of alerts
What are some of the best methods and strategies to start implementing an integration between Sentinel and TEAMS where when there are certain instances or alerts occurring, said alerts can be pinged to certain members on Microsoft TEAMS like through the use of playbooks, automations and setting up a API connection to integrate the two.Threat Intelligence Indicators in Microsoft Sentinel
Hello Microsoft Community, This is my first post and I hope it will be helpful for those who are trying to understand how the Threat Intelligence (TI) Indicators feature works on Microsoft and especially in Microsoft Sentinel. But before that, I would like to share my own experience. Working with a lot of customers I have had a big number of questions about how to automate the purging process of stale TI indicators from Sentinel. There is a way to do it manually, but if you have thousands of them, it will be a tough task to remove only one hundred at a time. I decided to automate this process and started investigating different kinds of automation (Graph API, PowerShell, etc.). After some investigation, the API command that allows to get a list of all indicators stored in Microsoft tenant was found. This one: GET https://graph.microsoft.com/beta/security/tiIndicators. And I decided to build a Logic App that will get all TI indicators, extract their IDs and then remove each of them by running DELETE https://graph.microsoft.com/beta/security/tiIndicators/{id}. Unfortunately, when I was testing it, I stuck with the situation when I was getting nothing even though I have more than one thousand indicators in my test environment. Trying to search on forums, asking questions I got no answers and decided to open a case for Microsoft Support. I really appreciate Microsoft Support team for providing a professional and fast response and explanation. Now, I will try to explain a little bit how the TI backend works on Microsoft. Let’s move to the technical part. 1. TI indicators ingestion There are a few ways to ingest TI indicators. The first one is to use a built-in TAXII connector. There are plenty of them. You can use, for example, Anomali, IBM X-Force, Pulsedive, and others. The configuration is simple, based on Microsoft you only need to get the TAXII server API Root and Collection ID, and then enable the Threat Intelligence - TAXII data connector in Microsoft Sentinel. The second way is to build a playbook that will pull TI indicators from a TI provider and push them into Sentinel Using Graph Security API. There is a great playbook for pulling TI indicators from Alien Vault: Azure-Sentinel/Playbooks/Get-AlienVault_OTX at master · Azure/Azure-Sentinel (github.com) Such kinds of playbooks require minor configuration and can be deployed from GitHub. The third way for adding TI indicators is flat file import. This feature is currently in Private Preview and will be available soon for the Public. Sentinel administrators will be able to import indicators from a CSV or JSON file. And the last way is manual creation. This is a good option only if you have a few indicators to add and have no time to write scripts and build automation. One more important thing to mention is the fact that Graph Security API serves Threat Intelligence by TenantID and AppID (the application ID that uploads the TI through GSA and was configured in Azure AD). If the TI indicators were uploaded using one application (AppID) and then queried with another application (AppID), the data will not be returned. For example, if you use the playbook mentioned above, you should Register an application in your Azure AD to ingest indicators. Then you will not be able to query those indicators with another application, for example, with Microsoft Graph Explorer. You must use the same application to get the list of indicators you uploaded. 2. TI indicators storing Based on Microsoft, when using the tiIndicators entity, you must specify the Microsoft security solution you want to utilize the indicators for via the targetProduct property and define the action (allow, block, or alert) to which the security solution should apply the indicators via the action property. In the playbook for pulling indicators from GitHub, we have the following parameter: “targetProduct” that should be “Azure Sentinel”. Yes, Azure and not Microsoft Sentinel. By setting this parameter, we configure the playbook to ingest logs into Sentinel Log Analytics Workspace and so we will be able to process the ingested indicators later. In Logs under Microsoft Sentinel, a new table is created “ThreatIntelligenceIndicator”. This is our final diagram for Microsoft Sentinel: TI indicators are not stored only in the Sentinel LAW. There are also stored in Microsoft backend with a retention period of 1 year or if deleted via the API. For Log Analytics Workspace the retention period is usually configured by a customer and data is there until deleted. 3. TI indicators pulling As well as for ingesting indicators, there are a few ways for pulling them from Microsoft backend and from Log Analytics Workspace. It was mentioned previously that to pull indicators from Microsoft Graph backend you should use Microsoft Security Graph API with the same Application and Tenant ID. Otherwise, you will get nothing. You should also pay attention to the expiration date of the ingested certificates. If you try to get a specific indicator(s) and get nothing, probably it has been expired and removed from the Graph backend. Use this resource Graph Explorer | Try Microsoft Graph APIs - Microsoft Graph to test the API. Pulling TI indicators from Sentinel Log Analytics Workspace is simpler. You need to open a Sentinel LAW and get them by running a KQL query. For example, this one: ThreatIntelligenceIndicator | project TimeGenerated, Description, IndicatorId | top 100 by IndicatorId This KQL will show you the first 100 indicators by IndicatorID. Don’t forget to set a date under “Time range”. You can also use built-in queries to protect your environment or build your own queries based on your company requirements. The last way to see TI indicators ingested into your Sentinel is by opening Threat Intelligence page in Microsoft Sentinel. This page will provide you with details for each indicator, allow you to remove them (only 100 at a time) and edit their details. Summary Microsoft's security ecosystem has a huge number of capabilities that help organizations to protect their environments from modern security threats. And TI indicators is only one piece of the puzzle called Threat Intelligence. It is important to understand how this feature works to gain the best results from it. I hope the information provided in this article will be helpful for the community and will allow to understand how the Microsoft TI works better. If you have any questions or suggestions for the text, I will be glad to hear them. There is an amazing webinar Threat Intelligence published by Microsoft Team: Cyber Threat Intelligence Demystified in Microsoft Sentinel - YouTube You can also find the article on LinkedIn: Threat Intelligence Indicators in Microsoft Sentinel | LinkedIn
Fortinet Integration with Sentinel
I am trying to integrate the Fortinet firewall to sentinel. I've followed the Data Connector page steps to set up the Linux VM by installing the CEF collector. Also, the configuration at firewall is performed. Both Linux VM(syslog) and Firewall are on Azure Cloud only. when i run the tcpdump command, I can see the logs from firewall to the Linux machine: Post locating the syslog directory, I am getting an error as below: Checking daemon incoming connection for tcp and udp This will take 60 seconds. sudo tcpdump -A -ni any port 25226 -vv Could not locate "CEF" message in tcpdump Simulating mock data which you can find in your workspace This will take 60 seconds. sudo tcpdump -A -ni any port 25226 -vv Could not locate "CEF" message in tcpdump Please make sure that traffic to the syslog daemon on port 514 and to the OMS agent on port 25226 are enabled on the internal firewall of the machine When I run the validation command given in the Data Connector page, I see the below output: [user@LinVM1 rsyslog.d]$ sudo wget -O cef_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py&&sudo python3 cef_troubleshoot.py 0b78f928-d0ee-454e-a004-10039e95fbc8 --2023-02-08 03:39:17-- https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/CEF/cef_troubleshoot.py Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.109.133, 185.199.110.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 40148 (39K) [text/plain] Saving to: ‘cef_troubleshoot.py’ cef_troubleshoot.py 100%[====================================================================================================>] 39.21K --.-KB/s in 0.002s 2023-02-08 03:39:17 (18.7 MB/s) - ‘cef_troubleshoot.py’ saved [40148/40148] Note this script should be run in elevated privileges Please validate you are sending CEF messages to agent machine. Trying to use the 'locate' command to locate omsagent Located 'omsagent' Located security_events.conf Validating /etc/opt/microsoft/omsagent/0b78f928-d0ee-454e-a004-10039e95fbc8/conf/omsagent.d/security_events.conf content. Current content of the daemon configuration is: <source> type syslog port 25226 bind 127.0.0.1 protocol_type tcp tag oms.security format /(?<time>(?:\w+ +){2,3}(?:\d+:){2}\d+|\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.[\w\-\:\+]{3,12}):?\s*(?:(?<host>[^: ]+) ?:?)?\s*(?<ident>.*CEF.+?(?=0\|)|%ASA[0-9\-]{8,10})\s*:?(?<message>0\|.*|.*)/ <parse> message_format auto </parse> </source> <filter oms.security.**> type filter_syslog_security </filter> Omsagent event configuration content is valid File permissions valid omsagent security configuration supports Cisco ASA parsing OMS Agent syslog field mapping is correct Checking if firewalld is installed. systemctl status firewalld Warning: you have a firewall running on your linux machine this can prevent communication between the syslog daemon and the omsagent. Checking if firewall has exception for omsagent port [25226] Found exception in the firewalld for the omsagent port.[25226] Checking if security enhanced linux is enabled getenforce ['root 135065 1 0 03:10 ? 00:00:05 /usr/sbin/rsyslogd -n'] Found rsyslog process running on this machine. Checking rsyslog daemon: Testing if the daemon configuration folder exists Located /etc/rsyslog.d directory. Checking omsagent configuration under the name of: 'security-config-omsagent.conf' Located security-config-omsagent.conf rsyslog daemon found, checking daemon configuration content - forwarding all data to port 514 Trying to validate the content of daemon configuration. For extra verification please make sure the configuration content is as defined in the documentation. /etc/rsyslog.d/security-config-omsagent.conf Current content of the daemon configuration is: if $rawmsg contains "CEF:" or $rawmsg contains "Fortigate-" or $rawmsg contains "ASA-" then @@127.0.0.1:25226 rsyslog daemon configuration was found valid. Trying to restart syslog daemon Restarting rsyslog daemon - 'sudo service rsyslog restart' Redirecting to /bin/systemctl restart rsyslog.service rsyslog daemon restarted. This will take a few seconds. Omsagent restarted. This will take a few seconds. Incoming port grep: 0.0.0.0:514 tcp 0 0 0.0.0.0:514 0.0.0.0:* LISTEN udp 0 0 0.0.0.0:514 0.0.0.0:* Daemon incoming port 514 is open Incoming port grep: 25226 unix 2 [ ACC ] STREAM LISTENING 25226 /var/run/lsm/ipc/sim Omsagent is listening to incoming port 25226 Validating CEF\ASA into rsyslog daemon - port 514 This will take 60 seconds. sudo tcpdump -A -ni any port 514 -vv b'tcpdump: listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes\n' b'03:39:56.847938 IP (tos 0x0, ttl 64, id 8437, offset 0, flags [none], proto UDP (17), length 739)\n' b' 10.1.0.4.qwave > 10.0.1.4.syslog: [udp sum ok] SYSLOG, length: 711\n' b'\tFacility local6 (22), Severity notice (5)\n' Received CEF\ASA message in daemon incoming port.[514] Notice: To tcp dump manually execute the following command - 'tcpdump -A -ni any port 514 -vv' Fetching CEF messages from daemon files. Taking 2 snapshots in 5 seconds diff and compering the amount of CEF messages. If found increasing CEF messages daemon is receiving CEF messages. Validating the CEF\ASA logs are received and are in the correct format when received by syslog daemon sudo tac /var/log/syslog Located 0 CEF\ASA messages Validating the CEF\ASA logs are received and are in the correct format when received by syslog daemon sudo tac /var/log/syslog Located 0 CEF\ASA messages Error: no CEF messages received by the daemon. Please validate that you do send CEF messages to agent. Checking daemon incoming connection for tcp and udp This will take 60 seconds. sudo tcpdump -A -ni any port 25226 -vv Could not locate "CEF" message in tcpdump Simulating mock data which you can find in your workspace This will take 60 seconds. sudo tcpdump -A -ni any port 25226 -vv Could not locate "CEF" message in tcpdump Please make sure that traffic to the syslog daemon on port 514 and to the OMS agent on port 25226 are enabled on the internal firewall of the machine No auto sync with the portal Validating that the OMI vulnerability patch is installed. Protected from OMI vulnerability, patch is installed. ['root 138310 1 0 03:39 ? 00:00:00 /usr/sbin/rsyslogd -n'] Found rsyslogd process running on this machine. [] Notice: syslog-ng is not running but found configuration directory for it. Warning: please make sure your logging daemon configuration does not store unnecessary logs. This may cause a full disk on your machine, which will disrupt the function of the oms agent installed. For more information: https://www.rsyslog.com/doc/master/configuration/actions.html Completed troubleshooting. Please check Log Analytics to see if your logs are arriving. All events streamed from these appliances appear in raw form in Log Analytics under CommonSecurityLog type Notice: If no logs appear in workspace try looking at omsagent logs: tail -f /var/opt/microsoft/omsagent/0b78f928-d0ee-454e-a004-10039e95fbc8/log/omsagent.log Warning: Make sure that the logs you send comply with RFC 5424. Kindly help here to fix the issue. Do let me know if any more info is required. Thanks. Much appreciated.Threat intelligence TAXII
I am trying to add the Threat intelligence - TAXII connector in Sentinel. Upon entering the asked details such as mentioned below: Friendly Name: TAXIIFeeds API: https://limo.anomali.com/api/v1/taxii2/feeds/ Collection ID: 107 (tried by entering 135, 136 as well) Username: guest Password: guest Selected Import Indicator as All Available(tried other options as well) and Polling Frequency as Once a day (tried other options as well). Post entering the above mentioned details, when I click Add, I am getting error as "TAXII connector already exists with the same API root URL and Collection ID or inputs are not valid." It seems the API https://limo.anomali.com/api/v1/taxii2/feeds/ is no more valid. When I try to open, it throws an error as "This site can’t be reached". Also, this URL (https://www.anomali.com/resources/limo) it says the API URL is changed. Not sure where the issue is. Can someone help on this please. Best regards.
Parse CEF logging and a map field within message
Hi all, I have successfully configured the oms-agent and the Microsoft CEF python log forward script on a Ubuntu 20.04 x64 VM to forward Fortinet Analyzer logging to Sentinel. I receive the CEF logging in Sentinel. Most fields are mapped correctly. However, not all fields within the <message> are mapped, such as "ad.srccountry". This field is now in the AdditionalExtensions variable along with other data. I want to have this variable in a separate field for log alerting purposes. Problem: Within CEF -> message -> AdditionalExtensions there is the ad.srccountry variable. I want the "ad.srccountry" variable mapped to 'cs1' ( DeviceCustomString1) in Sentinel. I got this idea from a list of all supported CEF message mapping field. However, I cannot figure out how to configure this additional parse within the message field. My configuration: vi /etc/opt/microsoft/omsagent/<tenantID>/conf/omsagent.d/security_events.conf <source> type syslog port 25226 bind 127.0.0.1 protocol_type tcp tag oms.security format /(?<time>(?:\w+ +){2,3}(?:\d+:){2}\d+|\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.[\w\-\:\+]{3,12}):?\s*(?:(?<host>[^: ]+) ?:?)?\s*(?<ident>.*CEF.+?(?=0\|)|%ASA[0-9\-]{8,10})\s*:?(?<message>0\|.*|.*)/ <parse> message_format auto </parse> </source> <filter oms.security.**> type filter_syslog_security </filter> Works (raw message and primary CEF filter): https://regex101.com/r/rdf4iP/1 vi /etc/opt/microsoft/omsagent/<tenantID>/conf/omsagent.d/syslog.conf <source> @type syslog port 25224 bind 127.0.0.1 protocol_type tcp tag oms.syslog </source> <filter oms.syslog.**> @type parser key_name message format /.*ad.srccountry=(?<cs1>.*?)\s.*$/ </filter> Raw message and secondary ad.srccountry filter: https://regex101.com/r/v6iZHs/1 systemctl restart omsagent-<TenantID>.service && systemctl restart syslog But the DeviceCustomString1 (cs1) field in the CommonSecurityLog table from Sentinel stays empty: Any tips or tricks how I can get this to work? Thanks!
