The process of moving from Splunk to Microsoft Sentinel via the SIEM Migration experience has been enhanced with three key additions that help customers get more context aware translations of their detections from Splunk to Sentinel. These features let customers provide more contextual details about their Splunk environment & usage to the Microsoft Sentinel SIEM Migration translation engine so it can account for them when converting the detections from SPL to KQL. These are:
Let talk about how these can make life easier when migrating to Microsoft Sentinel via the SIEM Migration experience:
Most traditional translation tools only factor in Grammar translations when translating from one query language to another. More precisely addressing the “how” in the queries – How are these queries structured? How are operational and computational logics defined? Among other things.
The What is often lost is translation. “What data sources are being queried”? “What do these data sources really map to in the target SIEM”?
The “what” is often environmental customer context that needs to be accounted for in translation to ensure that grammar translations are applied on the right sources.
Schema mappings in the SIEM Migration experience allows you to precisely define how Splunk sources (indexes, data models, etc.) map to Microsoft Sentinel tables within the new “Schema mapping” section of the UI Experience. This feature provides the flexibility and customization to ensure that your data is aligned with your migration needs. On uploading the Splunk export, the system extracts all the sources from the SPL queries. Known sources such as Splunk CIM schemas & data models are auto mapped to ASIM Schemas as applicable. The other custom sources queried in the detections are listed without being mapped and these will require manual mapping with existing Microsoft Sentinel/Azure Log Analytics tables. All mappings can then be reviewed, modified or new sources added. Mapping schemas is hierarchical, i.e., the Splunk sources map 1-1 with Sentinel tables in addition to the fields within these sources.
The best part? The manual changes to schema mapping are saved per workspace so that you do not have to repeat it again.
To leverage Schema Mappings,
Once the changes have been completed, click on “Save Changes”. Note that the Mapping state now changes to “Manually Mapped”.
Once the Schema Mappings are complete, the changes made are taken into account when the SPL saved searches are translated to KQL queries.
Splunk Lookups, like Sentinel Watchlists are lists with field-value combinations that can be queried/correlated against ingested data. The SIEM Migration experience addresses the translation & use of Splunk lookups in SPL queries (in Splunk detections) to Sentinel Watchlists’ use in the KQL queries generated.
Note: Sentinel Watchlists must be created as a pre-requisite to allow mapping these Sentinel Watchlists with Splunk Lookups when you start migrating.
Splunk lookups as a complete data set are defined and are available outside the bounds of the SPL query and the SPL query only references the lookups invoking it with the “lookup”, “inputlookup” and/or “outputlookup” keywords. The translation support is only available for the “lookup” & “inputlookup” keywords where lookup data can be queried/correlated against. The “outputlookup” operation – where data is written to a lookup – is not supported in translation but can be achieved by defining an Automation Rule in Microsoft Sentinel.
For translating the invocation of lookups, SIEM Migration’s translation engine uses the “_GetWatchlist()” KQL function to allow mapping to the correct Sentinel watchlist, supplemented in operation by other KQL functions to translate the complete logic.
To ensure the correct Splunk Lookup à Sentinel Watchlist mapping, its important for the SIEM Migration experience to have this mapping context. The experience now allows for customers to be able to map their Splunk lookups (automatically identified from the Splunk queries uploaded) to Sentinel Watchlists (Created outside the experience as a pre-requisite).
Follow the guidance here to create Sentinel Watchlists.
Once the Watchlists are created, follow the guidance below to map these Sentinel Watchlists to Splunk lookups:
Once the Sentinel Watchlist is selected, the field mappings can be completed by selecting the Watchlist field from the field dropdown corresponding to the Lookup field on the left.
Once all Splunk lookups have been reviewed, click on “Next: Configure Rules” to start translations to KQL.
NOTE: When a Splunk lookup does not have a corresponding Sentinel Watchlist mapped, the translation engine keeps the same name for both the Sentinel Watchlist and its fields as the Splunk lookup and fields.
How does this help?
A core tenet of developers is automation and functionality reuse. Macros are integral for quick development, but every architect silently curses these “shortcuts” when having to migrate to a different tech stack.
When upgrading the SIEM migration experience the team thought: What if someone told the architect “Hey, we got this covered”. All (SPL) detection queries will seamlessly be expanded by making inline replacements of the macro references by the respective macro definitions and passed on to the translation engine to ensure the core detection logic stays retained when the language translations happens.
The Approach
To enable this Macro expansion the experience needs more context and data. Be to context for the data field mapping or the Splunk code associated with the macros. This enrichment is done via the initial file query and uploader which now has a richer query to pull the necessary information - The metadata of the detections and in addition, all macro definitions. This extra information helps identify and ensure all pieces of the puzzle are in the right place before translation.
Do not worry, there are no extra steps here. 🙂
The experience: “Copy the query & run it on Splunk” to obtain the import file necessary for migration remains the same. As mentioned earlier the query has been enhanced to get a broader context with an updated format.
There are no extra touchpoints. The migration experience will take care of the rest and show you the expanded source query with macros references replaced in-line with the respective definitions in the “Configure Rules” tab.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.