Data Ingestion speed in Azure Sentinel

%3CLINGO-SUB%20id%3D%22lingo-sub-2631368%22%20slang%3D%22en-US%22%3EData%20Ingestion%20speed%20in%20Azure%20Sentinel%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2631368%22%20slang%3D%22en-US%22%3E%3CP%3EHi%2C%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EWe%20are%20working%20on%20creating%20a%20custom%20connector%20to%20ingest%20the%20data%20in%20Azure%20Sentinel.%20We%20are%20trying%20to%20replicate%20a%20product%20that%20we%20made%20in%20Splunk.%20However%2C%20the%20time%20taken%20to%20ingest%20our%20data%20in%26nbsp%3BAzure%20Sentinel%20is%20significantly%20higher%20than%20in%20Splunk.%20For%20ingesting%201.5K-2K%20data%20splunk%20takes%20a%20minute%20while%20Azure%20Sentinel%20takes%2010-15%20mins.%20Is%20there%20any%20way%20to%20improve%20ingestion%20speed%20in%20Azure%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2632577%22%20slang%3D%22en-US%22%3ERe%3A%20Data%20Ingestion%20speed%20in%20Azure%20Sentinel%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2632577%22%20slang%3D%22en-US%22%3EWhat%20technology%20are%20you%20using%2C%20Azure%20Functions%2C%20Playbook%2C%20API%20etc...%20Have%20you%20looked%20at%20%3CA%20href%3D%22https%3A%2F%2Fgithub.com%2FAzure%2FAzure-Sentinel%2Ftree%2Fmaster%2FDataConnectors%23guide-to-building-azure-sentinel-data-experiences%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Fgithub.com%2FAzure%2FAzure-Sentinel%2Ftree%2Fmaster%2FDataConnectors%23guide-to-building-azure-sentinel-data-experiences%3C%2FA%3E%20%3F%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2675485%22%20slang%3D%22en-US%22%3ERe%3A%20Data%20Ingestion%20speed%20in%20Azure%20Sentinel%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2675485%22%20slang%3D%22en-US%22%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F239477%22%20target%3D%22_blank%22%3E%40Clive%20Watson%3C%2FA%3E%2C%20We%20are%20using%20Azure%20Function%20(Python).%20We%20have%20followed%20the%20same%20link%20you%20provided%20above.However%2C%20we%20cannot%20identify%20any%20best%20practices%20to%20follow%20to%20make%20sure%20performance%20is%20improved%20in%20the%20data%20collection.%20Can%20you%20please%20help%20us%20with%20any%20checklist%20or%20best%20practices%20to%20follow%20to%20make%20the%20data%20collection%20intact%20and%20optimized%3F%3CBR%20%2F%3E%3C%2FLINGO-BODY%3E
Occasional Contributor

Hi,

 

We are working on creating a custom connector to ingest the data in Azure Sentinel. We are trying to replicate a product that we made in Splunk. However, the time taken to ingest our data in Azure Sentinel is significantly higher than in Splunk. For ingesting 1.5K-2K data splunk takes a minute while Azure Sentinel takes 10-15 mins. Is there any way to improve ingestion speed in Azure?

 

 

2 Replies
What technology are you using, Azure Functions, Playbook, API etc... Have you looked at https://github.com/Azure/Azure-Sentinel/tree/master/DataConnectors#guide-to-building-azure-sentinel-... ?
@CliveWatson, We are using Azure Function (Python). We have followed the same link you provided above.However, we cannot identify any best practices to follow to make sure performance is improved in the data collection. Can you please help us with any checklist or best practices to follow to make the data collection intact and optimized?