EPS calculation and log size

%3CLINGO-SUB%20id%3D%22lingo-sub-2007929%22%20slang%3D%22en-US%22%3EEPS%20calculation%20and%20log%20size%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2007929%22%20slang%3D%22en-US%22%3E%3CPRE%3E%3CSPAN%3EGood%20evening!%3CBR%20%2F%3EWe%20have%20some%20customers%20in%20the%20process%20of%20aligning%20about%20closing%20a%20partnership%20to%20obtain%20Azure%20Sentinel%2C%20however%2C%20now%20and%20then%20we%20are%20asked%20about%20the%20average%20expense%20that%20they%20may%20have%20through%20the%20acquisition%20of%20the%20solution%20so%20that%20the%20customer%20%E2%80%9Cprojects%E2%80%9D%20what%20average%20consumption%20they%20will%20have.%0A%0AI%20did%20some%20research%20on%20the%20web%20and%20found%20some%20content%20about%20average%20consumption%20of%20EPS%20by%20solutions%20and%20average%20log%20size%20of%20them%2C%20however%2C%20I%20did%20not%20feel%20%22firm%22%20with%20such%20information.%0A%0AI%20know%20it%20is%20subjective%20and%20depends%20on%20each%20solution%20%2F%20technology%2C%20among%20other%20variables%2C%20however%2C%20I%20would%20like%20to%20know%20if%20someone%20has%20%2F%20indicates%20some%20type%20of%20material%20%2F%20spreadsheet%20or%20something%20like%20that%20I%20can%20help%20%E2%80%9Cprice%E2%80%9D%20a%20new%20customer%20for%20Azure%20acquisition%20Sentinel.%0A%0AEx%3A%20Knowing%20on%20average%20that%20the%20customer%20has%20X%20equipment%20and%20technologies%2C%20as%20well%20as%20the%20EPS%20and%20average%20size%20of%20the%20logs%20of%20these%20artifacts%2C%20they%20arrive%20at%20an%20approximate%20value%20of%20X%2C%20we%20can%20%E2%80%9Cstipulate%E2%80%9D%20that%20they%20will%20be%20ingested%20in%20Sentinel%20%E2%80%9CX%20GB%E2%80%9D%2C%20which%20in%20turn%20%2C%20can%20result%20in%20%E2%80%9CX%E2%80%9D%20values.%3C%2FSPAN%3E%3C%2FPRE%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2009110%22%20slang%3D%22en-US%22%3ERe%3A%20EPS%20calculation%20and%20log%20size%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2009110%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F670054%22%20target%3D%22_blank%22%3E%40Luizao_f%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EAs%20you%20say%20there%20can%20be%20a%20large%20variance%20here.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-cpp%22%3E%3CCODE%3Eunion%20withsource%3D_TableName1%20*%0A%7C%20where%20_TimeReceived%20%20%26gt%3B%20ago(1d)%0A%7C%20summarize%20count()%20%2C%20Size%20%3D%20sum(_BilledSize)%20by%20bin(_TimeReceived%2C%201m)%2C%20Type%2C%20_IsBillable%2C%20_BilledSize%0A%7C%20extend%20counttemp%20%3Dcount_%20%2F%2060%0A%7C%20summarize%20%0A%20%20%20%20%20%20%20%20%20%20%20%5B'Average%20Events%20per%20Second%20(eps)'%5D%20%3D%20avg(counttemp)%2C%0A%20%20%20%20%20%20%20%20%20%20%20%5B'Average%20Bytes'%5D%3Davg(_BilledSize)%2C%0A%20%20%20%20%20%20%20%20%20%20%20%2F%2F%5B'Bytes'%5D%3Davg(counttemp)%20*%20avg(_BilledSize)%2C%0A%20%20%20%20%20%20%20%20%20%20%20%5B'Minimum%20eps'%5D%3Dmin%20(counttemp)%2C%0A%20%20%20%20%20%20%20%20%20%20%20%5B'Maximum%20eps'%5D%3Dmax(counttemp)%0A%20%20by%20%5B'Table%20Name'%5D%3DType%0A%7C%20order%20%20by%20%5B'Average%20Events%20per%20Second%20(eps)'%5D%20desc%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3EHere%20I%20run%20the%20above%20on%20the%20Microsoft%20demo%20data%3A%3C%2FP%3E%0A%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fms.portal.azure.com%23%4072f988bf-86f1-41af-91ab-2d7cd011db47%2Fblade%2FMicrosoft_Azure_Monitoring_Logs%2FDemoLogsBlade%2FresourceId%2F%252FDemo%2Fsource%2FLogsBlade.AnalyticsShareLinkToQuery%2Fq%2FH4sIAAAAAAAAA4WRP2%25252BDMBDFdyS%25252Bw9uACCnN0G5USqQOHdKhYUMRMnAilrBB2JCk6oevL5EKDE09%25252Bc%25252Fv3j2%25252FG7RsNc7Snkw79CUleSqKhj6Eog1WvveN84l6Qp5KRZ9UkhypAl4h6jbcVBETZlBK9PKLULaDtmGEGAc%25252BJvwU5jvZNFTxTYTiikLqcCEXY6OiGOm1oxj5u2GeTbjDVMqN6GJJV%25252FcullSH5LbNscbL09KJ72FaWbAdqRc14W0kbQ066nGgsnViIXUmCo7Oqxjr8Fc6iv9Q2F0tmeCYMD3%25252F2ZJfr7NgTk66WOFxaRbspZZqUHDOXLmSGg987cVlDovLjGXUBZ4Ft5mCh%25252BoYzpnDavvKxXAH%25252Fs%25252BnIlNyVSOVtHj2Pd%25252F7AVms0VE8AgAA%2Ftimespan%2FP1D%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3EGo%20to%20Log%20Analytics%20and%20run%20query%3C%2FA%3E%26nbsp%3B%20(top%205%20shown)%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CDIV%3E%0A%3CTABLE%20cellspacing%3D%221%22%20cellpadding%3D%225%22%3E%0A%3CTBODY%3E%0A%3CTR%3E%0A%3CTH%3ETable%20Name%3C%2FTH%3E%0A%3CTH%3EAverage%20Events%20per%20Second%20(eps)%3C%2FTH%3E%0A%3CTH%3EAverage%20Bytes%3C%2FTH%3E%0A%3CTH%3EMinimum%20eps%3C%2FTH%3E%0A%3CTH%3EMaximum%20eps%3C%2FTH%3E%0A%3C%2FTR%3E%0A%3CTR%3E%0A%3CTD%3EContainerLog%3C%2FTD%3E%0A%3CTD%3E1.6899142145116457%3C%2FTD%3E%0A%3CTD%3E350.67232579315635%3C%2FTD%3E%0A%3CTD%3E0%3C%2FTD%3E%0A%3CTD%3E119%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3CTR%3E%0A%3CTD%3EAzureNetworkAnalytics_CL%3C%2FTD%3E%0A%3CTD%3E0.5160871447121628%3C%2FTD%3E%0A%3CTD%3E681.2308469196364%3C%2FTD%3E%0A%3CTD%3E0%3C%2FTD%3E%0A%3CTD%3E227%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3CTR%3E%0A%3CTD%3EW3CIISLog%3C%2FTD%3E%0A%3CTD%3E0.10373542707589817%3C%2FTD%3E%0A%3CTD%3E518.1744672172938%3C%2FTD%3E%0A%3CTD%3E0%3C%2FTD%3E%0A%3CTD%3E18%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3CTR%3E%0A%3CTD%3EAppPageViews%3C%2FTD%3E%0A%3CTD%3E0.06976744186046512%3C%2FTD%3E%0A%3CTD%3E849.984952120383%3C%2FTD%3E%0A%3CTD%3E0%3C%2FTD%3E%0A%3CTD%3E2%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3CTR%3E%0A%3CTD%3EAppMetrics%3C%2FTD%3E%0A%3CTD%3E0.06423229212494501%3C%2FTD%3E%0A%3CTD%3E763.9826849349506%3C%2FTD%3E%0A%3CTD%3E0%3C%2FTD%3E%0A%3CTD%3E33%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3C%2FTBODY%3E%0A%3C%2FTABLE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWhat%20you%20can%20see%20is%20there%20is%20often%20a%20large%20difference%20between%20the%20min%2C%20avg%20and%20max%20%3CSTRONG%3Eeps%3C%2FSTRONG%3E.%26nbsp%3B%20And%20the%20average%20bytes%20varies%20as%20well.%26nbsp%3B%20You%20could%20use%20this%20as%20a%20basis%20of%20a%20calculation%2C%20adding%20in%20your%20device%20counts%20for%20the%20various%20technologies.%26nbsp%3B%20Many%20of%20the%20online%20calculators%20use%20200-500bytes%20as%20a%20baseline.%26nbsp%3B%3CBR%20%2F%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%3EIdeally%20you'd%20do%20a%20POC%20to%20test%20the%20incoming%20customer%20data%2C%20but%20I%20appreciate%20that%20isn't%20always%20possible.%26nbsp%3B%26nbsp%3B%3C%2FP%3E%0A%3C%2FDIV%3E%3C%2FLINGO-BODY%3E
Contributor
Good evening!
We have some customers in the process of aligning about closing a partnership to obtain Azure Sentinel, however, now and then we are asked about the average expense that they may have through the acquisition of the solution so that the customer “projects” what average consumption they will have. I did some research on the web and found some content about average consumption of EPS by solutions and average log size of them, however, I did not feel "firm" with such information. I know it is subjective and depends on each solution / technology, among other variables, however, I would like to know if someone has / indicates some type of material / spreadsheet or something like that I can help “price” a new customer for Azure acquisition Sentinel. Ex: Knowing on average that the customer has X equipment and technologies, as well as the EPS and average size of the logs of these artifacts, they arrive at an approximate value of X, we can “stipulate” that they will be ingested in Sentinel “X GB”, which in turn , can result in “X” values.
1 Reply

@Luizao_f 

 

As you say there can be a large variance here.

 

union withsource=_TableName1 *
| where _TimeReceived  > ago(1d)
| summarize count() , Size = sum(_BilledSize) by bin(_TimeReceived, 1m), Type, _IsBillable, _BilledSize
| extend counttemp =count_ / 60
| summarize 
           ['Average Events per Second (eps)'] = avg(counttemp),
           ['Average Bytes']=avg(_BilledSize),
           //['Bytes']=avg(counttemp) * avg(_BilledSize),
           ['Minimum eps']=min (counttemp),
           ['Maximum eps']=max(counttemp)
  by ['Table Name']=Type
| order  by ['Average Events per Second (eps)'] desc

Here I run the above on the Microsoft demo data:

Go to Log Analytics and run query  (top 5 shown)

Table Name Average Events per Second (eps) Average Bytes Minimum eps Maximum eps
ContainerLog 1.6899142145116457 350.67232579315635 0 119
AzureNetworkAnalytics_CL 0.5160871447121628 681.2308469196364 0 227
W3CIISLog 0.10373542707589817 518.1744672172938 0 18
AppPageViews 0.06976744186046512 849.984952120383 0 2
AppMetrics 0.06423229212494501 763.9826849349506 0 33

 

What you can see is there is often a large difference between the min, avg and max eps.  And the average bytes varies as well.  You could use this as a basis of a calculation, adding in your device counts for the various technologies.  Many of the online calculators use 200-500bytes as a baseline. 


Ideally you'd do a POC to test the incoming customer data, but I appreciate that isn't always possible.