%3CLINGO-SUB%20id%3D%22lingo-sub-1361534%22%20slang%3D%22en-US%22%3ERe%3A%20Best%20Practices%20for%20Common%20Event%20Format%20(CEF)%20collection%20in%20Azure%20Sentinel%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1361534%22%20slang%3D%22en-US%22%3E%3CP%3ECould%20I%20setup%20the%20CEF%20Collector%20on%20the%20same%20server%20I%20am%20currently%20using%20to%20push%20logs%20to%20Cloud%20App%20Security%3F%20Or%20would%20it%20be%20advisable%20to%20create%20yet%20another%20machine%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1364482%22%20slang%3D%22en-US%22%3ERe%3A%20Best%20Practices%20for%20Common%20Event%20Format%20(CEF)%20collection%20in%20Azure%20Sentinel%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1364482%22%20slang%3D%22en-US%22%3E%3CP%3EHi%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F446404%22%20target%3D%22_blank%22%3E%40DBR14%3C%2FA%3E%26nbsp%3B%2C%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EHope%20all%20is%20well.%20For%20a%20cleaner%20deployment%2C%20it%20is%20best%20to%20create%20another%20machine%20to%20serve%20as%20the%20CEF%20collector%20to%20ingest%20third%20party%20events%20into%20Azure%20Sentinel.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EBest%2C%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ECristhofer%20Romeo%20Munoz%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-969990%22%20slang%3D%22en-US%22%3EBest%20Practices%20for%20Common%20Event%20Format%20(CEF)%20collection%20in%20Azure%20Sentinel%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-969990%22%20slang%3D%22en-US%22%3E%3CH2%20id%3D%22toc-hId--1443962234%22%20id%3D%22toc-hId--1443962264%22%20id%3D%22toc-hId--1443962264%22%20id%3D%22toc-hId--1443962264%22%3E%3CSPAN%20style%3D%22display%3A%20inline%20!important%3B%20float%3A%20none%3B%20background-color%3A%20%23ffffff%3B%20color%3A%20%23333333%3B%20cursor%3A%20text%3B%20font-family%3A%20inherit%3B%20font-size%3A%2016px%3B%20font-style%3A%20normal%3B%20font-variant%3A%20normal%3B%20font-weight%3A%20300%3B%20letter-spacing%3A%20normal%3B%20line-height%3A%201.7142%3B%20orphans%3A%202%3B%20text-align%3A%20left%3B%20text-decoration%3A%20none%3B%20text-indent%3A%200px%3B%20text-transform%3A%20none%3B%20-webkit-text-stroke-width%3A%200px%3B%20white-space%3A%20normal%3B%20word-spacing%3A%200px%3B%22%3EBy%20Nicholas%20DiCola%20%26amp%3B%20Cristhofer%20Romeo%20Munoz%3C%2FSPAN%3E%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-1043550599%22%20id%3D%22toc-hId-1043550569%22%20id%3D%22toc-hId-1043550569%22%20id%3D%22toc-hId-1043550569%22%3EWhat%20is%20CEF%20collection%3F%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EMost%20network%20and%20security%20systems%20support%20either%20Syslog%20or%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fcommunity.microfocus.com%2Ft5%2FArcSight-Connectors%2FArcSight-Common-Event-Format-CEF-Implementation-Standard%2Fta-p%2F1645557%22%20target%3D%22_self%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3ECEF%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E(which%20stands%20for%20Common%20Event%20Format)%20over%20Syslog%20as%20means%20for%20sending%20data%20to%20a%20SIEM.%20Azure%20Sentinel%20provides%20the%20ability%20to%20ingest%20data%20from%20an%20external%20solution.%20If%20your%20appliance%20or%20system%20enables%20you%20to%20send%20logs%20over%20Syslog%20using%20the%20Common%20Event%20Format%20(CEF)%2C%20the%20integration%20with%20Azure%20Sentinel%20enables%20you%20to%20easily%20run%20analytics%2C%20and%20queries%20across%20the%20data.%20This%20makes%20Syslog%20or%20CEF%20the%20most%20straight%20forward%20ways%20to%20stream%20security%20and%20networking%20events%20to%20Azure%20Sentinel.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThe%20advantage%20of%20CEF%20over%20Syslog%20is%20that%20it%20ensures%20the%20data%20is%20normalized%20making%20it%20more%20immediately%20useful%20for%20analysis%20using%20Sentinel.%20However%2C%20unlike%20many%20other%20SIEM%20products%2C%20Sentinel%20allows%20ingesting%20unparsed%20Syslog%20events%20and%20performing%20analytics%20on%20them%20using%20query%20time%20parsing.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThe%20number%20of%20systems%20supporting%20Syslog%20or%20CEF%20is%20in%20the%20hundreds%2C%20please%20make%20sure%20to%20check%20out%20the%20%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzure-Sentinel%2FAzure-Sentinel-The-Syslog-and-CEF-source-configuration-grand%2Fba-p%2F803891%22%20target%3D%22_blank%22%20rel%3D%22noopener%22%3EAzure%20Sentinel%20grand%20list%3C%2FA%3E%26nbsp%3Bfor%20a%20comprehensive%20list%20of%20sources%20supporting%20CEF.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIn%20this%20blogpost%2C%20we%20will%20provide%20details%20on%20how%20CEF%20collection%20works%20and%20the%20best%20practices%20you%20should%20consider%20when%20configuring%20common%20event%20format%20collection%20in%20Azure%20Sentinel.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--763903864%22%20id%3D%22toc-hId--763903894%22%20id%3D%22toc-hId--763903894%22%20id%3D%22toc-hId--763903894%22%3EHow%20does%20it%20work%3F%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ECEF%20Collection%20in%20Azure%20Sentinel%20uses%20a%20Linux%20machine%20that%20is%20used%20as%20a%20log%20forwarder%20between%20your%20security%20solution%20and%20Azure%20Sentinel.%20The%20Linux%20machine%20can%20be%20inyour%20on-prem%20environment%2C%20in%20Azure%20or%20in%20other%20clouds.%20As%20part%20of%20the%20deployment%20process%2C%20the%20Log%20Analytics%20agent%20is%20installed%20on%20the%20Linux%20machine%20and%20serves%20to%20relay%20the%20events%20securely%20to%20your%20Azure%20Sentinel%20workspace.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%222020-07-22%2013_56_32-PowerPoint%20Slide%20Show%20%20-%20%20L400-M4%20On-Prem%20collection%20architecture.pptx.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F207091i132D8B8AC652A54E%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20title%3D%222020-07-22%2013_56_32-PowerPoint%20Slide%20Show%20%20-%20%20L400-M4%20On-Prem%20collection%20architecture.pptx.png%22%20alt%3D%222020-07-22%2013_56_32-PowerPoint%20Slide%20Show%20-%20L400-M4%20On-Prem%20collection%20architecture.pptx.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThe%20following%20flow%20chart%20details%20the%20high-level%20steps%20to%20configure%20CEF%20collection%20in%20Azure%20Sentinel%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%222020-07-22%2013_53_24-Presentation1%20-%20PowerPoint.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F207090i9613153087D9787B%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20title%3D%222020-07-22%2013_53_24-Presentation1%20-%20PowerPoint.png%22%20alt%3D%222020-07-22%2013_53_24-Presentation1%20-%20PowerPoint.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EFor%20detailed%20information%20on%20deploying%20CEF%20and%20how%20CEF%20collection%20works%2C%20please%20visit%20the%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fsentinel%2Fconnect-common-event-format%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure%20Setninel%20CEF%20documentation%3C%2FA%3E.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%3ETip%3A%20Want%20to%20ingest%20test%20CEF%20data%3F%20%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzure-Sentinel%2FIngest-Sample-CEF-data-into-Azure-Sentinel%2Fba-p%2F1064158%22%20target%3D%22_self%22%3Ehere%3C%2FA%3E%20is%20how%20to%20do%20that.%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-1723608969%22%20id%3D%22toc-hId-1723608939%22%20id%3D%22toc-hId-1723608939%22%20id%3D%22toc-hId-1723608939%22%3ESecuring%20your%20CEF%20collection%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH3%20id%3D%22toc-hId--1880796853%22%20id%3D%22toc-hId--1880796883%22%20id%3D%22toc-hId--1880796883%22%20id%3D%22toc-hId--1880796883%22%3ESecure%20your%20machine%3C%2FH3%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EMake%20sure%20to%20configure%20the%20machine's%20security%20according%20to%20your%20organization's%20security%20policy.%20For%20example%2C%20you%20can%20configure%20your%20network%20to%20align%20with%20your%20corporate%20network%20security%20policy%20and%20change%20the%20ports%20and%20protocols%20in%20the%20daemon%20to%20align%20with%20your%20requirements.%20You%20can%20use%20the%20following%20instructions%20to%20improve%20your%20machine%20security%20configuration%3A%20-ERR%3AREF-NOT-FOUND-Secure%20VM%20in%20azure%2C%20-ERR%3AREF-NOT-FOUND-Best%20practices%20for%20Network%20security.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ENote%3A%20To%20use%20TLS%20communication%20between%20the%20security%20solution%20and%20the%20Syslog%20machine%2C%20you%20will%20need%20to%20configure%20the%20Syslog%20daemon%20(rsyslog%20or%20syslog-ng)%20to%20communicate%20in%20TLS%3A%20-ERR%3AREF-NOT-FOUND-Encrypting%20Syslog%20Traffic%20with%20TLS%20-%20rsyslog%2C%20-ERR%3AREF-NOT-FOUND-Encrypting%20log%20messages%20with%20TLS%20%E2%80%93%20syslog-ng.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH3%20id%3D%22toc-hId-606715980%22%20id%3D%22toc-hId-606715950%22%20id%3D%22toc-hId-606715950%22%20id%3D%22toc-hId-606715950%22%3EProperly%20Secure%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3Ean%20Azure%20VM%20with%20Public%20IP%3C%2FH3%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ECommon%20Event%20Format%20uses%20UDP%20514%20therefore%20messages%20will%20be%20sent%20in%20clear%20text.%20If%20we%20use%20an%20unencrypted%20connection%20and%20a%20third%20party%20intercepts%20the%20connection%2C%20they%20can%20see%20all%20the%20information%20exchanged%20in%20plain%20text.%20%26nbsp%3BTLS%20can%20protect%20this%20communication%20traffic.%20Additionally%2C%20a%20bad%20actor%20can%20potentially%20intercept%20and%20send%20messages%20to%20your%20Azure%20VMs%20to%20create%20false%20records.%20Make%20sure%20to%20protect%20the%20Azure%20VM%20with%20either%20a%20Network%20Security%20Group%20or%20NVA.%20On%20the%20NSG%2C%20you%20should%20configure%20a%20list%20of%20source%20IP%20addresses%20you%20want%20to%20allow%20to%20communicate%20to%20the%20VM.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH3%20id%3D%22toc-hId--1200738483%22%20id%3D%22toc-hId--1200738513%22%20id%3D%22toc-hId--1200738513%22%20id%3D%22toc-hId--1200738513%22%3EDeploy%20CEF%20collector%20machine%20close%20to%20source%3C%2FH3%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ELet%E2%80%99s%20review%20the%20common%20scenarios%3A%3C%2FP%3E%0A%3CP%3EOn-prem%20source%20%E2%80%93%20We%20recommend%20you%20deploy%20the%20CEF%20collector%20on-prem.%20Syslog%20CEF%20is%20default%20sent%20over%20UDP%20port%20514%20in%20plain%20text.%20Once%20it%20reaches%20the%20CEF%20collector%2C%20it%20is%20sent%20to%20Azure%20Sentinel%20using%20HTTPs.%20Deploying%20on-prem%20ensure%20your%20data%20is%20not%20sent%20in%20the%20clear%20outside%20your%20network.%3C%2FP%3E%0A%3CP%3EIf%20on-prem%20deployment%20is%20not%20an%20option%20at%20all%2C%20we%20recommend%20you%20deploy%20in%20a%20cloud%20VM%20and%20use%20TCP%20over%20TLS%20or%20ensure%20there%20is%20a%20secure%20path%20(VPN)%20between%20the%20source%20and%20the%20collector%20VM.%3C%2FP%3E%0A%3CP%3ECloud%20source%20%E2%80%93%20We%20recommend%20you%20deploy%20the%20CEF%20collector%20in%20the%20same%20virtual%20network%20as%20the%20source.%20You%20can%20also%20use%20VNet%20peering%20to%20have%20multiple%20VNets%20connected%20to%20the%20CEF%20collector.%20This%20again%20prevents%20your%20traffic%20from%20being%20sent%20over%20the%20internet%20in%20the%20clear.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--1211241587%22%20id%3D%22toc-hId--1211241617%22%20id%3D%22toc-hId--1211241617%22%20id%3D%22toc-hId--1211241617%22%3EEnsure%20the%20quality%20of%20your%20CEF%20collection%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH3%20id%3D%22toc-hId--520680113%22%20id%3D%22toc-hId--520680143%22%20id%3D%22toc-hId--520680143%22%20id%3D%22toc-hId--520680143%22%3EDo%20not%20configure%20the%20source%20to%20send%20to%202%20CEF%20Collectors%3C%2FH3%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ETo%20avoid%20duplicate%20records%2Fmessages%20in%20Azure%20Sentinel%20do%20not%20configure%20your%20third-party%20security%20appliance%20to%20send%20data%20to%202%20Common%20Event%20Format%20collectors.%20Additionally%2C%20sending%20duplicate%20data%20will%20increase%20your%20consumption%20bill%20therefore%20ensure%20that%20your%20security%20appliances%20are%20set%20to%20send%20data%20to%20a%20single%20CEF%20collector.%20You%20may%20consider%20using%20a%20load%20balancer%20or%20rsyslog%20cluster%20(-ERR%3AREF-NOT-FOUND-%3CA%20href%3D%22https%3A%2F%2Fwww.rsyslog.com%2Fload-balancing-for-rsyslog%2F%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fwww.rsyslog.com%2Fload-balancing-for-rsyslog%2F%3C%2FA%3E)%20as%20an%20aleternative.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH3%20id%3D%22toc-hId-1966832720%22%20id%3D%22toc-hId-1966832690%22%20id%3D%22toc-hId-1966832690%22%20id%3D%22toc-hId-1966832690%22%3ETCP%20is%20the%20default%20protocol%2C%20always%20use%20%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3C%2FH3%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ESyslog%20CEF%20can%20support%20UDP%20or%20TCP.%20However%2C%20TCP%20is%20now%20the%20default%20protocol%20and%20should%20always%20be%20used%20unless%20the%20source%20device%20does%20not%20support%20TCP.%20The%20advantage%20is%20the%20reliability%20of%20the%20TCP%20protocol.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH3%20id%3D%22toc-hId--535785222%22%20id%3D%22toc-hId--535785252%22%20id%3D%22toc-hId--535785252%22%20id%3D%22toc-hId--535785252%22%3EScaling%20CEF%20collection%3C%2FH3%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWant%20to%20scale%20CEF%20or%20Syslog%20collection%3F%26nbsp%3B%3CSPAN%3E%26nbsp%3BUse%20a%20VM%20scale%20set%20as%20described%26nbsp%3B%3C%2FSPAN%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fazure-sentinel%2Fscaling-up-syslog-cef-collection%2Fba-p%2F1185854%22%20target%3D%22_self%22%3Ehere%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-1822644892%22%20id%3D%22toc-hId-1822644862%22%20id%3D%22toc-hId-1822644862%22%20id%3D%22toc-hId-1822644862%22%3EQuerying%20the%20Data%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EOnce%20you%20satisfy%20the%20configuration%20steps%20in%20the%20Common%20Event%20Format%20data%20connector%2C%20you%20can%20then%20query%20the%20data.%20CEF%20logs%20will%20reside%20within%20the%20CommonSecurityLog%20table%2C%20hence%20by%20querying%20for%20CommonSecurityLog%20you%20will%20have%20visibility%20to%20your%20CEF%20data.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22CommonSecurityLog.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F157877i4279312FDD83F8E8%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20title%3D%22CommonSecurityLog.png%22%20alt%3D%22CommonSecurityLog.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ENow%20that%20we%20can%20see%20the%20data%20in%20Azure%20Sentinel%2C%20we%20now%20can%20build%20workbooks%2C%20analytic%20rules%2C%20hunting%20queries%2C%20or%20associate%20it%20with%20other%20data%20for%20correlation.%3C%2FP%3E%0A%3CH2%20style%3D%22box-sizing%3A%20border-box%3B%20color%3A%20inherit%3B%20font-family%3A%20inherit%3B%20font-size%3A%2024px%3B%20font-style%3A%20normal%3B%20font-variant%3A%20normal%3B%20font-weight%3A%20normal%3B%20letter-spacing%3A%20normal%3B%20line-height%3A%201.2%3B%20margin-bottom%3A%2012px%3B%20margin-top%3A%2024px%3B%20orphans%3A%202%3B%20text-align%3A%20left%3B%20text-decoration%3A%20none%3B%20text-indent%3A%200px%3B%20text-transform%3A%20none%3B%20-webkit-text-stroke-width%3A%200px%3B%20white-space%3A%20normal%3B%20word-spacing%3A%200px%3B%22%20id%3D%22toc-hId-15190429%22%20id%3D%22toc-hId-15190399%22%20id%3D%22toc-hId-15190399%22%20id%3D%22toc-hId-15190399%22%3EConclusion%3C%2FH2%3E%0A%3CP%3EIn%20this%20blog%2C%20you%20learned%20how%20Common%20Event%20Format%20collection%20works%20and%20the%20best%20practices%20to%20consider%20when%20configuring%20CEF%20collection%20in%20Azure%20Sentinel.%20To%20learn%20more%20about%20Azure%20Sentinel%2C%20see%20the%20following%20articles%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-969990%22%20slang%3D%22en-US%22%3E%3CP%3ELearn%20more%20about%20Azure%20Sentinel's%20collection%20from%20CEF%20sources%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-969990%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20Sentinel%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EConnectors%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EDetection%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ESecurity%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1884292%22%20slang%3D%22en-US%22%3ERe%3A%20Best%20Practices%20for%20Common%20Event%20Format%20(CEF)%20collection%20in%20Azure%20Sentinel%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1884292%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F293879%22%20target%3D%22_blank%22%3E%40Ofer_Shezaf%3C%2FA%3E%26nbsp%3B%3A%20I%20am%20trying%20to%20get%20Palo%20Alto%20Network%20logs%20into%20sentinel%2C%20but%20it%20seems%20to%20lack%20fields.%26nbsp%3B%3C%2FP%3E%3CP%3EIn%20the%20CEF%20header%20I%20think%20there%20should%20be%20a%20%22name%22%20field%20and%20I%20am%20also%20missing%20a%20deviceEventCategory%20field.%20Is%20that%20by%20purpose%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EBest%20Regards%3C%2FP%3E%3CP%3EPhilip%3C%2FP%3E%3C%2FLINGO-BODY%3E

By Nicholas DiCola & Cristhofer Romeo Munoz

 

What is CEF collection?

 

Most network and security systems support either Syslog or CEF (which stands for Common Event Format) over Syslog as means for sending data to a SIEM. Azure Sentinel provides the ability to ingest data from an external solution. If your appliance or system enables you to send logs over Syslog using the Common Event Format (CEF), the integration with Azure Sentinel enables you to easily run analytics, and queries across the data. This makes Syslog or CEF the most straight forward ways to stream security and networking events to Azure Sentinel.

 

The advantage of CEF over Syslog is that it ensures the data is normalized making it more immediately useful for analysis using Sentinel. However, unlike many other SIEM products, Sentinel allows ingesting unparsed Syslog events and performing analytics on them using query time parsing. 

 

The number of systems supporting Syslog or CEF is in the hundreds, please make sure to check out the Azure Sentinel grand list for a comprehensive list of sources supporting CEF.

 

In this blogpost, we will provide details on how CEF collection works and the best practices you should consider when configuring common event format collection in Azure Sentinel.

 

How does it work?

 

CEF Collection in Azure Sentinel uses a Linux machine that is used as a log forwarder between your security solution and Azure Sentinel. The Linux machine can be inyour on-prem environment, in Azure or in other clouds. As part of the deployment process, the Log Analytics agent is installed on the Linux machine and serves to relay the events securely to your Azure Sentinel workspace.

 

2020-07-22 13_56_32-PowerPoint Slide Show  -  L400-M4 On-Prem collection architecture.pptx.png

 

 

The following flow chart details the high-level steps to configure CEF collection in Azure Sentinel:

 

2020-07-22 13_53_24-Presentation1 - PowerPoint.png

 

For detailed information on deploying CEF and how CEF collection works, please visit the Azure Setninel CEF documentation.

 

Tip: Want to ingest test CEF data? here is how to do that.

 

Securing your CEF collection

 

Secure your machine

 

Make sure to configure the machine's security according to your organization's security policy. For example, you can configure your network to align with your corporate network security policy and change the ports and protocols in the daemon to align with your requirements. You can use the following instructions to improve your machine security configuration: Secure VM in azure, Best practices for Network security.

 

Note: To use TLS communication between the security solution and the Syslog machine, you will need to configure the Syslog daemon (rsyslog or syslog-ng) to communicate in TLS: Encrypting Syslog Traffic with TLS - rsyslog, Encrypting log messages with TLS – syslog-ng.

 

Properly Secure an Azure VM with Public IP

 

Common Event Format uses UDP 514 therefore messages will be sent in clear text. If we use an unencrypted connection and a third party intercepts the connection, they can see all the information exchanged in plain text.  TLS can protect this communication traffic. Additionally, a bad actor can potentially intercept and send messages to your Azure VMs to create false records. Make sure to protect the Azure VM with either a Network Security Group or NVA. On the NSG, you should configure a list of source IP addresses you want to allow to communicate to the VM.

 

Deploy CEF collector machine close to source

 

Let’s review the common scenarios:

On-prem source – We recommend you deploy the CEF collector on-prem. Syslog CEF is default sent over UDP port 514 in plain text. Once it reaches the CEF collector, it is sent to Azure Sentinel using HTTPs. Deploying on-prem ensure your data is not sent in the clear outside your network.

If on-prem deployment is not an option at all, we recommend you deploy in a cloud VM and use TCP over TLS or ensure there is a secure path (VPN) between the source and the collector VM.

Cloud source – We recommend you deploy the CEF collector in the same virtual network as the source. You can also use VNet peering to have multiple VNets connected to the CEF collector. This again prevents your traffic from being sent over the internet in the clear.

 

Ensure the quality of your CEF collection

 

Do not configure the source to send to 2 CEF Collectors

 

To avoid duplicate records/messages in Azure Sentinel do not configure your third-party security appliance to send data to 2 Common Event Format collectors. Additionally, sending duplicate data will increase your consumption bill therefore ensure that your security appliances are set to send data to a single CEF collector. You may consider using a load balancer or rsyslog cluster (https://www.rsyslog.com/load-balancing-for-rsyslog/) as an aleternative.

 

TCP is the default protocol, always use  

 

Syslog CEF can support UDP or TCP. However, TCP is now the default protocol and should always be used unless the source device does not support TCP. The advantage is the reliability of the TCP protocol.

 

Scaling CEF collection

 

Want to scale CEF or Syslog collection?  Use a VM scale set as described here

 

Querying the Data

 

Once you satisfy the configuration steps in the Common Event Format data connector, you can then query the data. CEF logs will reside within the CommonSecurityLog table, hence by querying for CommonSecurityLog you will have visibility to your CEF data.

 

CommonSecurityLog.png

 

Now that we can see the data in Azure Sentinel, we now can build workbooks, analytic rules, hunting queries, or associate it with other data for correlation.

Conclusion

In this blog, you learned how Common Event Format collection works and the best practices to consider when configuring CEF collection in Azure Sentinel. To learn more about Azure Sentinel, see the following articles:

 

 

 

 

 

 

 

 

4 Comments
New Contributor

Could I setup the CEF Collector on the same server I am currently using to push logs to Cloud App Security? Or would it be advisable to create yet another machine?

 

 

Hi @DBR14 ,

 

Hope all is well. For a cleaner deployment, it is best to create another machine to serve as the CEF collector to ingest third party events into Azure Sentinel.

 

Best,

 

Cristhofer Romeo Munoz

New Contributor

@Cristhofer Munoz @Ofer_Shezaf Since the syslog server will store data locally first in plain text, there is a security risk of it being edited/accessed.

Is there a solution to this? we had a query on this requirement

In case of splunk and some other siems there is an option to send the syslog directly to splunk/others over a tcp port which then ingests, encrypts and handles the data.

https://docs.splunk.com/Documentation/Splunk/7.1.3/Data/Monitornetworkports

 

 

Senior Member

@Ofer_Shezaf : I am trying to get Palo Alto Network logs into sentinel, but it seems to lack fields. 

In the CEF header I think there should be a "name" field and I am also missing a deviceEventCategory field. Is that by purpose?

 

Best Regards

Philip