How to view ingested traffic logs on MCAS

%3CLINGO-SUB%20id%3D%22lingo-sub-1467042%22%20slang%3D%22en-US%22%3EHow%20to%20view%20ingested%20traffic%20logs%20on%20MCAS%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1467042%22%20slang%3D%22en-US%22%3E%3CP%3EI'm%20uploading%20my%20traffic%20logs%20in%20order%20to%20perform%20the%20Cloud%20App%20discovery%20using%20REST%20APIs%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fcloud-app-security%2Fapi-discovery%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fcloud-app-security%2Fapi-discovery%3C%2FA%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20can%20see%20the%20uploaded%20files%20being%20processed%20under%20%22Governance%20logs%22.%20I%20can%20also%20generate%20reports%20based%20on%20the%20ingested%20traffic%20logs.%20But%20is%20there%20any%20option%20to%20view%2Fvisualize%20the%20ingested%2Fuploaded%20logs%20in%20a%20table%20form%3F%20So%20that%20I%20can%20verify%20that%20all%20the%20logs%20that%20are%20supposed%20to%20be%20uploaded%20are%20actually%20being%20uploaded%20via%20my%20script%20and%20the%20reports%20that%20are%20generated%20are%20correct%20as%20per%20my%20logs.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks!%3C%2FP%3E%3CP%3EKaushal.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1467042%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3ECloud%20App%20Security%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ECloud%20Discovery%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1472726%22%20slang%3D%22en-US%22%3ERe%3A%20How%20to%20view%20ingested%20traffic%20logs%20on%20MCAS%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1472726%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F700524%22%20target%3D%22_blank%22%3E%40kaushal28%3C%2FA%3E%26nbsp%3BThanks%20for%20the%20feedback!%20If%20you%20go%20to%20Discovered%20apps%20%26gt%3B%20there%20is%20an%20export%20button%20where%20you%20can%20export%20the%20data%20in%20an%20excel%20form.%20Hope%20this%20helps!%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Caroline_Lee_0-1592438252094.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F199400i5B48DE5D1954E4B7%2Fimage-size%2Fmedium%3Fv%3D1.0%26amp%3Bpx%3D400%22%20title%3D%22Caroline_Lee_0-1592438252094.png%22%20alt%3D%22Caroline_Lee_0-1592438252094.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1473097%22%20slang%3D%22en-US%22%3ERe%3A%20How%20to%20view%20ingested%20traffic%20logs%20on%20MCAS%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1473097%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F587417%22%20target%3D%22_blank%22%3E%40Caroline_Lee%3C%2FA%3EThanks%20for%20your%20reply!%20I've%20tried%20exporting%20the%20data%20from%20where%20you%20mentioned%20but%20it%20seems%20like%20it's%20the%20data%20of%20discovered%20applications%20only%20and%20not%20the%20original%20CEF%20data%20which%20I've%20ingested%20for%20that%20discovery.%20The%20same%20options%20are%20also%20available%20under%20IP%20addresses%20and%20Users%20tab%2C%20but%20it%20only%20exports%20the%20respective%20discovery%20related%20data%20and%20not%20the%20original%20one.%3CBR%20%2F%3E%3CBR%20%2F%3EThe%20purpose%20of%20being%20able%20to%20view%2Fexport%20the%20exact%20data%20which%20I%20have%20ingested%20is%20to%20verify%20whether%20my%20ingestion%20script%20is%20working%20perfectly%20or%20not%20(I%20want%20to%20use%20continuous%20reports).%20Now%20if%20I%20can%20only%20see%20the%20data%20fields%2Frecords%20on%20which%20the%20discovery%20is%20successful%2C%20I%20would%20not%20be%20able%20to%20exactly%20say%20whether%20the%20data%20I%20ingested%20had%20no%20findings%20when%20MCAS%20ran%20discovery%20on%20it%20or%20my%20script%20messed%20up%20and%20MCAS%20never%20actually%20received%20that%20data%20to%20perform%20discovery%20on.%3CBR%20%2F%3E%3CBR%20%2F%3ESo%20for%20this%20purpose%2C%20only%20the%20count%20of%20the%20number%20of%20records%20received%20by%20MCAS%20is%20also%20enough.%20Is%20there%20such%20an%20option%20available%20yet%20on%20the%20MCAS%20platform%3F%3CBR%20%2F%3E%3CBR%20%2F%3EThanks!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1474152%22%20slang%3D%22en-US%22%3ERe%3A%20How%20to%20view%20ingested%20traffic%20logs%20on%20MCAS%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1474152%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F700524%22%20target%3D%22_blank%22%3E%40kaushal28%3C%2FA%3E%26nbsp%3BThanks%20for%20the%20clarification.%20Currently%2C%20you%20cannot%20see%20the%20actual%20data%20ingested%20in%20MCAS%20but%20you%20can%20see%20the%20%23%20of%20uploaded%20logs%20if%20you%20go%20to%20Settings%20%26gt%3B%20Log%20Collector%20%26gt%3B%20Datasource%20tab.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1476387%22%20slang%3D%22en-US%22%3ERe%3A%20How%20to%20view%20ingested%20traffic%20logs%20on%20MCAS%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1476387%22%20slang%3D%22en-US%22%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F587417%22%20target%3D%22_blank%22%3E%40Caroline_Lee%3C%2FA%3E%20I%20just%20checked%20it%20and%20it%20seems%20it's%20displaying%20the%20number%20of%20log%20files%20uploaded%20so%20far%20for%20any%20data%20source%20(and%20not%20the%20actual%20number%20of%20logs%20because%20each%20log%20file%20can%20have%20a%20large%20number%20of%20logs).%3CBR%20%2F%3E%3CBR%20%2F%3EAnyways%2C%20Thanks%20for%20your%20reply!%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1581199%22%20slang%3D%22en-US%22%3ERe%3A%20How%20to%20view%20ingested%20traffic%20logs%20on%20MCAS%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1581199%22%20slang%3D%22en-US%22%3E%3CP%3EHi%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F700524%22%20target%3D%22_blank%22%3E%40kaushal28%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EAs%20Caroline%20mentioned%20we%20do%20not%20have%20raw%20data%20investigation%20capability%20today.%3C%2FP%3E%0A%3CP%3EBut%20in%20order%20to%20verify%20your%20script%20functionality%2C%20I%20suggest%20to%20send%20a%20single%20file%20with%20several%20log%20lines%20of%20different%20app%20access'%20and%20verify%20all%20apps%20are%20discovered.%26nbsp%3B%3C%2FP%3E%0A%3CP%3EI%20also%20recommend%20creating%20a%20new%20data%20source%20to%20have%20a%20separate%20continuous%20report%20and%20upload%20these%20logs%20directly%20to%20it%20using%20the%20%22%3CSPAN%3EinputStreamName%22%20parameter.%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%3CSPAN%3ELater%20you%20could%20delete%20this%20data%20source.%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%3EBoris%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E
Occasional Contributor

I'm uploading my traffic logs in order to perform the Cloud App discovery using REST APIs: https://docs.microsoft.com/en-us/cloud-app-security/api-discovery

 

I can see the uploaded files being processed under "Governance logs". I can also generate reports based on the ingested traffic logs. But is there any option to view/visualize the ingested/uploaded logs in a table form? So that I can verify that all the logs that are supposed to be uploaded are actually being uploaded via my script and the reports that are generated are correct as per my logs.

 

Thanks!

Kaushal.

 

 

5 Replies

@kaushal28 Thanks for the feedback! If you go to Discovered apps > there is an export button where you can export the data in an excel form. Hope this helps!

 

Caroline_Lee_0-1592438252094.png

 

@Caroline_LeeThanks for your reply! I've tried exporting the data from where you mentioned but it seems like it's the data of discovered applications only and not the original CEF data which I've ingested for that discovery. The same options are also available under IP addresses and Users tab, but it only exports the respective discovery related data and not the original one.

The purpose of being able to view/export the exact data which I have ingested is to verify whether my ingestion script is working perfectly or not (I want to use continuous reports). Now if I can only see the data fields/records on which the discovery is successful, I would not be able to exactly say whether the data I ingested had no findings when MCAS ran discovery on it or my script messed up and MCAS never actually received that data to perform discovery on.

So for this purpose, only the count of the number of records received by MCAS is also enough. Is there such an option available yet on the MCAS platform?

Thanks!

@kaushal28 Thanks for the clarification. Currently, you cannot see the actual data ingested in MCAS but you can see the # of uploaded logs if you go to Settings > Log Collector > Datasource tab.

@Caroline_Lee I just checked it and it seems it's displaying the number of log files uploaded so far for any data source (and not the actual number of logs because each log file can have a large number of logs).

Anyways, Thanks for your reply!

Hi @kaushal28 

As Caroline mentioned we do not have raw data investigation capability today.

But in order to verify your script functionality, I suggest to send a single file with several log lines of different app access' and verify all apps are discovered. 

I also recommend creating a new data source to have a separate continuous report and upload these logs directly to it using the "inputStreamName" parameter.

Later you could delete this data source.

 

Boris