SOLVED

Logs uploading but Cloud App discovery dashboard not updating

%3CLINGO-SUB%20id%3D%22lingo-sub-228958%22%20slang%3D%22en-US%22%3ELogs%20uploading%20but%20Cloud%20App%20discovery%20dashboard%20not%20updating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-228958%22%20slang%3D%22en-US%22%3E%3CP%3EHi%2C%20I%20have%20implemented%26nbsp%3Ba%20log%20collector%20to%20send%20logs%20from%20my%20TMG%20server%20to%20CAS.%20The%20log%20collector%20is%20called%20TMG-02.%20The%20logs%20copied%2C%20using%20FTP%2C%20to%20the%20log%20collector%20which%20then%20uploads%20them%20to%20CAS.%20The%20first%20run%20of%20this%202%20files%20were%20uploaded%20and%20I%20could%20see%20these%20where%20successfully%20parser%20by%20viewing%20the%20governance%20log.%20After%20probably%2020%20minutes%20the%20discovery%20dashboard%20populated.%20However%2C%20if%20I%20change%20the%20scope%20from%20Global%20to%20just%20view%20TMG-02%20it%20is%20blank%2C%20which%20doesn't%20see%20right%20to%20me.%20Also%2C%20when%20the%20same%20files%2C%20which%20now%20contain%20more%20data%20are%20uploaded%20the%20Cloud%20App%20discovery%20dashboard%20doesn't%20update%20even%20though%20I%20can%20see%20they%20have%20been%20successfully%20uploaded%20and%20parser.%20Also%2C%20if%20I%20create%20a%20snapshot%20report%20from%20these%20files%20I%20see%20the%20information%20that%20I%20expect.%3C%2FP%3E%3CP%3EWhy%20am%20I%20not%20seeing%20any%20discovery%20data%20if%20I%20just%20select%20the%20log%20collector%20of%20TMG-02%20and%20more%20importantly%20why%20is%20the%20discovery%20dashboard%20updating%20with%20the%20information%20from%20the%20updated%20logs%3F%3C%2FP%3E%3CP%3ERegards%2C%3C%2FP%3E%3CP%3EStuart%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-228958%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3ECloud%20App%20Security%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-242049%22%20slang%3D%22en-US%22%3ERe%3A%20Logs%20uploading%20but%20Cloud%20App%20discovery%20dashboard%20not%20updating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-242049%22%20slang%3D%22en-US%22%3EIts%20a%20temporary%20error%20caused%20by%20reaching%20the%20API%20limits%20of%20the%20service.%20It%20resolves%20itself%20automatically%20and%20doesnt%20impact%20any%20functionality.%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-241873%22%20slang%3D%22en-US%22%3ERe%3A%20Logs%20uploading%20but%20Cloud%20App%20discovery%20dashboard%20not%20updating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-241873%22%20slang%3D%22en-US%22%3E%3CP%3EThanks%20for%20clearing%20this%20up%20for%20me.%20The%20reason%20that%20I%20asked%20this%20is%20that%20I%20have%207%20connected%20apps%20and%202%20of%20these%20are%20OneDrive%20for%20Business%20and%20SharePoint%20Online%20and%20these%20keep%20showing%20Connection%20Error%20(the%20others%20are%20fine).%20When%20I%20look%20at%20the%20error%20it%20is%20the%20same%20for%20both%20which%20is%3A%26nbsp%3B%3C%2FP%3E%3CDIV%3EGet%20Users%3A%20Failed%20fetching%20users%20from%20%2F%3A%3CBR%20%2F%3EHttpRequestFailure%3A%20Server%20returned%3A%20429%3CBR%20%2F%3E%5BContent-Type%3A%20text%2Fplain%3B%20charset%3Dutf-8%2C%20Retry-After%3A%2030%2C%20Server%3A%20Microsoft-IIS%2F8.5%2C%20SPRequestGuid%3A%2041638b9e-2001-0000-2468-b00fd299d6b4%2C%20request-id%3A%2041638b9e-2001-0000-2468-b00fd299d6b4%2C%20MS-CV%3A%20notjQQEgAAAkaLAP0pnWtA.0%2C%20Strict-Transport-Security%3A%20max-age%3D31536000%2C%20SPRequestDuration%3A%2032%2C%20SPIisLatency%3A%200%2C%20X-Powered-By%3A%20ASP.NET%2C%20MicrosoftSharePointTeamServices%3A%2016.0.0.8029%2C%20X-Content-Type-Options%3A%20nosniff%2C%20X-MS-InvokeApp%3A%201%3B%20RequireReadOnly%2C%20P3P%3A%20CP%3D%22ALL%20IND%20DSP%20COR%20ADM%20CONo%20CUR%20CUSo%20IVAo%20IVDo%20PSA%20PSD%20TAI%20TELo%20OUR%20SAMo%20CNT%20COM%20INT%20NAV%20ONL%20PHY%20PRE%20PUR%20UNI%22%2C%20Date%3A%20Tue%2C%2004%20Sep%202018%2010%3A01%3A00%20GMT%2C%20Content-Length%3A%2021%5D%3C%2FDIV%3E%3CDIV%3E429%20TOO%20MANY%20REQUESTS%3CBR%20%2F%3EGet%20Users%3A%20Success%3CBR%20%2F%3EGet%20Files%3A%20Failed%20listing%20site%20collection%20%3CA%20href%3D%22https%3A%2F%2Fcomputacenterems-my.sharepoint.com%2F%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fcomputacenterems-my.sharepoint.com%2F%3C%2FA%3E%20(path%20%2F)%3A%3CBR%20%2F%3EUser%20email%3A%20%2C%20Error%3A%20HttpRequestFailure%3A%20Server%20returned%3A%20429%3CBR%20%2F%3E%5BContent-Type%3A%20text%2Fplain%3B%20charset%3Dutf-8%2C%20Retry-After%3A%2030%2C%20Server%3A%20Microsoft-IIS%2F8.5%2C%20SPRequestGuid%3A%2076638b9e-0002-0000-30d2-2b61bc8ac573%2C%20request-id%3A%2076638b9e-0002-0000-30d2-2b61bc8ac573%2C%20MS-CV%3A%20notjdgIAAAAw0ithvIrFcw.0%2C%20Strict-Transport-Security%3A%20max-age%3D31536000%2C%20SPRequestDuration%3A%2036%2C%20SPIisLatency%3A%200%2C%20X-Powered-By%3A%20ASP.NET%2C%20MicrosoftSharePointTeamServices%3A%2016.0.0.8029%2C%20X-Content-Type-Options%3A%20nosniff%2C%20X-MS-InvokeApp%3A%201%3B%20RequireReadOnly%2C%20P3P%3A%20CP%3D%22ALL%20IND%20DSP%20COR%20ADM%20CONo%20CUR%20CUSo%20IVAo%20IVDo%20PSA%20PSD%20TAI%20TELo%20OUR%20SAMo%20CNT%20COM%20INT%20NAV%20ONL%20PHY%20PRE%20PUR%20UNI%22%2C%20Date%3A%20Tue%2C%2004%20Sep%202018%2010%3A04%3A38%20GMT%2C%20Content-Length%3A%2021%5D%3C%2FDIV%3E%3CDIV%3E429%20TOO%20MANY%20REQUESTS.%3C%2FDIV%3E%3CDIV%3E%26nbsp%3B%3C%2FDIV%3E%3CDIV%3EI've%20attached%20a%20screenshot%20of%20the%20connection%20error%20for%20OneDrive%20for%20Business%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-241703%22%20slang%3D%22en-US%22%3ERe%3A%20Logs%20uploading%20but%20Cloud%20App%20discovery%20dashboard%20not%20updating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-241703%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20Stuart%2C%3C%2FP%3E%0A%3CP%3EThe%20connection%20to%20Office%20365%20is%20created%20using%20app%20credentials%20and%20not%20the%20user%20ones.%20The%20user%20is%20only%20needed%20to%20verify%20you%20actually%20have%20the%20required%20permissions%20to%20initiate%20the%20connection%2C%20once%20established%20the%20connection%20is%20independent.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ERegards%2C%3C%2FP%3E%0A%3CP%3EDima%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-241681%22%20slang%3D%22en-US%22%3ERe%3A%20Logs%20uploading%20but%20Cloud%20App%20discovery%20dashboard%20not%20updating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-241681%22%20slang%3D%22en-US%22%3E%3CP%3EThanks%20Danny%2C%3C%2FP%3E%3CP%3EI%20have%20sent%20them%20an%20e-mail.%3C%2FP%3E%3CP%3EWould%20you%20mind%20answering%20a%20couple%20of%20other%20questions%20that%20I%20have%20as%20I'm%20currently%20evaluating%20CAS%26nbsp%3B%20with%20a%20view%20to%20recommending%20it%20to%20our%20customers%3F%3C%2FP%3E%3CP%3EMy%20questions%20are%3A%3C%2FP%3E%3CP%3E1.%20Is%20there%20anyway%20of%20changing%20the%20account%20used%20to%20connect%20to%20Office%20365%2C%20SharePoint%20and%20OneDrive%20as%20I%20connected%20using%20my%20account%20and%20I%20would%20have%20thought%20that%20you'd%20be%20better%20having%20a%20dedicated%20account%20as%20my%20password%20will%20change%20and%20also%20I%20have%20MFA%20enabled.%20Is%20the%20account%20that%20you%20are%20logged%20in%20with%20used%20ongoing%20for%20connecting%20to%20these%20Microsoft%20applications.%20I%20see%20that%20other%20apps%20you%20can%20specify%20and%20change%20the%20token.%3C%2FP%3E%3CP%3E2.%20I%20am%20just%20looking%20at%26nbsp%3BConditional%20Access%20App%20Control%20and%20I%20see%20that%26nbsp%3BOffice%20365%20applications%20are%20not%20configured%20with%20SAML%20so%20they%20are%20not%20currently%20supported.%20So%20does%20this%20mean%20that%20CAS%20cannot%20be%20used%20to%20force%20encryption%20on%20downloaded%20files%20or%20to%20even%20block%20downloads%20on%20unmanaged%20devices%20from%20OneDrive%20for%20Business%20and%20SharePoint%20Online%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks%20in%20advance%2C%3C%2FP%3E%3CP%3EStuart%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-241450%22%20slang%3D%22en-US%22%3ERe%3A%20Logs%20uploading%20but%20Cloud%20App%20discovery%20dashboard%20not%20updating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-241450%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20Stuart%2C%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EAPI%20connector%20can%20be%20disabled%20by%20our%20support%20team.%3C%2FP%3E%0A%3CP%3EPlease%20contact%20them%20in%26nbsp%3Bceilsupport%40microsoft.com.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EDanny.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-241435%22%20slang%3D%22en-US%22%3ERe%3A%20Logs%20uploading%20but%20Cloud%20App%20discovery%20dashboard%20not%20updating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-241435%22%20slang%3D%22en-US%22%3E%3CP%3EThanks%20Danny%20that%20was%20exactly%20what%20my%20problem%20was.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EAs%20a%20side%20note%2C%20do%20you%20know%20if%20there%20is%20a%20way%20of%20removing%20a%20connected%20app%20in%20the%20portal%3F%20I%20have%20accidentally%20connected%20to%20the%20same%20Okta%20organization%20twice%20and%20want%20to%20remove%20one%20of%20the%20connections%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-234056%22%20slang%3D%22en-US%22%3ERe%3A%20Logs%20uploading%20but%20Cloud%20App%20discovery%20dashboard%20not%20updating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-234056%22%20slang%3D%22en-US%22%3EHi%20Danny%2C%3CBR%20%2F%3EThanks%20for%20the%20reply.%20I%E2%80%99m%20out%20of%20the%20office%20at%20the%20moment%20but%20I%E2%80%99ll%20check%20when%20I%20get%20back.%20I%20had%20only%20waited%20a%20maximum%20of%203-4%20hours%20so%20I%20guess%20it%20will%20be%20updated%20now.%3CBR%20%2F%3EThanks%20again%2C%3CBR%20%2F%3EStuart%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-234022%22%20slang%3D%22en-US%22%3ERe%3A%20Logs%20uploading%20but%20Cloud%20App%20discovery%20dashboard%20not%20updating%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-234022%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20Stuart%2C%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EAs%20indicated%20in%20MCAS%20documentation%20(%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fcloud-app-security%2Fset-up-cloud-discovery%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fcloud-app-security%2Fset-up-cloud-discovery%3C%2FA%3E)%2C%20you%20can%20see%20that%20continuous%20report%20data%20is%20analyzed%20twice%20a%20day%2C%20and%20might%20take%20some%20time%20to%20be%20populated%20in%20case%20of%20newly%20created%20reports.%20Could%20you%20share%20some%20details%20on%20how%20long%20did%20you%20wait%20before%20checking%20the%20data%3F%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThanks%2C%3C%2FP%3E%0A%3CP%3EDanny.%3C%2FP%3E%3C%2FLINGO-BODY%3E
Occasional Contributor

Hi, I have implemented a log collector to send logs from my TMG server to CAS. The log collector is called TMG-02. The logs copied, using FTP, to the log collector which then uploads them to CAS. The first run of this 2 files were uploaded and I could see these where successfully parser by viewing the governance log. After probably 20 minutes the discovery dashboard populated. However, if I change the scope from Global to just view TMG-02 it is blank, which doesn't see right to me. Also, when the same files, which now contain more data are uploaded the Cloud App discovery dashboard doesn't update even though I can see they have been successfully uploaded and parser. Also, if I create a snapshot report from these files I see the information that I expect.

Why am I not seeing any discovery data if I just select the log collector of TMG-02 and more importantly why is the discovery dashboard updating with the information from the updated logs?

Regards,

Stuart

8 Replies

Hi Stuart,

 

As indicated in MCAS documentation (https://docs.microsoft.com/en-us/cloud-app-security/set-up-cloud-discovery), you can see that continuous report data is analyzed twice a day, and might take some time to be populated in case of newly created reports. Could you share some details on how long did you wait before checking the data?

 

Thanks,

Danny.

Best Response confirmed by stuart townsend (Occasional Contributor)
Solution
Hi Danny,
Thanks for the reply. I’m out of the office at the moment but I’ll check when I get back. I had only waited a maximum of 3-4 hours so I guess it will be updated now.
Thanks again,
Stuart

Thanks Danny that was exactly what my problem was.

 

As a side note, do you know if there is a way of removing a connected app in the portal? I have accidentally connected to the same Okta organization twice and want to remove one of the connections?

Hi Stuart,

 

API connector can be disabled by our support team.

Please contact them in ceilsupport@microsoft.com.

 

Danny.

 

Thanks Danny,

I have sent them an e-mail.

Would you mind answering a couple of other questions that I have as I'm currently evaluating CAS  with a view to recommending it to our customers?

My questions are:

1. Is there anyway of changing the account used to connect to Office 365, SharePoint and OneDrive as I connected using my account and I would have thought that you'd be better having a dedicated account as my password will change and also I have MFA enabled. Is the account that you are logged in with used ongoing for connecting to these Microsoft applications. I see that other apps you can specify and change the token.

2. I am just looking at Conditional Access App Control and I see that Office 365 applications are not configured with SAML so they are not currently supported. So does this mean that CAS cannot be used to force encryption on downloaded files or to even block downloads on unmanaged devices from OneDrive for Business and SharePoint Online?

 

Thanks in advance,

Stuart

Hi Stuart,

The connection to Office 365 is created using app credentials and not the user ones. The user is only needed to verify you actually have the required permissions to initiate the connection, once established the connection is independent.

 

Regards,

Dima

Thanks for clearing this up for me. The reason that I asked this is that I have 7 connected apps and 2 of these are OneDrive for Business and SharePoint Online and these keep showing Connection Error (the others are fine). When I look at the error it is the same for both which is: 

Get Users: Failed fetching users from /:
HttpRequestFailure: Server returned: 429
[Content-Type: text/plain; charset=utf-8, Retry-After: 30, Server: Microsoft-IIS/8.5, SPRequestGuid: 41638b9e-2001-0000-2468-b00fd299d6b4, request-id: 41638b9e-2001-0000-2468-b00fd299d6b4, MS-CV: notjQQEgAAAkaLAP0pnWtA.0, Strict-Transport-Security: max-age=31536000, SPRequestDuration: 32, SPIisLatency: 0, X-Powered-By: ASP.NET, MicrosoftSharePointTeamServices: 16.0.0.8029, X-Content-Type-Options: nosniff, X-MS-InvokeApp: 1; RequireReadOnly, P3P: CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI", Date: Tue, 04 Sep 2018 10:01:00 GMT, Content-Length: 21]
429 TOO MANY REQUESTS
Get Users: Success
Get Files: Failed listing site collection https://computacenterems-my.sharepoint.com/ (path /):
User email: , Error: HttpRequestFailure: Server returned: 429
[Content-Type: text/plain; charset=utf-8, Retry-After: 30, Server: Microsoft-IIS/8.5, SPRequestGuid: 76638b9e-0002-0000-30d2-2b61bc8ac573, request-id: 76638b9e-0002-0000-30d2-2b61bc8ac573, MS-CV: notjdgIAAAAw0ithvIrFcw.0, Strict-Transport-Security: max-age=31536000, SPRequestDuration: 36, SPIisLatency: 0, X-Powered-By: ASP.NET, MicrosoftSharePointTeamServices: 16.0.0.8029, X-Content-Type-Options: nosniff, X-MS-InvokeApp: 1; RequireReadOnly, P3P: CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI", Date: Tue, 04 Sep 2018 10:04:38 GMT, Content-Length: 21]
429 TOO MANY REQUESTS.
 
I've attached a screenshot of the connection error for OneDrive for Business
Its a temporary error caused by reaching the API limits of the service. It resolves itself automatically and doesnt impact any functionality.