SmartScreen is cause of consistent page/download delays (unitedstates.smartscreen.microsoft.com)

Copper Contributor

Version

Version: 103.0.1264.44 (Official build) (64-bit)

 

Problem

SmartScreen at unitedstates.smartscreen.microsoft.com is accessed for every page load for some pages (perhaps less popular domains, such as example.com (for real)), triggering a page load delay that does not show up in Network timing.

 

Demonstration

Slow, ~1sec delay due to SmartScreen: https://www.dropbox.com/s/wf0ch5bf4b52ox9/Speed%20-%20Edge%20-%20Slow.gif?dl=0

 

Chrome, immediate page load (No SmartScreen): https://www.dropbox.com/s/b85e50f4w7znqfp/Speed%20-%20Chrome%20-%20Fast.gif?dl=0

 

Analysis

I noticed that:

1. This only happens on some domains (eg, my website and also example.com -- perhaps its domains without a lot of reputation?)

2. It happens repeatedly, its slow every page click, and I can see the HTTPS API call to SmartScreen each time

3. I'm based in Australia but the region SmartScreen API hits is always unitedstates.smartscreen.microsoft.com

 

Solution

Geo-locate SmartScreen? Don't hit it for every request for the same domain?

 

Thanks,

Andrew

10 Replies
I want to see if this is due to a particular configuration you've got in your Edge install. Can you replicate these results in a brand new install of a different channel like Beta or Dev from www.microsoftedgeinsider.com?

@josh_bodner Thanks Josh; yes I can replicate with Canary build. I will note Edge is managed by InTune so perhaps its an organisational policy that is enforcing the constant checking?

 

Kind regards,

Andrew

@AndrewAus 

Could it be related to the new SmartScreen engine - which is the default engine since Edge version 103 if you have not configured the related browser policy setting "NewSmartScreenLibraryEnabled" to disabled?

 

Compare to

Microsoft Edge Browser Policy Documentation | Microsoft Docs

<...>

Allows the Microsoft Edge browser to load new SmartScreen library (libSmartScreenN) for any SmartScreen checks on site URLs or application downloads.

If you enable or don't configure this policy, Microsoft Edge will use the new SmartScreen library (libSmartScreenN).

If you disable this policy, Microsoft Edge will use the old SmartScreen library (libSmartScreen).

Before Microsoft Edge version 103, if you don't configure this policy, Microsoft Edge will use the old SmartScreen library (libSmartScreen).

The NewSmartScreenLibraryEnabled policy will allow enterprise customers to continue using the legacy version of the library until it’s deprecated in Microsoft Edge version 105.

<...>

 

Unfortunately we experience issues with the new SmartScreen engine on our Windows Server 2016 based systems, too, whereas the new engine is properly working on our Windows 10 based clients, which seems a little bit strange to me.

 

You can easily cross check by browsing the microsoft test page for SmartScreen:

https://demo.smartscreen.msft.net/

 

In our case, the old engine is properly working, the new engine unfortunately is not (checked by debug trace):

 

The old engine works without delays and properly blocks the dangerous actions (shows the desired behavior on the test page), the new engine does not, internally throws an error (no response) and just allows the action after the internal timeout:

 

OLD engine (SS_ERROR_TYPE_NONE, decision=kExploit):

{"args":{},"cat":"SmartScreen","id":"0x2f41dd80bc3778","name":"SendRequestProxy","ph":"b","pid":5712,"scope":"SmartScreen","tid":4192,"ts":93521246206},
{"args":{"server_uri":https://nav.smartscreen.microsoft.com/api/browser/edge/navigate/2/sync},"cat":"SmartScreen","id":"0x2f41dd80bc3778","name":"SendRequestProxy","ph":"n","pid":5712,"scope":"SmartScreen","tid":4192,"ts":93521246207},
{"args":{"request":"{\"config\":{\"device\":{\"appControl\":{\"level\":\"anywhere\"},\"appReputation\":{\"enforcedByPolicy\":false,\"level\":\"warn\"},\"pua\":null},\"user\":{\"uriReputation\":{\"enforcedByPolicy\":false,\"level\":\"warn\"}}},\"correlationId\":\"06931F68-AAF3-4E6F-8102-A319FC3C86AA\",\"destination\":{\"ip\":\"23.99.0.12\",\"uri\":\https://demo.smartscreen.msft.net/other/exploit.html\},\"forceServiceDetermination\":false,\"identity\":{\"caller\":{\"locale\":\"en-US\",\"name\":\"anaheim\",\"process\":null,\"version\":\"103.0.1264.49 (Official build) \"},\"client\":{\"data\":{\"customSettings\":\"F95BA787499AB4FA9EFFF472CE383A14\",\"customSynchronousLookupUris\":\"0\",\"edgeSettings\":\"2.0-2f9188b68640dbf72295f9083a21d674a314721ef06f82db281cbcb052ff8ec1\",\"synchronousLookupUris\":\"637928652443953878\",\"topTraffic\":\"637811103879324684\"},\"version\":\"281479416053761\"},\"device\":{\"architecture\":9,\"browser\":{\"internetExplorer\":\"9.11.14393.0\"},\"cloudSku\":false,\"customId\":null,\"enterprise\":null,\"family\":9,\"id\":null,\"locale\":\"en-US\",\"netJoinStatus\":3,\"onlineIdTicket\":\"t=GwAWAd9tBAAUa9NnKxE2EcNyf3boVjyC/9kKG5YOZgAAELAeEOMJS3E3tCal2CDDdQPgAAjdYXHl+RDzDkifveluYGXfVUwUY+hASt1HKHZEiiRCxJvSAthVIlm44A7NYRCpepRnfk4ExYyHzlkI937HAesH2EMohRgGHiVZElouy9UT3qiotJBGNlUhKR9ebkD1anWNxhRs7vJCGLcTpgsW7mRRzCRRzHFryF9HICZSMH4t04BqCFhcwJ5iGu2Knh5nSCxISUvvpp+BLkZ7arHFsub8/M8EZGluZqWRy24Zig2Fr5lRr0uLYRIoPprM81N8yhavbcsqmkx/MAhnHdvf+YCRKfg7GLKN4SIZBLZrlnLIIAE=&p=\",\"osVersion\":\"10.0.14393.5192.rs1_release\"},\"user\":{\"locale\":\"en-US\"}},\"referrer\":{\"ip\":\"23.99.0.12\",\"uri\":\https://demo.smartscreen.msft.net/\},\"serverContext\":null,\"signals\":null,\"synchronous\":true,\"systemSettings\":{\"battery\":null,\"network\":null},\"type\":\"top\",\"userAgent\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36 Edg/103.0.1264.49\"}"},"cat":"SmartScreen","id":"0x2f41dd80bc3778","name":"SendRequestProxy","ph":"n","pid":5712,"scope":"SmartScreen","tid":4192,"ts":93521246360},


{"args":{"response":"<Res>\r\n <S>0</S>\r\n <I>06931F68-AAF3-4E6F-8102-A319FC3C86AA</I>\r\n <W>637928616292506341</W>\r\n <R>\r\n <Rs>\r\n <M>ZGVtby5zbWFydHNjcmVlbi5tc2Z0Lm5ldC9vdGhlci9leHBsb2l0Lmh0bWw=</M>\r\n <C>XPLT:100:0:0</C>\r\n <T>7200</T>\r\n <L1>9db8fb1a</L1>\r\n <X>0</X>\r\n </Rs>\r\n </R>\r\n</Res>"},"cat":"SmartScreen","id":"0x2f41dd80bc3657","name":"SendRequestProxy","ph":"n","pid":5712,"scope":"SmartScreen","tid":4192,"ts":93521446375},
{"args":{},"cat":"SmartScreen","id":"0x2f41dd80bc3657","name":"SendRequestProxy","ph":"e","pid":5712,"scope":"SmartScreen","tid":4192,"ts":93521446379},
{"args":{"response":"{\"actions\":[{\"$type\":\"cache\",\"key\":{\"uri\":\"demo.smartscreen.msft.net/other/exploit.html\",\"inheritance\":\"none\"},\"maxAge\":100800000000,\"serverContext\":\"1;f94c025f-7523-6972-b613-ce2c246c55ce;UNKN:100;0.01\",\"responseCategory\":\"Allowed\",\"result\":{\"$type\":\"evaluate\"}}],\"serverContext\":\"1;f94c025f-7523-6972-b613-ce2c246c55ce;UNKN:100;0.01\",\"result\":{\"$type\":\"evaluate\",\"responseCategory\":\"Allowed\",\"serverContext\":\"1;f94c025f-7523-6972-b613-ce2c246c55ce;UNKN:100;0.01\",\"feedbackUrl\":\https://feedback.smartscreen.microsoft.com/feedback.aspx?v=6&t=12553&result=none&type=top&ur=UNKN%3A...}}"},"cat":"SmartScreen","id":"0x2f41dd80bc3778","name":"SendRequestProxy","ph":"n","pid":5712,"scope":"SmartScreen","tid":4192,"ts":93521446591},
{"args":{},"cat":"SmartScreen","id":"0x2f41dd80bc3778","name":"SendRequestProxy","ph":"e","pid":5712,"scope":"SmartScreen","tid":4192,"ts":93521446594},
{"args":{"result_decision":"decision=kExploit, \nurl:https://demo.smartscreen.msft.net/other/exploit.html","step":"OnComplete"},"cat":"SmartScreen","id":"0x2f41dd80bc2575","name":"CheckUrlsForNavigation","ph":"T","pid":5712,"scope":"CheckUrlsForNavigation","tid":4192,"ts":93521446607},
{"args":{"error":"SS_ERROR_TYPE_NONE","step":"SmartScreenError"},"cat":"SmartScreen","id":"0x2f41dd80bc2575","name":"CheckUrlsForNavigation","ph":"T","pid":5712,"scope":"CheckUrlsForNavigation","tid":4192,"ts":93521446612},

 

 

NEW engine: (SS_ERROR_TYPE_CLIENT_CRITICAL, decision=kAllow ?!)

{"args":{},"cat":"SmartScreen","id":"0x2f41dd8e282986","name":"SendRequestProxy","ph":"b","pid":576,"scope":"SmartScreen","tid":5024,"ts":93746423411},
{"args":{"server_uri":https://nav.smartscreen.microsoft.com/api/browser/edge/ssrs/3?MSURS-Client-Key=+c8/QC27iQm6lpyb8mwlo...},"cat":"SmartScreen","id":"0x2f41dd8e282986","name":"SendRequestProxy","ph":"n","pid":576,"scope":"SmartScreen","tid":5024,"ts":93746423413},
{"args":{"request":"<Rep v=\"3\"><G>{379BDC39-D58D-44AA-986B-FD2CBFFA75A6}</G><ORG></ORG><SEN></SEN><ID>{3577E9B1-CCA1-4CA3-8C69-B6C27D3B289C}</ID><C>Anaheim</C><OS>10.0.18363.720</OS><L>en-US</L><W>0</W><I>{381ddd1e-e600-42de-94ed-8c34bf73f16d}</I><R><Rq><U>aHR0cHM6Ly9kZW1vLnNtYXJ0c2NyZWVuLm1zZnQubmV0L290aGVyL2V4cGxvaXQuaHRtbA==</U><T>T</T></Rq></R></Rep>"},"cat":"SmartScreen","id":"0x2f41dd8e282986","name":"SendRequestProxy","ph":"n","pid":576,"scope":"SmartScreen","tid":5024,"ts":93746423438},

{"args":{"response":"NO RESPONSE"},"cat":"SmartScreen","id":"0x2f41dd8d478200","name":"SendRequestProxy","ph":"n","pid":576,"scope":"SmartScreen","tid":5024,"ts":93752716811},
{"args":{"result_decision":"decision=kAllow, \nurl:https://ntp.msn.com/edge/ntp?locale=en-US&title=New%20tab&dsp=1&sp=Bing&prerender=1","step":"OnComplete"},"cat":"SmartScreen","id":"0x2f41dd8d477a47","name":"CheckUrlsForNavigation","ph":"T","pid":576,"scope":"CheckUrlsForNavigation","tid":5024,"ts":93752716902},
{"args":{"error":"SS_ERROR_TYPE_CLIENT_CRITICAL","step":"SmartScreenError"},"cat":"SmartScreen","id":"0x2f41dd8d477a47","name":"CheckUrlsForNavigation","ph":"T","pid":576,"scope":"CheckUrlsForNavigation","tid":5024,"ts":93752716910},

Unfortunately, the latest Microsoft Edge update 103.0.1264.62 breaks the "old" SmartScreen engine on all our systems (64-bit, Windows Server 2016 / Windows 10 and 11 clients), the new engine is still not working properly on Windows Server 2016... Therefore, none of the two Edge integrated SmartScreen engines currently work fine on Server 2016 :(...

What are beta releases for when obviously untested code ends up in a stable channel?!

@CWallner @josh_bodner Unfortunately the issue persists, its a real drag on productivity.

 

I've tried whitelisting some corporate domains but it's not working; I submitted this issue via the feedback system too in Edge, I'm hoping its in a bug tracker somewhere? :'(

Yes, the team is definitely aware and working on it; we just don't have the capacity to reply directly to every bug report that gets filed!
Really appreciate it and understand, thank you!
Any news about this case? It was fixed in 106.0.1370.47 ?
Unfortunately not, I'm on Version 106.0.1370.52 (Official build) (64-bit) and its still an issue.

@AndrewAus 

 

Has this problem already been resolved?
I am having trouble facing a very similar situation.