SOLVED

Webhook notifications stopped working around August 1st

Copper Contributor

Hello everyone,

 

around August 1st the (Incoming Webhooks suddenly stopped working, with the following result message:

Webhook message delivery failed with error: Microsoft Teams endpoint returned HTTP error 403 with ContextId tcid=4179142..,server=DB3PEPF0..,cv=9eBYWxGK5E../Q.0..

We sent a Post Request to 

https://outlook.office.com/webhook/c28f0bb1-001d..@b692a557../IncomingWebhook/../.. 

and got the following message (IDs and URLs obfuscated, I hope it doesn't prevent finding a solution):

> POST /webhook/c28f0bb1-001.../IncomingWebhook/84b0be0d1dc74b73../a5643ff9-8f19-.. HTTP/1.1
> Host: outlook.office.com
> User-Agent: insomnia/6.5.4
> Content-type: application/json
> Accept: */*
> Content-Length: 23
{
    "text": "Test 123"
}
* upload completely sent off: 23 out of 23 bytes
< HTTP/1.1 200 OK
< Cache-Control: no-cache
< Pragma: no-cache
< Content-Length: 185
< Content-Type: text/plain; charset=utf-8
< Expires: -1
< request-id: d6ac1e0b-2f6e-..
< X-CalculatedBETarget: VI1PR02...eurprd02.prod.outlook.com
< X-BackEndHttpStatus: 200
< X-AspNet-Version: 4.0.30319
< X-CafeServer: VI1PR07...EURPRD07.PROD.OUTLOOK.COM
< X-BEServer: VI1PR02..
< X-Powered-By: ASP.NET
< X-FEServer: VI1PR07..
< X-MSEdge-Ref: Ref A: 05BB3984BEB84.. Ref B: WAW01E.. Ref C: 2019-08-06T07:58:53Z
< Date: Tue, 06 Aug 2019 07:58:54 GMT

 

Were any changes made to how the webhooks work, so do we need to change anything or ist this a bug?

 

Thanks in advance and best regards,

Ines

3 Replies

@Ines_Trenz_HIRSCHTEC 

 

We experienced this as well. We also can't create new webhooks - pressing "add" to channels with the webhooks connector doesn't do anything, and trying to add incoming webhooks connector to new channels doesn't work - it doesn't seem to appear. 

@Ines_Trenz_HIRSCHTEC Same here - all existing incoming webhooks stopped working on August 1st. 

 

The response we're seeing back is the same as yours:

 

Webhook message delivery failed with error: Microsoft Teams endpoint returned HTTP error 403 with ContextId tcid=... 

But also unable to create a new web hook...

best response confirmed by Ines_Trenz_HIRSCHTEC (Copper Contributor)
Solution

Yesterday our IT admins were able to fix the issue after we did some more digging and found these posts:

https://techcommunity.microsoft.com/t5/Microsoft-Teams/Incoming-Webhook-MS-Teams-returned-HTTP-error...

https://techcommunity.microsoft.com/t5/Microsoft-Teams-Blog/New-admin-controls-for-apps-in-Teams/ba-...

 

With this info our admins found out all apps were blocked in the tenant settings (due to some Teams updates, I'm not 100% sure about this though), thus preventing using or adding webhooks. They solved the issue by allowing apps again and then waiting for the sync (which took around 2h, it seemed). Now we can add webhooks to channels again, unfortunately we had to manually update all webhook adresses we use for the messages to show up correctly.

 

I hope this helps you somehow in solving the issue!

 

Best regards,

Ines

1 best response

Accepted Solutions
best response confirmed by Ines_Trenz_HIRSCHTEC (Copper Contributor)
Solution

Yesterday our IT admins were able to fix the issue after we did some more digging and found these posts:

https://techcommunity.microsoft.com/t5/Microsoft-Teams/Incoming-Webhook-MS-Teams-returned-HTTP-error...

https://techcommunity.microsoft.com/t5/Microsoft-Teams-Blog/New-admin-controls-for-apps-in-Teams/ba-...

 

With this info our admins found out all apps were blocked in the tenant settings (due to some Teams updates, I'm not 100% sure about this though), thus preventing using or adding webhooks. They solved the issue by allowing apps again and then waiting for the sync (which took around 2h, it seemed). Now we can add webhooks to channels again, unfortunately we had to manually update all webhook adresses we use for the messages to show up correctly.

 

I hope this helps you somehow in solving the issue!

 

Best regards,

Ines

View solution in original post