Random timeouts at webhook connectors to MS Teams

Copper Contributor

In my company we are using Elastic Watcher to monitor various system indicators. These watchers are sending notifications to our MS Teams via a webhook (https://<mycompany>.webhook.office.com:443/<channel-uri>). Most the time this works perfectly, but every now and then we are getting a socket exception ("Connection Timeout" or "Connection Reset"). Teams does not respond with a HTTP error code, but this is an exception on the sending system (Elastic).

 

The messages are far below 10 KB in size.

 

Unfortunately I don't have any more information, so it's hard to identify the root cause. But for me this soundy like an issue on the receiver side.

 

I appreciate any hints how to identify the cause and solve the problem.

5 Replies

@czger_tuito - Could you please share the request details along with tenant id and timestamp when the request is failing? Also please attach network trace/log.

@czger_tuito - Could you please share the request details along with tenant id and timestamp when the request is failing? Also please attach network trace/log.

@Prasad_Das-MSFT- Sorry for the late reply. Unfortunately I do not have any of this data. But our network admins are investigating if that problem is possibly caused on our side. I'll keep you on the track.

@czger_tuito 
we are having the same problem and we are also investigating. I don't think that its related to our networkinfrastructure; and the messages are also very small

if you find something please let me know ;)

@Thore_bechtle 

We didn't find any solution but setting the "connection_timeout" for the webhook action to 90 seconds.

https://www.elastic.co/guide/en/elasticsearch/reference/current/actions-webhook.html#_webhook_action...

At some point (I don't know details) this can be set globally in the Elastic config.

HTH