365 Outlook - exclude HTML email signature from calendar invite replies

Hi,

The formatting of my company wide HTML email signature breaks when someone replies to a calendar invite, e.g. accepts or rejects an invite. Does anyone know of a way to remove the HTML signature from calendar invite replies only?