As per https://stackoverflow.com/questions/65441392/does-outlook-web-app-support-dark-mode-changes-for-manifest-xml. It is difficult and limiting for add-ins to provide a single set of icons that...
Currently dealing with the same issue. Really frustrating that we want to match the dark mode Black theme set by Excel but we're not allowed to. Seems very odd.
The default black for the icons in Colorful or White theme is almost the same color as the background in the Black theme, making these icons invisible.
On Mac the black color changes to the light color automatically which solves this issue, but it doesn't happen on Windows and we're not allowed to set a custom icon or an SVG...