How to incorporate special characters (like umlauts) into CSV files and maintain the characters?

We have a contact export we use that has many names in it, and we now have several names with umlaut characters in them (Ü). When it comes into the export CSV file, the name looks like this: DÓ§rte when it should be Dörte. Any thoughts on how to fix this?

Manually editing the CSV file every time we export is not an ideal solution; as we get more names in the file with special characters, we will be more likely to miss the manual edit.