Creating an Out of Office for alias added to a primary mail in office 365

A little background on the current setup.

A business decision was made at one of the departments at my current job to change the name of their email address. I.E. Old email address was removed for privacy reasons to New email address was removed for privacy reasons for their own reasons.

The issue isn't with changing the name, which is easily doable using Office365 Admin but they want to add the alias of Oldname to the New email address removed for privacy reasons again, not an issue.

The question

Is it possible to somehow, add an Out of Office or Auto Reply to the Old email address removed for privacy reasons after it becomes an alias? 

They want to still have the ability for people who are sending emails to Old email addresses removed for privacy reasons to not have their emails bounced, instead they would get an auto-reply thanking them for sending their email, advising them that the email has been forwarded to New email address removed for privacy reasons and reminding them that there's a new email address to send their future emails to.

If there's a way to do it with a rule or something else, I would rather do that.