My comment on having awkward situation during subsequent live communication was just one of the more practical examples of this being problematic. But on a personal emotional level this feels completely off putting. People don't like talking to robots, especially being tricked to do so. This is tricking people to believe they are having nice conversation with their service provider. If they somehow learn that later or see similar phrases coming from different people it will leave a very bad taste for many customers. Google Duplex and its backlash comes to mind. And i don't understand why do you need to automate that much. I can see use in AI giving you a snippet of info about this customer's order, so you can paste it into your email or hitting OK button to quickly shoot a confirmation email. But not automating everything and especially social interactions. For me as IT person it is already looking creepy. I expect general public be even more horrified and disgusted with such developments.