Advanced artificial intelligence" I guess our aspirations for "hard AI" are becoming a little less realistic as the years go by...
As much as this is an incredible technology, I believe it carries with it many ethical challenges. The biggest one is that, when you enter into a conversation with a robotic assistant, you are tricked into speaking to someone who completely lacks a sense of empathy.
I can think of a number of situations in which this would be problematic. For instance, Google Duplex calls a hair salon just before closing. A woman picks up at the salon. She is the only employee left on the premise, and begins the process of booking an appointment. Two minutes into the call, she says, "There's a man with a mask outside. Oh my god, he's coming in!" The phone line disconnects abruptly. What does the Google Assistant do? What would a human do?
Ok, hopefully that's incredibly rare, but what about the more common niceties of interacting with another human? What about when you talk to someone on the phone, and you just get that sense that you're talking to someone very charming, or polite, or funny? What about when you hear a beautiful voice or accent on the other end of the phone, and perhaps you get into a conversation about where that person comes from? What about recognizing the voice on the other end of your phone as that guy you met at the party last weekend?
True, the process of booking an appointment is relatively simple, and most of the time, it is a straightforward transaction process that doesn't require empathy or nuance.
However, I think the ethical issues also present themselves on a larger scale when you start treating humans as simple agents who recite a script or book appointments and are devoid of any other aspect of their humanity. It creates a society which is deeply un-empathic towards one another, perhaps even sociopath. This kind of technology is impressive, and I'm not even saying it's not useful, but I think that usefulness comes at a cost.