At this year’s annual re:Mars conference, Amazon unveiled perhaps the scariest technology I’ve ever heard: an Alexa skill that can mimic other people’s voices.
On the surface, it doesn’t look too bad, right? The geek in me is already thinking about how I could feel like Tony Stark or Luke Skywalker with my own personal Paul Bettany speaker or Alexa-voiced Anthony Daniels (the voices of JARVIS from the MCU and C3-P0 from Star Wars, respectively). ).
But rather than going that celebrity route, Amazon Alexa Senior Vice President Rohit Prasad pitched the idea alongside a clip of a child being read by Alexa using the voice of her recently deceased grandfather. According to Engadget (opens in new tab)an Amazon spokesperson explained that this clip could be recreated using “just one minute of audio” from the person Alexa is impersonating.
The scene was likely meant to inspire heartbreaking feelings at seeing how Amazon technology helps a child process their grief, but it could just as easily have been a scene from Black Mirror. In fact, the Be Right Back episode has a virtually identical premise to it.
Be Right Back follows Domhnall Gleeson’s Ash, who is killed in a car accident, and, after discovering she is pregnant, Hayley Atwell’s Martha must experiment with new technology that allows her to communicate with an artificial intelligence mimicking Ash. It’s terrifying and definitely not a preview of anything good.
In his explanation of the Alexa tool, Rohit Prasad said that while it doesn’t take away the pain of loss, it “can definitely make the memories last” and can help ease any heartache.
This is a feeling I can definitely understand; Grief is a difficult emotion to process, especially at a young age, like the child in the clip Amazon shared. But blurring the lines between life and death doesn’t seem like the healthiest way to deal with loss.
Based on Prasad’s comment, it’s clear that Amazon sees this as an evolution of remembering loved ones using old photos and videos taken before they passed away, but that’s not the same thing. A photo or video was taken with their consent and is of something the person actually did before they died; this AI-led feature isn’t playing a recording of a book reading, it’s using your voice to create a fabricated memory.
If anyone wants their voice to live on through Alexa after they die, then give them power, I’m sure they’ll love this feature. Personally, I would like to be left to rest in peace.
More than just digital undead
Outside of the nightmare-inducing uses above, this copy feature can also pave the way for new hits. While many of us know how to ignore and turn off the robotic voices on the other end of the line, we may be more inclined to listen to our grandmother or son’s pleas for money.
Coupled with sophisticated video deepfaking tools, we could eventually find it impossible to believe any video we watch – anything could have been constructed from realistic fake audio and visuals.
Amazon won’t be the only company to develop AI voice tools like this mind. We suspect that Apple, Google and every other voice assistant makers are looking to make them more realistic and personal. But these fancy imitation features are very much a pandora’s box, something to be opened with care.
To that end, it seems that Amazon is aware of the need to be cautious. It hasn’t outlined a timeline for when it will roll out this feature to existing Alexa devices, nor has it confirmed if it will be publicly available.
But now that her audio immortality has been revealed to the world, it’s probably just a matter of time before Alexa gains the ability to speak like the dead.
