On Thursday, OpenAI released the “system card” for ChatGPT’s new GPT-4o AI model that details model limitations and safety testing procedures. Among other examples, the document reveals that in rare occurrences during testing, the model’s Advanced Voice Mode unintentionally imitated users’ voices without permission. Currently, OpenAI has safeguards in place that prevent this from happening, but the instance reflects the growing complexity of safely architecting with an AI chatbot that could potentially imitate any voice from a small clip.
Advanced Voice Mode is a feature of ChatGPT that allows users to have spoken conversations with the AI assistant.
In a section of the GPT-4o system card titled “Unauthorized voice generation,” OpenAI details an episode where a noisy input somehow prompted the model to suddenly imitate the user’s voice. “Voice generation can also occur in non-adversarial situations, such as our use of that ability to generate voices for ChatGPT’s advanced voice mode,” OpenAI writes. “During testing, we also observed rare instances where the model would unintentionally generate an output emulating the user’s voice.”
It would certainly be creepy to be talking to a machine and then have it unexpectedly begin talking to you in your own voice. Ordinarily, OpenAI has safeguards to prevent this, which is why the company says this occurrence was rare even before it developed ways to prevent it completely. But the example prompted BuzzFeed data scientist Max Woolf to tweet, “OpenAI just leaked the plot of Black Mirror’s next season.”
Currently, OpenAl has safeguards in place that prevent this from happening
Clearly not.
They put them in after it happened
There are no known safeguards against “AI” doing unexpected things. The “AI” does not understand what it is doing, and so is unable to have a concept of right or wrong, or any agenda. If it is able to do something, and has been trained to do it only in restricted contexts, it will do that thing randomly in other contexts.
“Sand Kings” by George R. R. Martin
“Sand Kings” by George R. R. Martin
Huh, I never knew that Outer Limits episode was (very loosely from the Wikipedia synopsis) based on a George R. R. Martin story.
Keep in mind, your voice sounds quite different to others than it does to you (because of conductance within your skull). So, unless you have a same-sex twin, would you even recognise the voice as your own?
In today’s technological world, who hasn’t heard a recording of themself?
Have you never heard your own voice?
Unless you’ve never heard a recording or seen a video of yourself, I’m going to go with no.
As someone who has done a lot of VO professionally, that is something you can train yourself to work around- the limitations of your own skull letting you hear yourself differently than everyone else. For a long time, I had to use headphones, but I got to the point where I understood my vocalizations enough to not need them much of the time.
