AI
Voice Clones Fool Listeners as Well as Real Voices
Voice cloning tools like ElevenLabs and B Cloner produce audio that sounds identical to the original speaker. Katherine Rackliffe demonstrated that listeners cannot distinguish a clone from a real recording, especially over a phone call. In her study, participants who heard both versions guessed wrong every time. Some thought the real voice was a clone because it sounded scripted. The clone sometimes sounded more natural than the professor reading from a script. This parity makes voice cloning a direct threat to voice-based authentication systems used by banks like Schwab and Chase.
Study Design: Cloned Professors and Four Message Types
Rackliffe obtained one minute of audio from five professors across accounting, biology, and other non-tech fields. She used B Cloner to generate a voice clone. The phishing message asked for a student ID number — information professors already have. She sent the message in four formats: plain text, robotic text-to-speech, AI cloned voice, and a real voice recording. Students received the message as an MP3 voice note from an unknown number. The study received IRB approval after months of ethics paperwork. Students gave implied consent via a vague survey taken two weeks before the attack.
Results: Text Messages Drew Most Responses, Voice Clone Matched Real Voice
Text messages had the highest response rate — students found it easy to reply. The robotic text-to-speech still tricked some students, who assumed the professor was too busy to type. The AI cloned voice and the real voice produced identical response rates. In follow-up interviews, no participant could identify which recording was a clone. The biggest red flag was the unknown phone number, not the voice quality. Several students asked the professor to verify by describing the previous lecture. A few reported the message as phishing, but most either ignored it or sent their ID.
Defenses: Unknown Numbers Are the Biggest Red Flag
Rackliffe emphasized that awareness is the primary defense. Most people do not know voice cloning exists or that a phone call can be faked. The study showed that the unknown caller ID was the strongest cue — no one cited the voice quality as suspicious. She recommended that organizations train users to verify unexpected calls through a separate channel. Future research should target senior citizens, who are more frequently victimized, and test live phone calls with AI agents. Spoofing the professor’s number would increase realism but raise ethical and legal hurdles.
Q&A
How did you obtain the professors’ voice samples? Professors spoke for about one minute on a generic topic, knowing the audio would be used to train a voice clone. ▶ 10:30
Did you use professors from different subjects? Yes, five professors from accounting, biology, and other fields; we avoided tech and computer science. ▶ 10:59
What would you change in a future study? Speed up ethics paperwork, target senior citizens instead of college students, and test voicemail attacks. ▶ 11:41
Did participants ever cite the voice quality as suspicious? No. The unknown number was the only reason they gave for not responding. ▶ 12:35
How did you handle student consent? Students took a vague survey two weeks before the attack, then received a debrief message 48 hours after the phishing message. ▶ 15:24
Notable Quotes
it only takes about like 5 minutes of work and it’s really easy to replicate Katherine Rackliffe · ▶ 2:32
people can’t tell the difference between the voice clone and the real voice and so they would just respond uh the same Katherine Rackliffe · ▶ 8:28
it came from an unknown number so I didn’t trust it Katherine Rackliffe · ▶ 12:44
Key Takeaways
- AI voice clones are indistinguishable from real voices over the phone.
- Unknown caller ID is a stronger phishing cue than voice quality.
- Free cloning tools make vishing attacks cheap and accessible to anyone.
About the Speaker(s)
Katherine Rackliffe recently graduated in the cybersecurity program at Brigham Young University, and an incoming PhD student for the University of Wisconsin-Madison.