What is a deepfake and voice cloning?
A deepfake is image, audio or video content created or manipulated with AI so that it resembles a real person or event and may be taken as genuine.
Generative AI can imitate a person's face or voice, for example from pictures and recordings already available online. Voice cloning is used in fraud by phone call and voice message, among other things, where there is no picture to examine.
Fraudsters use the technique to pose as a manager, a colleague or an authority and get someone to pay, hand over information or sign in. In May 2025 the FBI warned of AI-generated voice messages claiming to come from senior US officials.
The EU AI Act defines the concept and generally requires anyone using AI to create a deepfake to disclose that the content has been artificially generated or manipulated.
How it works
An AI model uses recordings or pictures of a person to imitate their voice, tone or face. The result can be used in a phone call, a voice message or a video meeting.
An example. A finance assistant receives a short voice message from someone who sounds like the manager, then an email with an urgent payment to a new account. The voice is cloned, and the account belongs to the fraudster.
Why it matters
The technique is used to sound like someone people trust, and the FBI notes that AI-generated content is often difficult to identify. It undermines routines that rely on recognising a voice or a face. Read more about generative AI.
What to keep in mind
Verify unusual payments and changed account details through another channel, and let two people approve under the four-eyes principle.
Agree a code word within the team. Then nobody has to judge whether the voice is genuine.
What it means for you
Build your protection on routines. Call back on a number you already have, and agree a code word for unusual requests.
Common questions
Can you hear whether a voice has been cloned?
Not reliably. A cloned voice need not be perfect to persuade during a short call. That is why verifying through another channel is safer than trying to judge the voice.
What do we do if we suspect a fraud attempt?
End the call and ring back on a number you already have. Make no payments or changes until it has been confirmed, and report the attempt to the police.
Which rules apply to deepfakes?
The EU AI Act generally requires anyone who creates a deepfake with AI to disclose that the content is artificial. Fraud is a crime whatever technology is used.
Sources: Regulation (EU) 2024/1689 on EUR-LexFortifikationsverket, Voice clone or the boss? (in Swedish)FBI on AI-generated voice messages (2025)
Related terms
Read more
All terms Updated