In January 2024, an employee at the global engineering firm Arup joined what appeared to be a routine video conference call with the company’s Chief Financial Officer and several colleagues. The CFO requested a series of urgent wire transfers. The employee complied. Fifteen transactions later, $25.6 million had been transferred to accounts controlled by criminals. Every person on that video call — the CFO, the colleagues, the entire meeting — was a deepfake. Real-time AI-generated video and audio convincingly replicated the appearance and voice of people the employee knew and trusted, in a live interactive format, for long enough to authorise tens of millions of dollars in fraud. Hong Kong Police confirmed the incident in February 2024. It remains the largest publicly documented single deepfake fraud loss on record.
The Arup case is not an outlier — it is a demonstration of the threat’s scale and sophistication that the statistics confirm across every dimension. More than 90 percent of online media content now contains some AI-assisted element, whether through generation, enhancement, or manipulation. Humans detect high-quality deepfakes with approximately 0.1 percent accuracy according to iProov’s detection study — functionally indistinguishable from random guessing. Deloitte estimates $40 billion in US fraud losses from generative AI by 2027. Gartner projects that by 2026, 30 percent of enterprises will consider identity verification solutions unreliable in isolation because of deepfake attacks on face biometrics. One in four job candidate profiles globally will be fake by 2028, according to Gartner’s projection — after Pindrop Security found that more than one-third of 300 analysed job applicant profiles were entirely fabricated with AI-generated resumes and deepfake video interviews.
This guide covers what deepfakes and synthetic media are and how they are made, the full range of harms they cause across fraud, politics, and personal safety, the detection arms race and why it cannot fully be won by technology alone, the regulatory responses now taking effect globally, the provenance and watermarking solutions being deployed, and what individuals and organisations can do to protect themselves in a world where seeing is no longer believing.
What Deepfakes Are — and How They Are Made
The term “deepfake” combines “deep learning” and “fake” — referring to synthetic media created using deep learning AI systems that can generate, replace, or manipulate visual and audio content with sufficient realism to be indistinguishable from authentic material to the human eye and ear. The technology has advanced from hobbyist face-swapping tools requiring hours of training data and significant computational resources to real-time systems requiring only seconds of reference material and operating on consumer hardware.
The primary technical approaches to deepfake creation are: face swapping (replacing one person’s face with another’s in video, using generative adversarial networks or diffusion models); face re-enactment (animating a target person’s face to match the facial movements of a different person in a reference video, enabling the target to appear to say things they never said); voice cloning (generating synthetic audio in a target person’s voice from as little as three to ten seconds of reference audio, producing arbitrary speech in their voice); and full synthetic generation (creating entirely artificial video of people who do not exist, or of real people doing things they never did, without any starting reference footage of the target).
The accessibility transformation that makes deepfakes a 2026 mass phenomenon rather than a specialist capability is the commoditisation of the underlying tools. Sophisticated deepfake generation capabilities are now available through consumer applications, online services, and open-source models that require no technical expertise to operate. The democratisation of creation has outpaced the democratisation of detection — generating a convincing deepfake is now faster, cheaper, and requires less skill than detecting one with confidence.
The modality expansion is equally significant. Deepfakes began as a video phenomenon — face swapping in clips. In 2026, synthetic media operates across video, audio, images, text, and behavioural signals simultaneously. Multi-modal deepfakes — where video, audio, and supporting documentation are all synthetically generated in coordination — are qualitatively harder to detect than single-modality fakes because the cross-channel consistency that human and automated detection relies on is maintained across all channels simultaneously. The Arup incident worked not despite being a live video call but partly because of it — the real-time interactive format created the social expectation of authenticity that single-modality content does not.
The Harm Landscape: What Deepfakes Are Actually Used For
Understanding deepfake harms requires distinguishing between the categories of use — each with different mechanisms, different affected populations, and different appropriate responses.
Financial fraud is the most immediately costly category, and the one where deepfakes have achieved the most sophisticated operational deployment. CEO fraud and Business Email Compromise — already the highest-cost cybercrime category before deepfakes — has been significantly amplified by AI voice and video cloning that makes impersonation of senior executives convincing enough to bypass the human scepticism that unusual financial requests would otherwise trigger. The Arup case is the most extreme example, but Europol reports that AI-enabled fraud and impersonation attacks targeting banks, executives, and public officials increased sharply between 2024 and 2026. Ferrari and WPP both disclosed incidents in which executives were targeted by deepfake impersonation attempts — Ferrari’s was foiled only because an executive asked an unscripted question the impersonator could not answer; WPP’s was detected because a team member was already suspicious before the call began. Process, not technology, did the defensive work in both cases.
Non-consensual intimate imagery is the category causing the most widespread personal harm in terms of affected individual volume. AI-powered tools capable of generating realistic non-consensual intimate images from innocuous photographs of real people have proliferated rapidly, enabling targeted abuse at a scale that manual creation could never have achieved. The victims are disproportionately women and girls. The psychological harm — the experience of having synthetic intimate images of yourself created and distributed without consent — is severe, documented, and often compounded by the technical difficulty of removing content that propagates faster than any takedown mechanism can respond. This harm category was significant enough to prompt the US Congress to pass the TAKE IT DOWN Act in May 2025, which criminalises publishing non-consensual intimate deepfakes with penalties of up to two years imprisonment (three years for minors) and requires covered platforms to remove such content within 48 hours of valid takedown notices.
Political misinformation and election manipulation represent the harm category with the most profound potential consequences for democratic systems. The World Economic Forum’s March 2026 analysis notes that with 2026 densely packed with elections across continents, the speed and scale of synthetic media is “a compounding risk” for democratic processes. Deepfake videos of political candidates saying things they never said, audio clips of officials making statements they never made, and synthetic documentary “evidence” of events that never occurred have all appeared in political contexts across multiple countries in the 2024-2026 period. Even when debunked, deepfake political content can achieve viral spread before the correction reaches the same audience, and the mere existence of deepfakes creates what researchers call the “liar’s dividend” — the ability for bad actors to claim authentic video evidence of their conduct is a deepfake fabrication, undermining accountability even for content they actually produced.
Identity fraud and synthetic identity creation extend deepfakes beyond impersonation of specific individuals into the creation of entirely fictional identities. Gartner’s projection that one in four job candidate profiles will be fake by 2028 reflects the threat identified by Pindrop Security: synthetic candidates with AI-generated resumes, fabricated work histories, and deepfake video interview appearances passing initial screening and even in-person interview processes designed to verify identity. The KYC (Know Your Customer) processes that financial institutions use to verify new account holders are facing similar challenges — iProov’s tracking shows a 2,665 percent increase in attacks using “native virtual cameras” (software that intercepts video streams and injects synthetic content) against liveness detection systems designed to verify that a live person is present during verification.
The Detection Arms Race: Why Technology Alone Cannot Win
Deepfake detection has attracted significant research and commercial investment, with detection systems deployed at scale by major platforms, financial institutions, and identity verification providers. The fundamental challenge — the arms race dynamic that makes any specific detection technique vulnerable — is structural rather than merely a current state of insufficient capability. Generative AI and detection AI are trained on similar architectures using similar data, and advances in detection are rapidly incorporated into improved generation to evade those detection signals. The loop is self-reinforcing: better detection drives better generation drives better detection.
Current detection systems use multiple technical approaches. Biological signal analysis detects inconsistencies in physiological signals that are difficult for current generation systems to replicate: micro-expressions, subtle skin colour variations from blood flow, spontaneous eye movement patterns, and breathing synchronisation that humans exhibit naturally and AI generation does not yet fully replicate in all combinations simultaneously. Active liveness detection challenges the subject with randomised prompts during verification — a blink request, a head turn, a phrase to repeat — that real-time synthetic video struggles to execute without latency or artefacts that detection systems identify. Forensic digital analysis examines metadata, compression artefacts, lighting inconsistencies, and spatial coherence at the pixel level for evidence of synthetic generation or manipulation.
The practical limitations of detection are significant. iProov’s finding that human accuracy is 0.1 percent establishes that unaided human detection is not viable at any meaningful scale. Automated detection systems, while dramatically more accurate than humans, achieve their best performance rates in controlled conditions that do not reflect the diversity of real-world deployment — different devices, different compression codecs, different lighting conditions, and different generation tools all affect detection accuracy in ways that laboratory benchmarks do not capture. UC Berkeley experts note that the real-world effectiveness of detection systems claimed to have over 90 percent accuracy “is still unproven at scale.” And counter-adaptive generation — explicitly training generation models to evade specific detection signatures — means that any detection technique that becomes widely deployed becomes a known target for generation improvement.
The defender stack that is emerging in response to these limitations is layered and multi-signal. Active liveness detection with challenge-response protocols is now considered table stakes for identity verification — passive liveness alone is insufficient after the 2,665 percent increase in virtual camera attacks. Voice biometrics with real-time synthetic voice detection (Pindrop’s Pulse and similar systems) provide a second layer in audio-involved interactions. C2PA content credentials (discussed in the next section) provide provenance verification for content that has been authentically tagged at source. And process controls — verification protocols requiring out-of-band confirmation for unusual financial requests, regardless of how convincing the in-band communication appears — provide the most reliable current defence against the highest-value deepfake fraud scenarios.
C2PA and Content Credentials: Building Provenance into Media
The Coalition for Content Provenance and Authenticity (C2PA) is an open technical standard developed by Adobe, Microsoft, Intel, Arm, BBC, and others that allows media content to carry cryptographically verifiable provenance information — a digital chain of custody that records when and where content was captured, what software processed it, and whether it has been modified since capture. Content credentials attach to images, video, and audio in a machine-readable format that platforms and applications can verify, enabling users to know whether a piece of content was captured authentically and whether it has been modified.
The EU AI Act’s Article 50, effective August 2, 2026, requires that AI-generated content be “marked in a machine-readable format and detectable as artificially generated” — a requirement that C2PA’s technical standard is positioned to satisfy. Adobe’s implementation of content credentials in Photoshop, Premiere, and Firefly tags AI-generated and AI-modified content at the point of creation. Camera manufacturers including Sony, Leica, and Nikon are integrating C2PA signing into camera hardware, enabling authentic photographs to carry tamper-evident provenance from the point of capture. Social media platforms including LinkedIn have begun displaying content credential indicators on verified content — a “Cr” icon that users can inspect to see the content’s origin and modification history.
The limitation of content credentials is adoption dependency: they only work when content is created with tools that implement them and viewed on platforms that display them. The open-source tools and platforms that do not implement content credentials — which are numerous — create gaps in the provenance chain that bad actors exploit. A deepfake created with a non-credentialed tool and distributed through a non-credentialed platform appears indistinguishable from authentic content even to users who understand what credentials are and how to look for them. Universal adoption is necessary for content credentials to function as a reliable signal, and universal adoption is not achievable through technical standards alone — it requires either regulatory mandates (as the EU AI Act is beginning to provide) or platform policy requirements that create adoption pressure throughout the content ecosystem.
The Regulatory Response: Laws Taking Effect in 2026
The regulatory response to synthetic media has accelerated significantly in 2025 and 2026, driven by the compounding evidence of harm across fraud, non-consensual imagery, and political manipulation. Multiple distinct legal frameworks are now in force or entering enforcement.
The EU AI Act’s Article 50, which took effect August 2, 2026, establishes the most comprehensive transparency requirement for synthetic media globally. Providers of AI systems that generate or manipulate content must ensure that content is marked in machine-readable format as artificially generated or manipulated. Deployers must disclose to users when they are interacting with synthetic content at the first interaction, with narrow exceptions for authorised law enforcement, national security, and artistic contexts. Violations are subject to penalties of up to €15 million or 3 percent of global annual turnover. The EU’s regulation also explicitly prohibits emotion recognition in public spaces and AI-generated content designed to manipulate electoral processes — providing specific legal grounding for deepfake prosecution in political contexts.
The TAKE IT DOWN Act (Public Law 119-12), signed into law by President Trump on May 19, 2025, specifically addresses non-consensual intimate deepfakes. It criminalises the publication of non-consensual intimate imagery including AI-generated synthetic intimate images, with penalties up to two years imprisonment for adult victims and three years for minors. Covered platforms — social media, adult content sites, and communication services — must implement procedures for receiving and processing takedown notices and remove qualifying content within 48 hours of a valid notice. The law creates a federal private right of action for victims, enabling direct civil suits against both the distributors of content and, in some circumstances, the platforms that host it after failing to remove it within the required window.
State-level regulation in the US has been prolific and fragmented: 169 state deepfake laws enacted since 2022, with 146 bills introduced in 2025 alone. The majority address specific harm categories — electoral deepfakes (prohibiting AI-generated political content without disclosure in specified windows before elections), non-consensual intimate imagery, and identity fraud. The fragmentation creates compliance complexity for national platforms and individuals operating across state lines — a challenge that federal legislation has not yet resolved with a comprehensive statutory framework equivalent to the EU AI Act.
China’s synthetic media regulations, which require labelling of AI-generated content and disclosure of deepfake usage on platforms operating in China, have been in force since 2022 and have been extended and strengthened through 2025 and 2026. China’s approach is particularly significant because it requires platforms to register their synthetic media generation capabilities with regulators before deployment — a proactive licensing model rather than the reactive penalty model that characterises most Western regulation.
The Misinformation Effect: The Liar’s Dividend and Epistemic Damage
Beyond the direct harms of specific deepfake attacks, synthetic media creates a broader epistemic damage that may prove more consequential in the long run than any individual incident. The “liar’s dividend” — the term coined by law professors Robert Chesney and Danielle Citron for the benefit that deepfakes provide to bad actors who deny authentic evidence of their conduct by claiming it is fabricated — describes a mechanism by which the existence of deepfake technology undermines trust in authentic documentation even when no deepfake is present.
When any public figure can plausibly claim that a video of them saying something was AI-generated, the evidentiary value of video documentation of public conduct collapses. When any audio recording can be dismissed as a voice clone, journalism based on recorded sources becomes easier to challenge. When financial regulators cannot be certain that video-verified customer identities are authentic, the entire KYC infrastructure built around visual verification becomes unreliable. The problem is not just the fake content that deepfakes create — it is the doubt that deepfakes cast on authentic content by making forgery indistinguishable from documentation.
The World Economic Forum’s framing — “just knowing deepfakes exist can make us doubt things we read and see — even the truth” — captures the mechanism precisely. A population that has learned to distrust visual media is one that is more vulnerable to manipulators who benefit from that distrust, and less protected by the documentary accountability that authentic media enables. The solution requires not just better detection technology but what the WEF calls “resilience” — the institutional, educational, and technical capabilities that allow people to navigate information environments where synthetic media is pervasive without either credulously accepting everything or sceptically rejecting all documentation.
Protecting Yourself and Your Organisation
The practical response to the deepfake threat environment requires both individual habit changes and organisational process changes. Technology alone provides inadequate protection; the most effective defences are process controls that create friction for high-stakes actions regardless of how convincing the communication requesting them appears.
For individuals, the most important habit change is treating any unsolicited request — by video call, phone, email, or message — for financial action, credential provision, or sensitive information as unverified until confirmed through an independently established channel. An unexpected video call from a trusted person requesting urgent action is precisely the attack pattern that the Arup incident demonstrates. Calling back on a known phone number, meeting in person, or confirming through a pre-established code phrase are all effective out-of-band verification approaches that deepfake technology cannot intercept.
For organisations, the critical process control is an explicit policy requiring out-of-band confirmation for any financial transaction above a defined threshold, regardless of how the request was delivered or how confident the requester appears. This policy should be accompanied by training that specifically addresses the deepfake threat — helping employees understand that convincing appearance and familiar voice are no longer sufficient authentication for high-value actions. Ferrari’s executive who asked an unscripted question that the impersonator could not answer demonstrated the value of natural scepticism; building that scepticism into process rather than relying on individual judgment under social pressure is what organisational process design is for.
For platforms and organisations processing identity verification, the deployment of active liveness detection with challenge-response protocols — rather than passive video verification alone — provides meaningful protection against the virtual camera attacks that drove the 2,665 percent increase iProov documented. Multi-factor verification that includes a channel the user controls independently of the verification interaction (a code sent to a phone number, a confirmation email to a verified address) adds a layer that real-time deepfake generation cannot intercept without compromising a second channel simultaneously.
The broader societal response to synthetic media is the most difficult and most important challenge. Detection technology is advancing but will not provide complete protection. Regulation is taking effect but cannot prevent all harm before it occurs. Content credentials provide provenance verification but only where universally adopted. Education about how to identify and respond to synthetic media provides some protection but faces the fundamental problem that high-quality deepfakes are genuinely indistinguishable from authentic content by human perception alone. The answer to the deepfake challenge is not any single solution but a layered combination of all of them — technology, regulation, provenance standards, and human literacy — applied with the understanding that the underlying generation capability will continue improving and that the goal is not to make deepfakes impossible but to make their consequences manageable. In 2026, “seeing is no longer believing” is not hyperbole. It is a description of the information environment in which every person with a smartphone now lives.
0 Comments
No comments yet. Be the first to share your thoughts!