Deepfake Detection Techniques

Modern deepfake detection uses multiple complementary approaches, from CNN-based binary classifiers to biological signal analysis and frequency domain forensics. Each technique targets different artifacts left by generation processes.

1. CNN-Based Binary Classification

The most common approach: train a CNN to classify images/frames as real or fake. Popular architectures include EfficientNet, XceptionNet, and ResNet variants:

import torch
import torchvision.models as models
import torch.nn as nn

class DeepfakeDetector(nn.Module):
    def __init__(self):
        super().__init__()
        # Use EfficientNet-B4 as backbone
        self.backbone = models.efficientnet_b4(pretrained=True)
        # Replace classifier for binary detection
        num_features = self.backbone.classifier[1].in_features
        self.backbone.classifier = nn.Sequential(
            nn.Dropout(p=0.4),
            nn.Linear(num_features, 1)  # Binary: real vs fake
        )

    def forward(self, x):
        return torch.sigmoid(self.backbone(x))

# Training pipeline
detector = DeepfakeDetector()
optimizer = torch.optim.Adam(detector.parameters(), lr=1e-4)
criterion = nn.BCELoss()

# Label: 0 = real, 1 = fake
for images, labels in dataloader:
    preds = detector(images).squeeze()
    loss = criterion(preds, labels.float())
    loss.backward()
    optimizer.step()

2. Biological Signal Analysis

Real humans exhibit physiological signals that deepfakes often fail to reproduce:

• Eye blinking: Early deepfakes rarely blinked. While improved, blinking patterns can still be inconsistent.
• Remote photoplethysmography (rPPG): Real faces show subtle color changes from blood flow. Deepfakes lack this biological pulse signal.
• Micro-expressions: Involuntary facial movements lasting 40-200ms are difficult for generators to reproduce naturally.
• Head pose estimation: 3D head position consistency between frames - deepfakes often show unnatural stabilization.
• Gaze direction: Inconsistent or impossible eye movement patterns can indicate manipulation.

3. Frequency Domain Analysis

GAN-generated images leave distinctive artifacts in the frequency domain that are invisible to the human eye:

import numpy as np
import cv2

def compute_frequency_spectrum(image):
    """Compute 2D FFT spectrum for deepfake analysis."""
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    # 2D Discrete Fourier Transform
    f_transform = np.fft.fft2(gray)
    f_shift = np.fft.fftshift(f_transform)
    magnitude_spectrum = np.log(np.abs(f_shift) + 1)
    return magnitude_spectrum

def azimuthal_average(spectrum):
    """Compute 1D azimuthal average of 2D spectrum.
    GAN-generated images show characteristic peaks."""
    center = np.array(spectrum.shape) // 2
    y, x = np.indices(spectrum.shape)
    r = np.sqrt((x - center[1])**2 + (y - center[0])**2).astype(int)
    radial_mean = np.bincount(r.ravel(), spectrum.ravel()) / np.bincount(r.ravel())
    return radial_mean

4. Temporal Consistency Analysis

For video deepfakes, analyzing consistency across frames reveals manipulation:

• Face landmark tracking: Track 68+ facial landmarks across frames. Deepfakes show jitter and drift in landmark positions.
• Identity consistency: Extract face embeddings per frame. In real video, embeddings cluster tightly; in deepfakes, they show higher variance.
• Optical flow analysis: Motion patterns between frames can reveal unnatural warping from face replacement.
• Background consistency: The boundary between the manipulated face and the original background can show temporal artifacts.

5. Attention-Based Detection

Attention mechanisms help detectors focus on the most discriminative regions:

• Multi-Attention: Use multiple attention maps to focus on different facial regions (eyes, nose, mouth, edges)
• Self-Attention: Vision Transformers (ViTs) capture global consistency patterns across the entire face
• Cross-Attention: Compare face region features with surrounding context for inconsistency detection

Detection Method Comparison