Multimodal Deepfake Generation and Detection:
Challenges, Methods, and Future Directions

Summary

The rapid advancement of generative AI has revolutionized the creation and manipulation of synthetic media across different use cases. However, this progress has also fuelled the rise of deepfakes; highly realistic audio, image, and video content generated without the consent of the individuals which has potential negative societal impact. These manipulated media artifacts can misrepresent people by making them appear to say or do things they never did, posing significant threats to privacy, trust, and public discourse.

Deepfakes have become tools for disinformation, misinformation, online harassment, and even fraud, undermining systems such as facial recognition and video-based authentication (e.g., KYC). As these manipulations grow more convincing, it becomes critical for platforms and systems to detect and mitigate them effectively.

This tutorial will provide a comprehensive introduction to the generation, detection, and societal impact of deepfakes from a multimodal perspective. It will equip participants with foundational knowledge of generative techniques, explore cutting-edge deepfake detection methods, and offer hands-on experience with open-source tools. Attendees will also engage in discussion around ethical considerations and real-world deployment challenges, fostering a robust understanding of how to combat synthetic media threats in interactive systems.

Details & Schedule

Key Topics

• Overview of generative models (GANs, diffusion models) for deepfake creation
• Detection techniques: CNNs, transformers, and multimodal fusion
• Dataset curation and annotation challenges for in-the-wild deepfakes
• Evaluation protocols and robustness to adversarial attacks
• Ethical and societal implications of deepfakes
• Interactive demonstration of deepfake generation and detection tools
• Explanation deepfakes detection and Data/Model watermarking

Learning Objectives

• Describe fundamental architectures behind visual and audio deepfake generation
• Compare detection methodologies across modalities
• Implement or apply at least one open-source detection pipeline
• Evaluate model robustness in real-world, multimodal deepfake scenarios
• Analyze ethical implications of deepfake technology in interaction systems

Target Audience

• Level: Interested undergraduate and masters’ students, PhD students, industry people and researchers
• Prerequisites: Basic knowledge of machine learning and signal processing
• Fields: Affective computing, multimodal interaction, computer vision, audio processing, HCI, cybersecurity

Resources

• GitHub repository with slide deck and reading list (TBA)

For details, please contact Shreya Ghosh.
Last updated: 01 Oct 2025