Detect and Locate : Exposing Face Manipulation by Semantic- and Noise-level Telltales

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

40 Scopus Citations
View graph of relations

Detail(s)

Original languageEnglish
Pages (from-to)1741-1756
Journal / PublicationIEEE Transactions on Information Forensics and Security
Volume17
Online published28 Apr 2022
Publication statusPublished - 2022

Abstract

The technological advancements of deep learning have enabled sophisticated face manipulation schemes, raising severe trust issues and security concerns in modern society. Generally speaking, detecting manipulated faces and locating the potentially altered regions are challenging tasks. Herein, we propose a conceptually simple but effective method to efficiently detect forged faces in an image while simultaneously locating the manipulated regions. The proposed scheme relies on a segmentation map that delivers meaningful high-level semantic information clues about the image. Furthermore, a noise map is estimated, playing a complementary role in capturing low-level clues and subsequently empowering decision-making. Finally, the features from these two modules are combined to distinguish fake faces. Extensive experiments show that the proposed model achieves state-of-the-art detection accuracy and remarkable localization performance.

Research Area(s)

  • Data mining, Face forensics, face forgery detection, face manipulation localization, Faces, Feature extraction, Forgery, Location awareness, Semantics, Training