Deepfakes just got even harder to detect with a realistic heartbeat

As artificial intelligence (AI) continues to advance, deepfakes – images, videos, and audio doctored to appear like the real person is saying or doing something – are becoming increasingly realistic, most recently featuring a lifelike heartbeat that makes them even harder to detect.

Indeed, it seems that the deepfake technology has gained a significant advantage – the lack of a realistic pulse that used to be a telltale sign that something is a deepfake, according to a new scientific study published recently in Frontiers in Imaging.

As Dr. Peter Eisert, a professor at Humboldt University of Berlin and the study’s corresponding author, explained: 

“Here we show for the first time that recent high-quality deepfake videos can feature a realistic heartbeat and minute changes in the color of the face, which makes them much harder to detect.”

Specifically, the scientists first coded a state-of-the-art deepfake detector that automatically extracts and analyzes the pulse rate from videos, using novel methods to compensate for movement and remove noise, only requiring an input video of the face of a single person for just 10 seconds to work.

Additionally, they created their own dataset of driving videos used to create deepfakes. As they filmed the videos, an ECG monitored the heartbeat of the protagonists, allowing them to confirm that the measurements of the so-called remote photoplethysmography (rPPP), a webcam-based telehealthcare method for estimating vital signs, made by their new detector were very accurate.

Turning the tables with a realistic heartbeat

However, when using the same detector to analyze known deepfakes, things got a bit more complicated. Surprisingly, their detector recognized a pulse in these deepfakes as well – even though they hadn’t consciously put one in, and the nonexistent pulse typically appeared exceptionally realistic. In Eisert’s words:

“Our results show that a realistic heartbeat may be added by an attacker on purpose, but can also be ‘inherited’ inadvertently from the driving genuine video. Small variations in skin tone of the real person get transferred to the deepfake together with facial motion so that the original pulse is replicated in the fake video.”

That said, there’s still a way to tweak the deepfake detectors to catch up with the most recent advancement – using blood flow. As Eisert noted, deepfakes still can’t fake the physiologically realistic variations in blood flow across space and time within the person’s face, paving the way for better deepfake detectors.