Papers
arxiv:2611.21114

VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection

Published on Nov 16
· Submitted by
Ke Sun
on Nov 19
Authors:
,
,

Abstract

Visible artifacts are an unreliable cue for detecting generated videos, because newer generators learn to remove them. We instead detect a trace left by the generation pipeline itself. Most video generators decode latents with a 3D causal VAE that groups consecutive frames in time, so the fidelity of generated frames rises and falls periodically with their position in a group. We call this periodic trace VAE Rhythm. Real videos are captured frame by frame and do not have it. To reveal it without training, we propose a phase scan. The scan reconstructs a clip with an off-the-shelf VAE from successive start offsets and measures the periodicity of the reconstruction error across these offsets. We find VAE Rhythm in most of the 39 variants of 31 generators we test, including the closed-source Sora, Kling 3.0 and Seedance 2.0. We use the rhythm as a zero-shot score, as a plug-in for existing detectors and as a data engine that pairs real videos with their VAE reconstructions to train our detector CaDet. On DF26, the training-free rhythm score outperforms all 15 retrained baselines, and as a plug-in it improves all 19 tested detectors by 6.7% to 23.7% AUROC. CaDet achieves the best AUROC on AIGVDBench, DF26 and three 2026 closed-source models, and its variant trained without any generated video still exceeds 90% on AIGVDBench.

Community

Paper author Paper submitter

Hi everyone! We are excited to share our work VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection from Show Lab, NUS.

As newer video generators learn to suppress visible artifacts, appearance-based cues become unreliable. Instead of chasing generator-specific visual flaws, we target a structural trace left by the generation pipeline itself: most video generators decode latents with a 3D causal VAE that groups consecutive frames in time (typically 4 frames, or 8 in models like LTX and Kling 3.0). As a result, frame fidelity rises and falls periodically with a frame's position inside its decoding group—a trace we call VAE Rhythm, which frame-by-frame real camera videos do not have.

How to read it without training

Our phase scan reconstructs a clip with an off-the-shelf 4× video VAE across successive frame start offsets and measures the periodicity (not the magnitude) of the reconstruction error. Whenever the scan's frame groups align with the clip's decoding groups, reconstruction residual drops—producing a clear periodic oscillation on generated clips while remaining nearly flat on real ones.

Key Highlights

  • Widespread across generators: Observed across most of the 39 variants of 31 generators we tested (2023–2026), including closed-source Sora, Kling 3.0, and Seedance 2.0.
  • Training-free zero-shot score: Without any training, the rhythm score reaches 76.2% AUROC on DF26, outperforming all 15 retrained baselines.
  • Universal training-free plug-in: Adding positive rhythm evidence boosts all 19 tested detectors on DF26 by +6.7% to +23.7% AUROC (raising our detector to 85.2% AUROC).
  • VAE-pair engine & CaDet: Using VAE round-trips on real clips under shared post-processing, we synthesize matched pairs to train CaDet (two lightweight paths on one frozen DINOv3 ViT-L/16 backbone). CaDet achieves state-of-the-art AUROC on AIGVDBench (93.9% open / 95.8% closed), DF26 (78.5% all / 79.0% closed / 87.8% paired 2AFC), and three 2026 closed-source models (88.3%). Even without any generated training clips (CaDet-GF), it reaches 90.9% / 91.9% AUROC on AIGVDBench.

Links & Resources

Feedback and questions are very welcome!

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.21114
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.21114 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.21114 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.