Video Inpainting Models in 2026: From Optical Flow to One-Step Diffusion, and What Each Generation Means for Your Watermarks
Last updated: September 2026
Every week our processing queue receives files that were cleaned by other tools first — footage where a logo was smeared into a soft blur, or where the patched region shimmers the moment anything moves behind it. The tool usually isn't broken. The model behind it belongs to an older generation of video inpainting, and each generation fails in a characteristic way. This video inpainting models guide walks the whole 2026 landscape — optical flow, diffusion, diffusion transformers, one-step distillation — so you can tell from a single output clip which generation produced it, and what to ask any vendor before you upload. The same generational split shows up in the platform stamps we handle: a Veo corner badge, a Hailuo logo, a Sora mark that drifts.
What Is Video Inpainting Actually Solving?
Video inpainting reconstructs the region behind a mask — a watermark, a logo, burned-in text — so the fill looks like it was never touched. The DiffuEraser paper, from Alibaba's Tongyi Lab, splits the problem into two pixel classes that determine everything: pixels that are visible somewhere else in the clip, and pixels that never appear at all.
That split is the whole story. Known pixels — a brick that was exposed in frame 12 before the logo slid over it — can be copied or propagated from where they were visible. Unknown pixels — the part of the brick wall that sat behind an opaque logo for the entire clip — must be invented. Propagation is accurate but blurs when over-stretched; generation is sharp but can hallucinate. Every model family below is a different bet on where the failure hurts less. For watermark removal specifically, the mask is usually small, which is why the choice of generation shows up as texture quality and flicker rather than as obviously wrong content.
Why Did Optical Flow Shape the First Generation?
The first serious video removal tools treated each frame independently, then got smarter about time. Image inpainting models like LaMa ran per frame; the patched regions looked fine as stills but flickered in playback, because each frame reinvented the fill independently. Flow-guided models — STTN, E2FGVI, and eventually ProPainter — aligned pixels across frames before filling, which is where the field stabilized.
ProPainter is still the open-source default in 2026, and it is worth understanding precisely. The DiffuEraser authors describe it this way: "The state-of-the-art algorithm, Propainter, exemplifies this approach and comprises three key modules: recurrent flow completion, dual-domain propagation, and mask-guided sparse Transformer" — Xiaowen Li, DiffuEraser Technical Report. Released at ICCV 2023 by S-Lab at Nanyang Technological University, it estimates motion between frames, propagates known pixels along that motion, and uses a sparse transformer for what propagation cannot cover.
The weakness is the flow itself. "Although flow-based techniques improve temporal coherence by propagating valid pixels across frames, they still struggle to handle large occlusions, leading to noticeable artifacts" — Dingming Liu and the UnderEraser team (Peking University and Tencent's WeChat Vision). Flow estimation fails on occlusions, motion blur, and fast camera moves — exactly the shots where watermarks sit over busy backgrounds. And when the mask is large enough that many pixels are never visible anywhere, propagation has nothing to copy. Our own detection-to-reconstruction walkthrough relies on propagation for the easy majority of pixels, but the residual unknown pixels are where quality is won or lost.
One licensing detail the tool listicles skip: ProPainter ships under the S-Lab License, which is written for non-commercial research — commercial use requires written permission from the authors. Free to download is not free to build a business on.
What Changed When Diffusion Models Entered?
Diffusion models flipped the bet. Instead of copying pixels along estimated motion, they generate the masked region from a learned prior over what video looks like. FloED (HKUST and Alibaba's DAMO Academy, late 2024) added flow guidance to a diffusion backbone; DiffuEraser (January 2025, Tongyi Lab) built directly on Stable Diffusion, using a ProPainter-fed prior as initialization.
The generative leap is real, and so is the motivation. As the DiffuEraser authors state: "these approaches often encounter blurring and temporal inconsistencies when dealing with large masks, highlighting the need for models with enhanced generative capabilities" — Xiaowen Li, DiffuEraser Technical Report. Where propagation smooths, diffusion synthesizes texture that holds up at stroke level — fabric weave, brick grout, water ripple.
The cost was compute. Multi-step denoising is slow, and early diffusion inpainting needed the very optical-flow priors it was meant to escape, which capped its upside: if the flow map feeding the model is wrong, the generation inherits the error. DiffuEraser mitigated this with weak conditioning and expanded temporal receptive fields, but the pipeline still carried two models and many denoising steps. For creators running open-source stacks, that meant GPU patience; for services, it meant cost per minute of video. In our analysis of the flow-guided submissions that still reach our queue, the two-model pipeline shows up the same way — as the slowest jobs, not the worst-looking ones.
Why Is the Field Converging on Diffusion Transformers?
The 2026 generation — Diffusion Transformers, or DiT — replaces both compromises at once. DiT-based video models use 3D full attention across space and time, learning temporal coherence directly instead of borrowing it from an optical-flow estimate — and shipping hardware caught up with the compute.
"Traditional methods predominantly rely on flow-based propagation and spatio-temporal Transformers, but these approaches face limitations in effectively leveraging long-term temporal features and ensuring temporal consistency in the completion results, particularly when dealing with large masks" — Jie Liu and Zheng Hui of Mango TV, in the EraserDiT paper, which applies a video DiT to object removal.
Speed stopped being the deal-breaker, too: "it takes only 65 seconds (testing on one NVIDIA H800 GPU) to complete a video with a resolution of 2160 × 2100 with 97 frames without any acceleration method" — Jie Liu, EraserDiT. That is production-adjacent territory, not lab-adjacent.
Productized systems moved the same direction. Volcano Engine (ByteDance's cloud arm) describes a subtitle-erasure product built on a DiT inpainting model that, per its engineering write-up, removed the text-prompt and cross-attention modules entirely — learning frame-to-frame dependency from raw video sequences — and reports an overall PSNR above 38 on its benchmark set (vendor-reported, not independently verified). Two details in that write-up matter beyond the marketing: the two-stage training recipe (broad pretraining on random masks, then domain-specific fine-tuning on font-level masks at higher resolution), and the claim that pixel-precise character masks beat OCR bounding-box masks because smaller repair regions preserve the surrounding guiding pixels. Tencent's WeChat Vision team published in the same direction this summer, targeting "object-induced side effects (e.g., shadows, reflections, and illumination changes)" — Dingming Liu, UnderEraser — which is the failure mode you see when a removed logo takes its reflection with it.
Can One-Step Models Make Removal Real-Time?
The newest line attacks the latency that diffusion left behind. Distillation compresses a multi-step diffusion teacher into a single-step student, keeping most of the generative quality while cutting inference from minutes to seconds. For anyone shipping removal as a service, that conversion is what turns a lab result into a product feature. When we evaluated distillation for our own backend, it was the difference between batch-only and interactive — once a re-run costs a second instead of a minute, you compare attempts instead of committing to the first one.
"Traditional approaches based on optical flow or attention mechanisms often introduce noticeable artifacts and yield unnatural results. In contrast, diffusion-based methods improve visual realism but demand multiple denoising steps, limiting their practicality" — Zizhao Chen and colleagues at Xi'an Jiaotong University, working with Baidu and SGIT AI Lab on the D2DF framework. Their result: "The denoising process for a single video takes only about 1 second" — Zizhao Chen, D2DF.
For a batch pipeline, the difference between sixty-five seconds and one second per clip is the difference between a GPU farm and a laptop. It also changes what interactive tools can offer — re-run and compare becomes viable when each attempt costs a second rather than a minute. The commercial removal tools shipping one-step backends in 2026 are, mostly, this idea wearing a product interface. We run the same class of thinking in reverse for batch workloads: throughput is a feature, not an infrastructure detail.
What Does This Mean for Watermark Removal Specifically?
Watermark removal is a friendlier case than the object-removal benchmarks most of these papers target. A corner logo covers a small, mostly static region — mostly known pixels, easily propagated, with only the never-visible core left to generate. That is why the method comparison we keep for creators holds across generations: masking-plus-reconstruction wins, and newer generations raise its ceiling.
Three specifics still matter:
- Moving marks stress the temporal layer. Kling's stamp and TikTok's bouncing logo change position mid-clip, so per-frame masks are wrong by design; the model must track across frames. This is where flow-free DiT attention genuinely outperforms the older stack — no flow error to compound.
- Mask size is the quality lever. A bounding box around a subtitle block or a wide watermark discards the guiding pixels between characters and inflates the repair region — the exact failure Volcano's font-level masking targets, and the same economics we apply to segmentation after detection. Tighter mask, fewer unknown pixels, less hallucination.
- Semi-transparent marks change the detection problem before inpainting ever runs. A 40%-opacity overlay reads differently over every background, which is why detection and segmentation deserve their own models — and why the generation of the inpainting model is only half of the quality story.
We compared outputs from every generation in our own queue, and the pattern holds across all of it: propagation handles what the camera already showed you, generation handles what it hid, and the 2026 winners spend their compute only on the second part.
Which Generation Should You Actually Use?
Match the stack to the job. Open-source on your own GPU: ProPainter remains the reference implementation — free to run, mature, well-documented — with the S-Lab non-commercial caveat if money is involved. DiffuEraser's code is public and worth trying on large-mask cases, at diffusion speeds. One-step and DiT-era models are appearing in open repositories, but weights and licenses vary; check both.
Commercial services: ask which generation their backend runs — anything flow-only is 2023-era quality at 2026 prices — and ask about masks, because a service that boxes whole regions is discarding quality before the model starts. Our own pipeline is deliberately DiT-generation on the reconstruction side with pixel-precise masks upstream, and we would not ship the reverse. If you own the footage or have permission — the legal baseline we keep repeating, with the DMCA §1202 metadata traps as a separate layer — upload the file, compare before and after at full resolution, and judge the patched region where it counts: texture continuity in a still, then flicker in motion.
FAQ
Is ProPainter still good in 2026?
For small static masks, yes — propagation fills most pixels from neighboring frames and results hold up well. It struggles on large masks, fast motion, and occlusion-heavy scenes, and its S-Lab license restricts commercial use. The DiT-era papers benchmark past it on exactly those hard cases.
What is the difference between DiffuEraser and ProPainter?
ProPainter propagates known pixels along estimated optical flow and uses a sparse transformer for the rest; DiffuEraser generates the masked region with a Stable Diffusion backbone, using a ProPainter-derived prior only as initialization. DiffuEraser wins texture and temporal stability on large masks; ProPainter wins on speed for small, static regions.
What is a diffusion transformer (DiT) doing differently in video inpainting?
It uses 3D full attention across width, height, and time, so temporal coherence is learned end-to-end rather than borrowed from an optical-flow estimate. That removes the flow-error ceiling and handles moving marks without a separate tracking stage.
Do faster one-step models sacrifice quality?
The D2DF authors report one-step results surpassing multi-step baselines on their benchmarks, and distillation has generally closed the gap for constrained tasks like object removal. Extreme cases — huge masks, minute-long shots — are where multi-step teachers can still lead.
Conclusion
The 2026 video inpainting landscape reads as five answers to one question: what do you do with pixels the camera never showed you? Optical flow copied neighbors and blurred; per-frame diffusion flickered; flow-guided diffusion carried its own priors; diffusion transformers learned time directly; one-step distillation made it all fast enough to ship. For watermark removal, the practical takeaway is smaller than the theory — masks are small, marks are mostly static, and every generation since ProPainter has been fighting over the same shrinking residue of unknown pixels. When a result looks smeared or shimmery, you are looking at that residue losing, and the fix is a newer generation plus a tighter mask, not a harder push on the same old stack.
References
- DiffuEraser: A Diffusion Model for Video Inpainting — Xiaowen Li, Haolan Xue, Peiran Ren, Liefeng Bo (Tongyi Lab, Alibaba Group), arXiv 2501.10018
- EraserDiT: Fast Video Inpainting with Diffusion Transformer Model — Jie Liu, Zheng Hui (Mango TV), AAAI 2026, arXiv 2506.12853
- From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Planting — Zizhao Chen et al. (Xi'an Jiaotong University, Baidu, SGIT AI Lab), ECCV 2026, arXiv 2607.14976
- From Understanding to Erasing: Towards Complete and Stable Video Object Removal — Dingming Liu et al. (Peking University, WeChat Vision Tencent), arXiv 2604.01693
- ProPainter: Improving Propagation and Transformer for Video Inpainting — Zhou, Li, Chan, Loy (S-Lab, Nanyang Technological University), ICCV 2023 (GitHub, S-Lab License)
- 火山引擎视频点播：基于 DiT 大模型与字体级分割的无痕字幕擦除（厂商技术说明）
Last updated: September 2026
