Out of View, Still in Motion: Dynamic Entity Re-entry in Video World Models

Project Overview

Abstract

Recent advances in video world models enable realistic and controllable simulation, yet long-horizon dynamic memory remains challenging when evolving entities leave the field of view and later return. Existing memory metrics can assign similar scores to distinct recovery failures. We introduce a phase-aware evaluation protocol that separately measures entity reappearance, identity consistency, and post-re-entry temporal stability. Using this protocol, we find that recovery failures persist even when the complete observation history remains accessible. Controlled experiments show that entity-region-weighted training strengthens identity recovery, while stochastic sampling correction improves temporal matching. Applying these interventions selectively requires entity localization during denoising, when intermediate predictions remain noisy latent representations. Obtaining regions through RGB-based segmentation would require intermediate decoding and additional segmentation calls. These findings motivate ReentryMem, a self-localized framework that identifies entity regions directly from noisy video latents through text cross-attention. The shared localization mechanism guides entity-weighted flow matching during training and localized overshooting with re-noising during inference, without additional memory modules or architectural modifications. Experiments on two video backbones demonstrate the effectiveness of this approach. On the main evaluation, ReentryMem improves subject consistency from 0.8204 to 0.8378 and temporal consistency match from 0.9154 to 0.9318 over the baseline. Under block-causal attention, temporal consistency match increases from 0.8506 to 0.9116. Qualitative results further illustrate applicability to multiple entities, scene interactions, and changing backgrounds.

Method

ReentryMem method overview, showing the self-localized entity support, entity-aware training, and region-localized overshooting at inference.
Localize the entity
Read presence and shape from selected text cross-attention heads to estimate spatial support for the subject.
Weight training supervision
ReentryMem-Train increases the flow-matching loss weight inside the predicted entity support.
Correct the sampled region
ReentryMem uses the same trained weights and applies localized overshooting and re-noising during inference.

Experiments

Comparison with Other Models

Table 1 · Quantitative comparison
Method Dynamic memory Camera Existing metrics
Rea-Score ↑SC ↑TCM ↑ Cam. Gain → 1ObjCons ↑ SubjCons ↑DSC ↑
Released world models
Matrix-Game 3.590.6.7548.9144.638.6732.8156.8459
EchoWM91.1.7495.8796.421.6551.7622.8199
SANA-WM97.8.7734.8979.345.6493.7982.8178
HY-WorldPlay 1.586.1.7780.9107.534.6122.8020.8490
LingBot-World-V296.6.8115.9232.634.6402.7761.8361
Memory-oriented method
HyDRA†78.9.7465.9071.668.6193.7669.8486
Controlled comparison
Baseline96.1.8204.91541.064.6536.7658.8720
ReentryMem-Train99.2.8380.92311.093.6520.7670.8757
ReentryMem(Ours)97.2.8378.93181.091.6513.7659.8750

Rea-Score in percent. Best and second best follow the paper. † Reproduced with official code on our training set.

Controlled Comparison

Baseline uses the standard training objective and sampler. ReentryMem-Train gives more training weight to the predicted entity region. ReentryMem(Ours) uses the ReentryMem-Train checkpoint with region-localized overshooting during sampling.

More Qualitative Results

Multiple Entities

Interactions

Substantial Background Changes

More Results