Out of View, Still in Motion: Dynamic Entity Re-entry in Video World Models
Abstract
Recent advances in video world models enable realistic and controllable simulation, yet long-horizon dynamic memory remains challenging when evolving entities leave the field of view and later return. Existing memory metrics can assign similar scores to distinct recovery failures. We introduce a phase-aware evaluation protocol that separately measures entity reappearance, identity consistency, and post-re-entry temporal stability. Using this protocol, we find that recovery failures persist even when the complete observation history remains accessible. Controlled experiments show that entity-region-weighted training strengthens identity recovery, while stochastic sampling correction improves temporal matching. Applying these interventions selectively requires entity localization during denoising, when intermediate predictions remain noisy latent representations. Obtaining regions through RGB-based segmentation would require intermediate decoding and additional segmentation calls. These findings motivate ReentryMem, a self-localized framework that identifies entity regions directly from noisy video latents through text cross-attention. The shared localization mechanism guides entity-weighted flow matching during training and localized overshooting with re-noising during inference, without additional memory modules or architectural modifications. Experiments on two video backbones demonstrate the effectiveness of this approach. On the main evaluation, ReentryMem improves subject consistency from 0.8204 to 0.8378 and temporal consistency match from 0.9154 to 0.9318 over the baseline. Under block-causal attention, temporal consistency match increases from 0.8506 to 0.9116. Qualitative results further illustrate applicability to multiple entities, scene interactions, and changing backgrounds.
Method
- Localize the entity
- Read presence and shape from selected text cross-attention heads to estimate spatial support for the subject.
- Weight training supervision
- ReentryMem-Train increases the flow-matching loss weight inside the predicted entity support.
- Correct the sampled region
- ReentryMem uses the same trained weights and applies localized overshooting and re-noising during inference.
Experiments
Comparison with Other Models
| Method | Dynamic memory | Camera | Existing metrics | ||||
|---|---|---|---|---|---|---|---|
| Rea-Score ↑ | SC ↑ | TCM ↑ | Cam. Gain → 1 | ObjCons ↑ | SubjCons ↑ | DSC ↑ | |
| Released world models | |||||||
| Matrix-Game 3.5 | 90.6 | .7548 | .9144 | .638 | .6732 | .8156 | .8459 |
| EchoWM | 91.1 | .7495 | .8796 | .421 | .6551 | .7622 | .8199 |
| SANA-WM | 97.8 | .7734 | .8979 | .345 | .6493 | .7982 | .8178 |
| HY-WorldPlay 1.5 | 86.1 | .7780 | .9107 | .534 | .6122 | .8020 | .8490 |
| LingBot-World-V2 | 96.6 | .8115 | .9232 | .634 | .6402 | .7761 | .8361 |
| Memory-oriented method | |||||||
| HyDRA† | 78.9 | .7465 | .9071 | .668 | .6193 | .7669 | .8486 |
| Controlled comparison | |||||||
| Baseline | 96.1 | .8204 | .9154 | 1.064 | .6536 | .7658 | .8720 |
| ReentryMem-Train | 99.2 | .8380 | .9231 | 1.093 | .6520 | .7670 | .8757 |
| ReentryMem(Ours) | 97.2 | .8378 | .9318 | 1.091 | .6513 | .7659 | .8750 |
Rea-Score in percent. Best and second best follow the paper. † Reproduced with official code on our training set.
Controlled Comparison
Baseline uses the standard training objective and sampler. ReentryMem-Train gives more training weight to the predicted entity region. ReentryMem(Ours) uses the ReentryMem-Train checkpoint with region-localized overshooting during sampling.