Project page · Preprint

BadWAM: When World-Action Models Dream Right but Act Wrong

¹National University of Singapore · ²The Hong Kong Polytechnic University · ^†Corresponding author

BadWAM shows that a world action model can appear to imagine a plausible future while executing actions that have been adversarially shifted toward task failure.

Paper PDF Code BibTeX

BadWAM method overview — BadWAM injects small visual perturbations and searches online over a frozen WAM to desynchronize the action pathway from the imagination pathway.

Abstract

Future prediction is not automatically a safety mechanism.

World-action models (WAMs) couple action generation with future world prediction, a design often viewed as a source of robustness, interpretability, and safety. BadWAM challenges this assumption. It models World-Action Drift Attacks: a new class of WAM-specific adversarial attacks that use small visual perturbations to break the alignment between what a WAM imagines and what it executes.

BadWAM instantiates two complementary objectives. The action-only adversarial attack prioritizes disruption by driving the model toward task-failing actions. The imagination-preserving adversarial attack additionally keeps the predicted future close to the clean imagination, producing a stealthier failure mode. Across WAM variants and closed-loop robot tasks, BadWAM substantially reduces task success and shows that plausible imagined futures alone are not sufficient evidence of safe execution.

Method

BadWAM attacks the synchronization between imagining and acting.

Action-only adversarial attack

The attacker maximizes the deviation between clean and attacked action chunks under a bounded visual perturbation. This captures high-strength action hijacking when disruption is the main goal.

Imagination-preserving adversarial attack

The attacker still shifts the action, but regularizes the predicted future to remain close to the clean imagination. This creates a stealthier WAM-specific failure.

Closed-loop execution

Small replan-level shifts accumulate through robot execution, especially on spatial and long-horizon tasks where timing and geometry leave little tolerance for action drift.

Action and future statistics — Failed episodes tend to have larger action shifts, while predicted-future shifts can overlap with successful episodes.

Action component analysis — The attack induces structured shifts across action channels and horizons, rather than unstructured noise.

Main results

BadWAM reliably degrades closed-loop task success.

Action-only WAM 96.5 → 43.1

LIBERO success under action-only attack.

Joint WAM 98.1 → 61.5

LIBERO success under action-only attack.

IDM WAM 98.4 → 66.1

LIBERO success under action-only attack.

RoboTwin 92.1 → 84.4

Action-only WAM success under the same attack protocol.

Task-level failure profile: attacked tasks move into lower success bins.

Per-suite success heatmap — Spatial and long-horizon suites are especially vulnerable to accumulated action errors.

Success over repeated trials by suite — The success gap persists as more trials are averaged, showing that BadWAM is not merely exploiting a few unlucky seeds.

Analysis

Attack strength and stealthiness form a measurable trade-off.

Matched-strength stealth dynamics — Under matched perturbation and query budgets, the imagination-preserving objective maintains comparable attack strength while reducing predicted-future drift.

Efficiency and perturbation budget — Query budget and perturbation budget shape the strength-runtime frontier.

Qualitative cases

BadWAM failures are structured and visually interpretable.

Clean and attacked action case study — A representative action-failure case: the attacked trajectory remains visually plausible early on, then drifts into task failure through accumulated action error.

Predicted future without preservation — Without preservation, the predicted future drifts more visibly.

Predicted future with preservation — With preservation, the imagined future remains closer while the action still fails.

Qualitative examples

BadWAM failures remain visually plausible during execution.

We show cropped rollout videos comparing clean and attacked executions.

Action-only attack Example 01. LIBERO Spatial: pick the black bowl. The left rollout is clean; the right rollout is attacked.

Imagination-preserving attack Example 02. LIBERO-10: put the cream cheese and butter. The left rollout is clean; the right rollout is attacked.

Citation

BibTeX

@article{li2026badwam,
  title   = {BadWAM: When World-Action Models Dream Right but Act Wrong},
  author = {Li, Qi and Yang, Xingyi and Wang, Xinchao},
  journal = {arXiv preprint arXiv:2607.15207},
  year    = {2026}
}