CW-WAMA Structured Contact-Wrench World Action Model
for Force-Sensitive Manipulation

Anonymous Authors

Video
Square peg insertion Power-adapter insertion Whiteboard wiping Spring-scale transport

Abstract

Force-sensitive manipulation requires contact and load information that is not provided explicitly by visual observations. We present CW-WAM, a world action model that jointly predicts video, motion, and contact wrench. Inspired by hybrid motion–force control, it contains a unified interaction module that treats both motion and contact wrench as modeling targets and predicts time-aligned motion–wrench chunks as control references. A pair-causal attention mechanism structures the motion–wrench dependencies, while a long fixed-length wrench memory retains past interaction information. The predicted wrench chunk acts as a force reference for a 20-Hz outer-loop admittance controller within a slow–fast control system that maintains force feedback and smooth motion between model queries. CW-WAM improves contact prediction and increases the insertion success rate from 25% to 70% for square pegs with 1-mm clearance per side and from 40% to 95% for power adapters, compared with WAM without wrench input. It also improves lifting-height accuracy across different loads. Our framework with an admittance controller reduces force-reference RMSE from 4.8 to 2.7 N for peg insertion and from 2.0 to 1.3 N for wiping, respectively, with smaller force fluctuations during successful execution.

Video

Real-robot evaluation and metrics

We select four tasks to test contact-guided alignment, precise seating, force regulation, and adaptation to payload.

Square peg insertion

The robot inserts a 38-mm square peg into a 40-mm hole with 1-mm clearance per side and an insertion depth of about 50 mm.

Power-adapter insertion

The robot grasps a loosely placed adapter and seats it in another socket. This tests precise positioning and insertion with contact.

Whiteboard wiping

The robot picks up an eraser and removes a marker trace through sustained surface contact.

Spring-scale transport

The robot grasps a spring scale attached to a box and transfers the box between bins. The boxes weigh either 30 g or 830 g but look identical.

CW-WAM: Contact-Wrench World Action Model

Wrench input and prediction

CW-WAM helps robots use contact forces to guide their movements.
Fig. 1. CW-WAM helps robots use contact forces to guide their movements. It learns from past motions and wrist force measurements to predict both how to move and the forces expected during contact. These predictions guide motion corrections during execution. Peg insertion success increases from 25% with WAM to 70% with CW-WAM; adding force feedback to CW-WAM reduces force-tracking error from 4.8 to 2.7 N.

Structured action–wrench modeling

CW-WAM jointly predicts video, motion, and wrench from visual observations, language, robot state, and interaction history. Paired motion–wrench tokens capture recent interactions, while long-term wrench memory retains earlier contact information.
Fig. 2. CW-WAM jointly predicts video, motion, and wrench from visual observations, language, robot state, and interaction history.

Recent paired history

The interaction module pairs past motion commands with measured wrenches from the same time interval.

Pair-causal attention

Pair-causal attention links each future wrench to its paired action and earlier pairs.

Long-term wrench memory

It retains preceding payload tension or sustained resistance.

Slow–fast motion–force control

The admittance controller computes pose corrections at 20 Hz. The corrected motion waypoints are then interpolated into smooth trajectories for a 1-kHz robot controller.

Admittance corrects nominal base-frame poses before IK.
Fig. 3. Admittance corrects base-frame poses before IK.

Task outcomes and execution quality

Top: insertion or marker-removal success out of 20 trials; hatched bars show successful release. Quality rows compare all 20 trials (light, hatched) with successful subsets (dark bars); these groups coincide for wiping. A dash indicates no wrench prediction.
Fig. 4. Top: insertion or marker-removal success out of 20 trials; hatched bars show successful release. Quality rows compare all 20 trials (light, hatched) with successful subsets (dark bars); these groups coincide for wiping. A dash indicates no wrench prediction.
Colored curves show measured force magnitudes across successful trials. Thick blue and green curves show CW-WAM without and with admittance, respectively. The black curve and gray band show the demonstration reference. Curves are aligned by contact phase, with low-force intervals compressed. Adapter unloading includes only trials with successful release.
Fig. 5. Colored curves show measured force magnitudes across successful trials. Thick blue and green curves show CW-WAM without and with admittance, respectively. The black curve and gray band show the demonstration reference. Curves are aligned by contact phase, with low-force intervals compressed. Adapter unloading includes only trials with successful release.
Curves show representative end-effector height trajectories. Dashed lines indicate approximate heights needed to clear the bin rim. The lower panels show peak heights from all ten trials per method and load. Diamonds mark the trials shown in the trajectories. Gray lines and bands show demonstration references.
Fig. 6. Curves show representative end-effector height trajectories. Dashed lines indicate approximate heights needed to clear the bin rim. The lower panels show peak heights from all ten trials per method and load. Diamonds mark the trials shown in the trajectories. Gray lines and bands show demonstration references.
Force and torque tracking for peg insertion (left) and whiteboard wiping (right). (a–f) Measured and reference magnitudes, with three-axis force RMSE labels. Each example is selected from the trial whose force variation is closest to the group median. (g,h) Force variation (solid bars, left axis) and torque variation (hatched bars, right axis). Aggregate bars summarize 10 trials per condition.
Fig. 7. Force and torque tracking for peg insertion (left) and whiteboard wiping (right). (a–f) Measured and reference magnitudes, with three-axis force RMSE labels. Each example is selected from the trial whose force variation is closest to the group median. (g,h) Force variation (solid bars, left axis) and torque variation (hatched bars, right axis). Aggregate bars summarize 10 trials per condition.

Citation

@misc{cw_wam,
  title = {{CW-WAM}: A Structured Contact-Wrench World Action Model for Force-Sensitive Manipulation},
  author = {{Anonymous Authors}},
  url = {https://cw-wam-project.github.io/}
}