CoRL 2026 Sim-to-Real-to-Field Workshop | Late-Breaking Track

When Looking Changes the Grasp

We compare two physical interventions from the same simulated manipulation state to examine how acquiring a clearer view can affect subsequent task completion.

Condition G: Clearance only
Gripper commanded closed. Arm executes a 25-step clearance motion while maintaining grasp clamp. The red cube moves with the gripper into the clearance pose, allowing frozen SmolVLA to resume and complete pick-and-place.
Condition H: Open + clearance
Gripper commanded open. Arm executes the identical 25-step clearance motion while forcing jaws open. The cube remains on the table as the gripper withdraws, stranding the robot out of reach where the policy times out.
Aggregate Experimental Result: Clearance only: 5/6 full-task successes. Open + clearance: 0/6.

Opening the gripper alone (5/6) and moving to clearance alone (5/6) each achieved 5/6 full-task successes. Equal aggregate counts do not imply identical scene outcomes. Combining gripper opening with arm withdrawal resulted in 0/6 full-task successes.

Experimental Scope: All experiments are conducted in MuJoCo simulation. The selected video illustrates one matched continuation; the aggregate results summarize six reused validation scenarios.

Publication Figure: Matched Intervention and Causal Ablation

Matched Physical Intervention and Five-Condition Causal Ablation

Figure: (a) Matched physical intervention starting from identical tabletop contact at t = 293 and branching into Condition G (clearance only, cube lifted) vs Condition H (open + clearance, cube remains on tabletop at t = 318). (b) Five-condition causal ablation across six matched scenes showing that only compound jaw opening and arm retreat causes task failure. (c) Offline passive observability comparison across 14 benchmark grasp attempts.