Code

Abstract

Motion planning in unstructured environments is a bottleneck for robot deployment. Current visuomotor policies predominantly rely on external cameras, introducing perceptual blind spots and fragility due to occlusions, clutter, and calibration constraints. To overcome these limitations, we propose ESCAPE, an online egocentric sensing framework for collision-free trajectory generation. ESCAPE aggregates sparse measurements from distributed depth sensors, converting them into a structured, obstacle-aware point cloud representation. Our policy leverages a flow matching-based generative framework, and trains on a high-quality dataset comprising over 100K safety-critical trajectories. With joint optimization of goal-reaching and collision objectives, it produces safe trajectories directly from these egocentric inputs. Crucially, ESCAPE is trained entirely in simulation and transfers zero-shot to physical hardware without any real-world fine-tuning. Extensive simulation and real-world evaluations demonstrate that ESCAPE achieves stable motion planning using only egocentric perception, outperforming external-view and voxelized-planning baselines across both simulation and real-world evaluations.


Method Overview

Image description

ESCAPE is an online flow-matching policy for safe motion generation in cluttered environments. First, distributed body-mounted depth measurements are temporally aggregated into a structured, collision-aware point cloud. Then, a DiT-X Transformer takes this egocentric representation, the joint goal, and the current joint position as inputs to generate action chunks. The framework is trained via multi-loss joint optimization to output safe trajectories.


Simulation Evaluations

We evaluate ESCAPE on a unified simulation benchmark of 937 Franka reaching problems in static cluttered environments.The benchmark is divided into Easy and Hard splits, with Hard targets placed closer to obstacle boundaries or narrow feasible regions.

Shelf_Hard

Microwave_Hard

Box_Hard

Cage_Hard

Cubby_Hard

Dishwasher_Hard

Wallcabinet_Hard

Real World Evaluations

We evaluate ESCAPE on a real Franka robot across six cluttered reaching scenes: ShelfEasy, ShelfHard, WallCabinet, Box, Cage, and Microwave, with 92 valid trials in total.

All real-world videos of ESCAPE are recorded at 1x speed.

Evaluation Scenes

We compare ESCAPE with NeuralMP, cuRobo-Vox, and DRP under the same task set, where ESCAPE is trained entirely in simulation and deployed zero-shot using only robot-mounted near-field sensing.

Shelf_Hard

Microwave

Baselines

DRP

NeuralMP

cuRobo-Vox

Success Rate: ESCAPE 91.30% | DRP 71.74% | NeuralMP 8.70% | cuRobo-Voxels 47.83%

Mixed Scenes

We introduce mixed scenes unseen during training to validate the model’s generalization capability to unknown environments.

Microwave & Box

Microwave & Cage

Wallcabinet & Box

ESCAPE Failure Cases

The obstacle avoidance performance of ESCAPE regarding the gripper has room for improvement; the gripper is prone to minor collisions with protruding structures.