The collection process of expert demonstrations for robot manipulation inevitably produces failure action trajectories. Existing methods either ignore failures, or reuse some failed trajectories through offline reinforcement learning or similarity-based method. However, it is often ignored that each failed trajectory may contain heterogeneous segments: some segments still make task progress, while others involve critical mistakes that the policy should avoid.
To this end, we propose CARF, a Contrastive Attraction–Repulsion Framework for failure-guided flow matching that explicitly separates and leverages both useful and harmful segments in each imperfect demonstration for policy training. CARF first learns an importance-based scorer from successful trajectories to estimate the contribution of each action segment to the task fulfillment. The scorer is then used to guide flow matching policy training, attracting the policy towards progressive segments while repelling it from failure-critical behaviors through a bounded contrastive objective. This attraction-repulsion process enables CARF to exploit both successful and failed demonstrations without coarse binary value bootstrapping or ambiguous similarity-based supervision.
Overview framework of CARF. After automatically labeling success dataset based on subtask segmentation, CARF trains a progress prediction scorer that provides importance score for observation-action chunking sequence, which is used for attraction-repulsion flow matching training.
Manipulation tasks for evaluating our method cover diverse manipulation modes, such as Basic Interaction, Visual Object Detection, Precise Manipulation and Long Horizon Capability.
CARF outperforms all baseline methods in both simulation tasks and real-world tasks, demonstrating its effectiveness in leveraging both successful and failed demonstrations for improved policy learning.
CARF addresses several key weaknesses of imitation learning, such as Accuracy, Smoothness, Stability, and Efficiency.
We visualize the sequence-level importance scores derived from the scorer predictions within a failed trajectory. We observe that the importance scores soar upon the completion of a subtask, while they drop below zero when obvious errors occur or the trajectory starts deviatingdrop below zero when obvious errors occur or the trajectory starts deviating, which is fully consistent with our expectation.
@article{zhao2025dexctrl,
title={DexCtrl: Towards Sim-to-Real Dexterity with Adaptive Controller Learning},
author={Zhao, Shuqi and Yang, Ke and Chen, Yuxin and Li, Chenran and Xie, Yichen and Zhang, Xiang and Wang, Changhao and Tomizuka, Masayoshi},
journal={arXiv preprint arXiv:2505.00991},
year={2025}
}