ASPIRE: Agentic /Skills Discovery for Robotics

1. Introduction: The Granularity Gap in Robotic Debugging

In the current landscape of embodied AI, traditional “code-as-policy” (CaP) systems are hitting a wall known as the “granularity gap.” As Lu et al. (2026) observe, most autonomous agents are forced to learn from coarse, task-level feedback—essentially a binary signal of success or failure at the end of a rollout. The analogy is debugging a complex distributed system with nothing but a “500 Internal Server Error” message. It lacks the necessary failure attribution to determine if a missed grasp was caused by a perception shift, a motion-planning conflict, or poor contact dynamics.

Human engineers do not debug in such a vacuum; they perform a “fail-fast” analysis by inspecting perception overlays and motion traces. To bridge this gap, the authors introduce ASPIRE (Agentic Skill Programming through Iterative Robot Exploration). Built on the open-source CaP-X framework and the MuJoCo Playground simulator, ASPIRE is designed to autonomously inspect fine-grained execution traces and transform failures into a persistent, evolving library of robotic expertise.

2. The Three Pillars of the ASPIRE Framework

The ASPIRE architecture moves beyond simple trial-and-error by implementing a closed learning loop supported by three technical pillars:

• The Robot Execution Engine

  • Mission Statement: Replace coarse rollout-level feedback with per-primitive multimodal evidence to enable autonomous failure localization and repair validation.
  • This engine provides the agent with “multimodal traces”—including RGB keyframes, grasp candidates, and object poses—for every perception, planning, and control call. This allows the agent to pinpoint the specific causal link in a failure chain.

• The Skill Library

  • Mission Statement: Distill validated fixes into persistent, reusable robotic knowledge that provides in-context guidance for future tasks, bypassing the “rediscovery” phase.
  • Rather than storing ephemeral task code, the library preserves heterogeneous skills—such as localization heuristics and navigation recovery strategies—allowing the agent to perform cross-embodiment zero-shot transfer.

• Evolutionary Search

  • Mission Statement: Explore diverse program hypotheses by conditioning the next generation of strategies on both surviving successes and residual failure traces.
  • To prevent the agent from collapsing into “local repair loops,” this component proposes a population of candidate programs. It uses trace-guided evolution to ensure that the search is informed by what specifically went wrong in previous iterations, not just whether they failed.

3. Case Study: The Multi-Angle Approach to Failure Recovery

The authors demonstrate the effectiveness of trace-guided debugging through a “navigate-and-pick-up-radio” task in the BEHAVIOR-1K benchmark. The following walkthrough highlights the autonomous diagnosis-and-repair cycle:

1. The Initial Failure: The robot identifies the radio but fails to approach it. Traditional feedback would simply report a task failure.
2. The Multimodal Diagnosis: The ASPIRE agent inspects the primitive logs from the cuRobo collision engine. It localizes a repeated PLANNING_ERROR and identifies that the navigation goals fall within a 20cm collision-avoidance buffer surrounding the table.
3. The Program Patch: The agent synthesizes a “Multi-Angle Approach” routine. It samples alternative navigation targets at 90° and 180° offsets. By re-perceiving the radio from a reachable side, the agent identifies a clean path that clears the table’s collision buffer.
4. Skill Admission: Once the radio is successfully grasped and the patch validated, the coordinator admits the “Multi-Angle Approach” into the skill library as a general strategy for obstacle-boundary recovery.

4. Quantifying Robustness: Benchmark Performance

ASPIRE reports substantially higher success rates than the baseline CaP-Agent0 and Vision-Language-Action (VLA) models like $\pi0.5$. Notably, ASPIRE surpasses prior methods by up to 77 points on specific LIBERO-Pro suites.

Benchmark Suite	Metric Focus	Baseline (CaP-Agent0)	Baseline (VLA - $\pi0.5$)	ASPIRE
LIBERO-Pro	Robustness under Perturbation	18%	13%	72%
Robosuite	Bimanual Handover	20%	N/A	92%
BEHAVIOR-1K	Long-horizon Manipulation	56%	N/A	88%
LIBERO-Pro Long	Zero-Shot Transfer	4%	N/A	31%

Note: Success rates are macro-averaged. The 72% score on LIBERO-Pro represents a macro-average across all perturbation axes, while the 77-point delta refers to the maximum improvement over baselines in specific sub-suites.

5. The Power of Persistence: Zero-Shot Transfer and Sim-to-Real

The scaling laws of ASPIRE are evident in its library progression. As the library grows from $N=0$ to $N=90$, success rates on unseen tasks rise steadily. On the LIBERO-Pro Long benchmark, ASPIRE’s 31% success rate—achieved without test-time reasoning—improves on the 4% best prior result by 27 points.

The framework’s most compelling result is its “Cross-Embodiment” transfer. Skills discovered in a Franka-based simulation were applied to a real-world YAM manipulation station using OpenAI Codex GPT-5.5 (reasoning-xhigh mode). By retrieving simulation-derived guidance, the agent bypassed the “rediscovery” of basic mechanics (like drawer manipulation), reducing the total token cost from 334.91M to 81.67M. In the “lift soda can” task, the reasoning effort fell from 61.94M tokens to 6.58M, demonstrating that simulation-discovered repairs can significantly lower the overhead of real-world deployment.

6. Technical Limitations and Research Frontiers

For practitioners, several hurdles remain before ASPIRE can reach fully autonomous lifelong learning:

• Frontier Model Dependency: The system currently relies on high-frontier LLMs (specifically Claude Opus 4.6). Whether smaller, locally-hosted models can interpret complex multimodal traces remains an open question.
• The “Safe Reset” Problem: Unlike simulation, real-world environments lack programmatic scene resets. Autonomous learning is currently bottlenecked by the need for human-led resets and safety monitoring.
• Success Detection: Real-world autonomy requires robust, automated verification that a task was actually completed, which is often as complex as the task itself.
• Stale Memory Management: As the skill library expands, the system must navigate redundant or misleading entries. Future research must address the pruning and ranking of “stale” knowledge.

7. Conclusion: Takeaways for Failure-First Research

The work by Lu et al. (2026) offers three critical takeaways for the future of autonomous systems:

1. Fine-grained multimodal traces are non-negotiable. Binary success/failure signals are insufficient for the failure attribution required to solve complex, contact-rich manipulation.
2. Trace-guided evolutionary search prevents local minima. By conditioning exploration on residual failure traces, agents can iterate toward fundamentally different strategies rather than repeating flawed patterns.
3. Skill persistence enables cross-embodiment scaling. Distilling validated repairs into high-level guidance allows robots to inherit experience across different APIs and physical embodiments, providing a viable path to reducing real-world programming costs.

Read the full paper on arXiv · PDF