Distributed Multi Robot Lunar Cargo Transportation via Phase Decomposed Reinforcement Learning

1. Introduction: The Challenge of Lunar Surface Operations

Sustained lunar surface missions necessitate robotic systems capable of autonomous construction and logistical support under significant terrain uncertainty and communication delays. The transportation of heavy payloads—ranging from construction materials to sensitive scientific instruments—across the lunar regolith requires a high degree of coordination and mechanical adaptability.

Modular reconfigurable robots have emerged as a primary solution for these environments due to their repairability, redundancy, and functional adaptability. By assembling task-specific morphologies from standardized units, these systems can theoretically meet diverse operational requirements. However, as demonstrated by Mishra et al., the complexity of coordinating distributed robotic units increases exponentially when they are physically coupled by a shared payload. This configuration creates a “closed kinematic chain,” where the motion of any individual subsystem generates immediate dynamic interactions across the entire assembly, often leading to mechanical instability or mission failure.

2. The “Monolithic” Failure: Why Standard AI Struggles with Multi-Stage Tasks

Standard Reinforcement Learning (RL) frameworks typically utilize a “monolithic” policy designed to manage the entire mission horizon. The authors argue that such a unified approach is fundamentally ill-suited for cooperative lunar transport for two technical reasons:

• Configuration-Dependent Kinematics: The physical coupling via the cargo alters the actuation topology, making it mathematically difficult for a single policy to generalize across the shifting contact regimes of a mission.
• Gradient Interference across Heterogeneous Dynamics: The physics governing different mission stages are distinct. The transition from a static, force-intensive “lifting” regime to a dynamic “transportation” regime (characterized by rolling friction and wheel slip) creates conflicting mathematical gradients during optimization. This “gradient interference” prevents a single policy from converging on a stable solution that satisfies the requirements of both regimes.

The Baseline Breakdown

In evaluations of a single centralized policy trained without phase decomposition (referenced as Figure 8 in the authors’ study and Figure 11 in the source data), the system failed to achieve stability. Over a training horizon of $7.5 \times 10^4$ timesteps:

• Non-Convergent Rewards: Mean rewards remained highly oscillatory with no evidence of a plateau.
• Conflicting Behaviors: Robots exhibited unstable behavior, such as attempting forward translation while simultaneously executing lifting or placement maneuvers, resulting in immediate payload drops.

3. The Methodology: Breaking Down the Mission into Phases

To mitigate gradient interference, the authors propose a Phase-Decomposed Reinforcement Learning framework. The mission is partitioned into three distinct Markov Decision Processes (MDPs): Lifting, Transportation, and Placement. Each stage is optimized independently, allowing the dedicated policies ($\pi_L, \pi_T, \pi_P$) to capture the specific physics of their respective contact regimes.

Phase	Primary Objective	Key Reward Constraints
Lifting ($r_L$)	Raise cargo to target height $h^*$	Minimizes height error $\left(\frac{h_t - h^*}{h_{tol}}\right)^2$, vertical velocity $\dot{h}_t$, and cargo tilt $\theta_{tilt}$. Uses force variance $\sigma_f$ to enforce balanced load sharing.
Transportation ($r_T$)	Planar motion from $p_A$ to $p_B$	Minimizes position error $\left(\frac{\|p_t - p_B\|}{p_{tol}}\right)^2$ and velocity deviation from $v_{cmd}$. Regulates height $h_t$ and tilt $\theta_{tilt}$ for stability.
Placement ($r_P$)	Controlled lowering and alignment	Penalizes residual height $h_t$ and vertical speed $\dot{h}_t$ to ensure soft contact. Enforces alignment via $\theta_{tilt}$ constraints.

The sequence is managed by a Phase MDP Gating mechanism. A discrete mode variable $m_t \in \{0, 1, 2, 3\}$ (representing Idle, Lift, Transport, and Place) regulates the system. The active action $a_t$ is selected via the following indicator gating equation: $a_t = \mathbb{I}[m_t = 1]\pi_L(s_t) + \mathbb{I}[m_t = 2]\pi_T(s_t) + \mathbb{I}[m_t = 3]\pi_P(s_t)$ Transitions between these modes are triggered by measurable task events, such as the payload height $h_{payload}$ reaching a minimum threshold $h_{min}$, while a safety-stop signal $\sigma_t$ can immediately revert the system to an “Idle” state.

4. Ensuring Physical Safety: The Joint Synchronization Layer

Even with decomposed policies, uncoordinated movements in a closed kinematic chain can induce “internal cargo stress,” where robots inadvertently work against one another. To prevent this, the authors implement a deterministic Joint Synchronization layer that acts as a physical safety “clamp.”

This layer restricts the RL policy’s raw commands to an admissible region ($V_k$) centered on the real-time sensed state of the robots. The controller calculates the progress of four subsystems (two arms and two wheel bases) using an $\ell_\infty$ maximum distance calculation. This distance $d$ reaches its limit when any subsystem hits a normalized tolerance:

• Arm translation error ($d_{max}$): $0.03\text{ m}$
• Arm rotation error ($\theta_{max}$): $5^\circ$
• Wheel deviation ($\Delta_w$): $[0.02, 0.02]^\top\text{ m}$

The final command $U_k$ is derived through interpolation $T(t, S_k, F_k)$, where $t_{max}$ represents the largest feasible step toward the target: $t_{max}^k \triangleq \max \{ t \in [0, 1] \mid T(t, S_k, F_k) \in V_k \}$ Crucially, the subsystem closest to its deviation bound caps the value of $t_{max}$, effectively slowing the entire assembly to the speed of the lagging unit and preventing the asynchronous motion that leads to mechanical failure.

5. From IsaacLab to JAXA: Simulation and Hardware Validation

The authors utilized a robust sim-to-real pipeline to validate the framework:

• Simulation: Environments were instantiated in NVIDIA IsaacLab using 2048–4096 parallel environments. Domain randomization included friction coefficients $(\mu \in [0.6, 1.0])$, terrain heightfield perturbations, and actuation noise. Analysis of the lifting phase revealed that the plateau reward decreased as target height increased (e.g., from 105.9 at 0.3 m to 94.7 at 0.75 m) due to higher torque demand and increased sensitivity to coupling.
• Hardware: Experiments were conducted at the JAXA Advanced Facility for Space Exploration on granular, non-uniformly compacted soil.

The researchers tested three configurations: a Sledge (5.3 kg), a Cargo Box (7.5 kg), and Cargo with Bricks (9.5 kg).

Performance Metrics:

• Velocity Tracking: The transportation policy achieved an RMS velocity tracking error of 0.023 m/s (approx. 11% of command).
• Height Retention: Height retention for the 5.3 kg sledge was 91%, but dropped to 70% for the 9.5 kg bricks. The authors attributed this degradation to terrain sinkage and “amplified arm torque coupling” under heavy loads.
• Placement Safety: The system achieved a soft-touchdown impact velocity of 0.030 m/s.
• Residual Height: The 9.5 kg load showed a higher residual height (12.1 mm) due to “sand penetration” and local terrain deformation.

6. Key Takeaways for AI Safety and Embodied AI

• Decomposition as a Stability Tool: Splitting tasks into phase-specific MDPs prevents gradient interference across heterogeneous dynamics, allowing for stable convergence in contact-rich environments.
• The Necessity of Synchronization: In closed kinematic chains, high-level AI policies must be constrained by low-level “safety clamps.” Without the $\ell_\infty$ synchronization bound, uncoordinated motion inevitably leads to internal cargo stress and mechanical decoupling.
• Generalization to New Geometries: The system successfully handled the “sledge” configuration despite it having a different geometry than the training model, indicating that the phase-structured approach captures fundamental physical coordination.

7. Conclusion: A Step Toward Autonomous Lunar Bases

The research by Mishra et al. demonstrates that a structured, phase-decomposed approach to reinforcement learning is essential for managing the complex interactions of modular robotic teams. By integrating a deterministic synchronization layer, the authors achieved a stable sim-to-real transfer for tasks that typically fail under monolithic optimization. This methodology provides a vital blueprint for the deployment of autonomous robotic workforces in extreme, unmodeled environments.

---

Citation: Mishra, A., Neppel, E., Santra, S., Jonquières, A., Naufal, M. A., Uno, K., & Yoshida, K. (2026). Distributed Multi Robot Lunar Cargo Transportation via Phase Decomposed Reinforcement Learning. Presented at IROS 2026 / Available at arXiv:2607.00160.

Read the full paper on arXiv · PDF