Deep-space Communications and Relay Resource Management
Deep-space communications operate under intermittent connectivity, long propagation delays, and relay storage constraints. In such networks, the Bundle Protocol enables store-carry-forward communication, requiring relay nodes to manage limited storage and scarce contact opportunities. Relay-side scheduling and dropping decisions are inherently goal-oriented: transmission resources should be allocated to bundles most valuable to downstream mission objectives. Conventional schedulers, such as FIFO, do not explicitly optimize task utility and may provide unfair service across heterogeneous planetary sources.
This paper presents a closed-loop modeling and learning framework for relay-side bundle and flow management in a contact-driven deep-space network, where lunar and Martian updates are forwarded through a shared-buffer relay to Earth. These bundles are stored in a shared relay buffer and carry metadata including source and destination identities, generation time, and time-to-live.
Excess-Age Objective and Information Freshness
Destination-side information freshness is quantified using excess age Δ_i^e(t) = [Δ_i(t) - d_i(t)]^+, where Δ_i(t) denotes the Age of Information of source i and d_i(t) is a source-dependent delay baseline. The Age of Information captures the time elapsed since the freshest update currently available on Earth was generated. A policy-independent baseline d_i(t) is derived from the known contact plan, allowing separation of relay-controlled staleness from source-dependent contact and propagation conditions.
Based on this metric, a goal-oriented system loss is defined that penalizes excess age in a source-dependent and nonlinear manner. This allows the relay to prioritize information according to mission relevance. Over an episode, the relay minimizes the expected cumulative freshness cost, subject to operational and source-specific constraints.
Constrained SMDP Formulation
Relay control is formulated as a constrained semi-Markov decision process (CSMDP). Decision epochs occur at irregular times determined by contact changes, bundle arrivals, transmission completions, Earth deliveries, and expirations. The relay observes a state constructed from buffer contents, service status, and freshness variables. Feasibility-aware action masking restricts decisions to transmissions that can complete within the remaining contact window.
The constrained objective incorporates returns and constraint costs related to bundle drops and expirations. Policy optimization uses a Lagrangian approach to handle these constraints while maximizing the freshness-based reward. The event-driven formulation preserves the delayed effect of relay actions and the changing set of feasible decisions across contact-driven dynamics.
Masked Lagrangian PPO with Variable-Cardinality Set Pooling
The problem is solved using masked Lagrangian Proximal Policy Optimization (PPO) with permutation-invariant encoding of the variable-size buffer state. Stage-specific feasibility masking restricts the action set to bundles that can finish transmission before contact termination. Causal observation and set pooling enable the policy to process the unordered buffer contents as a permutation-invariant set.
Holding-time-aware Lagrangian PPO incorporates the duration between decision epochs into the optimization. The Lagrange-multiplier update adjusts constraint penalties based on constraint violation observed during training. The training procedure alternates between policy improvement and constraint violation reduction.
DTNSim Simulations and Results
DTNSim simulations indicate improved age-based utility and inter-flow fairness compared with baseline schedulers. The proposed framework jointly optimizes relay-side bundle scheduling, dropping, and flow control under contact-driven dynamics with explicit fairness considerations across heterogeneous sources. Simulation results demonstrate that the approach significantly improves age-based utility and achieves substantially better inter-flow fairness compared to conventional schedulers such as FIFO.
Operational performance metrics show the framework respects contact window constraints while delivering information freshness gains. Constraint compliance is maintained across all evaluated scenarios, and the framework handles the heterogeneous traffic patterns generated by lunar and Martian sources.
Conclusion
The proposed framework provides a goal-oriented approach to relay-side bundle and flow management in deep-space networks. By explicitly prioritizing information according to downstream task relevance and using this signal to guide transmission and dropping decisions, the framework improves goal-oriented information delivery. The combination of excess-age objective, constrained SMDP formulation, and masked Lagrangian PPO enables joint optimization of scheduling, dropping, and flow control under contact-driven dynamics with explicit fairness considerations across heterogeneous sources.