The default move in learned sequential decision-making is to map a state directly to an action. That approach works well when the action space is small and fixed. It breaks down when candidate actions are numerous, geometrically structured, and rebuilt with every state transition. One-to-many mobile charging makes this problem concrete: a mobile charger must keep wireless sensor nodes alive by physically traveling to charging stops, where each stop simultaneously serves every sensor within its radius. With 250 sensors, the initial state induces roughly 1,125 candidate charging stops. Each chosen stop commits hundreds of seconds of travel, dwell time, and energy delivery. As sensors die irreversibly, the entire action universe is reconstructed from scratch at every decision.
LP-BTS (Learned-Prior Budgeted Tree Search) is a planning architecture designed for exactly this kind of problem. It does not try to map state to action in one shot. Instead, it divides the work: a graph proposal policy identifies a small set of promising candidates, a learned value critic evaluates shallow leaf states, and edge-budgeted PUCT tree search compares short simulated futures before committing to an action. The architecture is deliberately small. The policy has 39,681 parameters; the critic has 30,593. A single frozen checkpoint covers every evaluated setting, from action universes of 736 to 2,813 stops, because the policy scores candidates without a fixed output head.
Why Direct Policies Break Down in Dynamic Action Spaces
The standard approach in learning-based WRSN (Wireless Rechargeable Sensor Network) charging is to cellularize the field into a fixed grid, typically hexagons matching the charging radius, and train a network to pick the next cell and charging amount. This works, but it creates a hard coupling between the charging radius and the network architecture. A 30-meter radius at 200 sensors produces 471 cells and a 2.1-million-parameter network. Reduce the radius to 10 meters and the cell count jumps to 4,012, the parameter count to 17.6 million, and the trained checkpoint becomes unloadable. A separate model must be trained for each radius setting.
LP-BTS avoids this entirely. Its action space is not a fixed grid but a dynamically reconstructed set of physical charging stops derived from the geometry of alive sensors. At every decision, four families of geometric proposals are generated: each alive sensor's own position (atomic), midpoints of nearby sensor pairs within twice the charging radius (midpoint), intersection points of radius-R circles centered on sensor pairs (intersection), and centers of minimum enclosing circles for qualifying sensor triples (triple_center). These are then canonicalized: coordinates are quantized to a fixed grid, and proposals sharing a quantized coordinate collapse into one action. The result is 736 stops at 200 sensors, 1,125 at 250, and 2,813 at 400. The policy processes candidates in chunks of 256 with shared weights, so reordering candidates reorders scores and the scorer handles any universe size. One frozen checkpoint works everywhere.
The Graph Proposal Policy and Value Critic
Both networks are small graph neural networks operating on the same state description. Sensors are nodes with nine features: normalized coordinates, energy fraction, capacity, consumption rate, clipped time-to-death, alive flag, distance to charger, and current array index. A k-nearest-neighbor graph (k=12) carries five edge features. Six global features summarize time, charger position, charger energy, and alive fractions. No normalization layers, no dropout.
The policy scores each candidate stop by forming a relation embedding for every sensor, applying two max-pools (one unmasked, one restricted to in-range sensors), concatenating with candidate features and a graph vector, and passing through a two-layer MLP to produce a single logit. The softmax over all M logits yields the policy distribution. The value critic takes the mean sensor embedding and graph vector through a separate MLP with sigmoid output, bounding the prediction to (0,1) on the scale of terminal survival.
The key architectural choice is that candidates are scored through learned relation embeddings and pooled, not through a fixed classification head. This is what decouples the network from the action-universe size. The cost is that the policy cannot exploit the geometric structure of the full candidate set in a single forward pass; the benefit is universality across settings without retraining.
From Policy to Search: The Budgeted Planning Loop
The policy produces logits over all M candidates. A tempered, smoothed distribution is derived from these logits, and K=32 candidates are drawn with replacement, collapsing to typically 24 to 32 unique arms. At the initial decision with 250 sensors, this samples roughly 2.8% of the universe. The search then uses a sampling-corrected prior that divides empirical draw frequencies by their sampling probabilities, compensating for the tempering and smoothing, so the prior the search uses tracks the true policy weighting rather than the sampling distribution.
PUCT (c_puct=1.5) expands a tree over these sampled arms. Every tree-edge traversal executes one simulated environment transition: travel, dwell, drain, and forced base returns when the charger's energy reserve is insufficient. The budget is B_edge=2,048 transitions per decision, with a safety depth ceiling of 16 that is never binding. Measured mean leaf depth is 2.19, maximum 5. Under this budget the search chooses breadth over depth. Leaves are evaluated by the frozen critic. The visit-count argmax is executed.
Two diagnostics explain why shallow search helps. At B_edge=64, about 90% of sampled root arms are already explored (28 of roughly 31), but the share of tokens reaching depth 2 or greater rises from 21% at 32 edges to 78% at 128. The resolved positive survival difference over direct policy selection first appears at B_edge=128. Depth itself stays shallow: the useful computation is one-step verification of leading stops, not deep rollouts. Even at B_edge=1,024, the search chooses the same initial stop as the full 2,048-edge configuration in only 6 of 10 development scenarios, yet achieves similar survival. Search resolves continuation effects among near-equivalent actions rather than relying on one brittle root choice.
Training: Bootstrap, Cutoff, and One Autonomous Update
The training procedure has three stages, all conducted on a central-physics scenario family with 120 expert episodes (seeds 1000-1119, disjoint from every evaluation bank).
First, a handcrafted MCTS planner serves as teacher. It runs 64 simulations of depth 3 per decision with a hand-designed leaf evaluator and urgency prior, capped at 32 candidates. For each decision, the executed stop and its 32-action candidate set are recorded. A fresh policy is trained with a listwise target over that set plus 224 uniformly drawn negatives (executed stop 0.5, remaining teacher candidates share 0.5, negatives 0) under sampled-softmax cross-entropy. Adam, learning rate 1e-3, 8 epochs, one decision per step, final-epoch weights kept. This is checkpoint A.
Second, the expert is permanently removed. No expert labels, queries, or teacher-derived signals exist anywhere in subsequent training or evaluation. Checkpoint A, with the frozen critic, plays 40 self-play episodes (seeds 800-839), yielding 2,861 decisions. The policy is warm-started from A and fitted to the search's own root visit distribution over the sampled arms, again with 224 negatives and the same optimizer setup. Validation cross-entropy falls from 3.51 (checkpoint A) to 3.21. The final-epoch weights become the LP-BTS policy.
Third, the critic was trained in an earlier development generation from 79 self-play episodes under a different search operator, with terminal survival as the regression target. A refit on the method's own episodes improved offline metrics but left survival unchanged (delta of minus 0.0008, 95% CI [-0.0053, +0.0040], n=30) and increased travel. The pre-specified rule retained the simpler frozen critic.
The final LP-BTS configuration (internal identifier BK32) was fixed before the sensitivity grid was evaluated. On the central development scenarios, paired per scenario, S(LP-BTS) minus S(A) is minus 0.0012 (95% CI [-0.0108, +0.0096]): the teacher-free update yields a final policy whose inference is fully independent of the expert, without a measurable change in closed-loop survival.
Sealed Confirmatory Results on 30 Held-Out Scenarios
The evaluation contract was committed before the 30 confirmatory scenarios (generation seeds 600-629, central physics) were generated, and none of them appears in any training, development, or ablation artifact. Every method was evaluated once on every scenario on a single platform.
LP-BTS posts the highest observed survival (0.4545) and alive-AUC (0.8031). Its estimated survival advantage over HQARRF-SC, the strongest domain-engineered comparator (a crisis-aware hybrid scheduler with per-episode tabular Q-learning), is plus 0.0066 (95% CI [-0.0037, +0.0184]). The paired difference is unresolved: a few large LP-BTS gains coexist with many small losses, producing a positive mean and slightly negative median. The resolved-positive/unresolved/resolved-negative count across scenarios is 13/1/16.
Against every other baseline, LP-BTS wins on all 30 scenarios with intervals excluding zero: plus 0.050 over K-EDF (K-node earliest-death-first scheduler), plus 0.053 over reconstructed OTM3DQN (the source-derived one-to-many double-dueling DQN from Gong et al.), plus 0.123 over reconstructed RMP-RL-cell (the cell-based RL comparator), and plus 0.145 over the idle floor. Descriptively, LP-BTS also travels 15.5% less than HQARRF-SC (53,015 vs. 62,777 meters), though no movement interval was pre-specified.
Ablations: What Each Component Contributes
Four matched ablations isolate component contributions on the frozen central development scenarios (paired, n=10), executed on a separate Linux host with same-platform controls to avoid cross-platform numerical divergence.
Uniform sampling versus learned proposal: replacing the learned proposal with uniform-logits sampling at the same budget (2,048 edges) and same frozen critic costs 8.8 survival percentage points. With only 32 draws from roughly 1,125 candidates, uniform sampling cannot reliably place valuable stops in the planning support. Its movement is also lower (40,699 vs. 53,534 meters), so the result is not explained by the full method simply traveling farther.
PUCT search versus direct policy selection: holding the learned proposal support fixed, replacing PUCT with the policy argmax over the full universe sacrifices 0.014 survival, roughly 3.5 of 250 sensors, and increases travel by 23% (65,094 vs. 53,036 meters). Search adds a measurable continuation check beyond the learned proposal on both service retention and movement.
Budget saturation: the resolved positive difference over policy-only first appears at B_edge=128. From there upward, no budget is measurably better on this setting. At B_edge=512 (4 times smaller than the frozen configuration), mean survival is nearly identical (0.4460 vs. 0.4464). This characterizes cost against quality on one setting; B_edge=2,048 remains the frozen configuration as a conservative choice.
Bootstrap versus final checkpoint (A versus LP-BTS): the teacher-free update is statistically unresolved on central development, consistent with the reading that the update shifts policy behavior without changing closed-loop survival at this scale.
Where Lookahead Pays and Where It Does Not
The development grid reveals a coherent regime boundary through 25 non-radius settings. LP-BTS is resolved positive in all 25 against K-EDF, OTM3DQN, RMP-RL-cell, and the idle floor. Against HQARRF-SC, the strongest comparator, 4 settings are resolved positive, 18 unresolved, and 3 resolved negative. The patterns trace a boundary: lookahead pays where the charger's own resources make scheduling the binding problem, and a zero-latency heuristic suffices where the environment approaches a pure reaction race.
At low charger capacity (5,000), LP-BTS gains plus 0.021 over HQARRF-SC. At high capacity (20,000), it loses minus 0.034. At low charger speed (2.5 m/s), it loses minus 0.012; from speed 6.25 onward, point estimates are positive but unresolved. At extreme consumption rates (0.006), it loses minus 0.015; at moderate drain, differences are unresolved. The interpretation is straightforward: when resources are tight, planning where to go matters more than reacting to the nearest urgency. When resources are abundant or consumption is extreme, the cheapest heuristic wins because the environment rewards raw speed over foresight.
What Transfers Beyond Mobile Charging
The paper's central claim is not that LP-BTS is the best mobile charging scheduler. It is that in a large, dynamic, structured action space, dividing work between learned breadth selection and explicit lookahead produces measurably better behavior than either alone. The proposal policy handles the breadth of the candidate universe. The value critic evaluates states the search reaches. PUCT compares short simulated futures at decision time.
The architecture requires no fixed output head, no per-setting retraining, and no action-space discretization tied to a physical parameter. The frozen checkpoint covers universes from 736 to 2,813 actions with no modification. This is a practical property for any domain where the action set is rebuilt from the state geometry: robot manipulation with dynamically sampled grasp candidates, resource allocation with variable task sets, or logistics routing with state-dependent stop locations.
The ablation results suggest the benefit is robust to budget: 128 transitions per decision capture most of the survival gain, and the search resolves continuation effects among near-equivalent actions rather than relying on one brittle root choice. This is a useful property for deployment, where computation budgets are real.
Limitations
The primary comparison with HQARRF-SC is unresolved. A few large gains and many small losses produce a positive mean survival with a confidence interval spanning zero. The development-grid regime analysis is exploratory and not multiplicity-corrected. The critic was trained in an earlier development generation under different conditions and frozen; a refit improved offline metrics but not survival. The training procedure uses a single expert bootstrap and one autonomous update; the authors report no sweep of expert quality, update count, or critic freshness.
The evaluation is limited to simulation with a star topology (every sensor communicates directly with the base station) and no multi-hop relay-load effects. The scenario bank is 30 confirmatory scenarios under central physics. The five perturbed-radius families are held out of the grid because no learned comparator can appear beside LP-BTS on that axis.
Despite these constraints, the results provide controlled evidence that learning-guided planning, even with tiny networks and shallow search, can outperform both pure learning and pure heuristics in a setting where direct state-to-action mappings are structurally disadvantaged.