Multi-Agent Flow Matching with Decoupled Generative Guidance

Generative modeling has become a cornerstone for producing diverse objects from complex, multimodal distributions. Diffusion models and flow matching frameworks have demonstrated remarkable success across domains ranging from language and vision to robotics and scientific discovery. Yet a persistent limitation remains: the expressivity of these models does not, in general, come with formal guarantees that generated objects satisfy hard constraints or requirements. In single-agent settings, guidance mechanisms such as control barrier functions can impose constraints during the generative process. In multi-agent generation, the problem becomes substantially more challenging because a hard requirement can depend on multiple agents interacting together, while each agent must determine its own guidance input without relying on the simultaneously computed inputs of its teammates.

When a requirement couples multiple agents—such as ensuring that a team of robots can jointly traverse a gap or that a set of objects can be arranged so that each is independently accessible—naively applying centralized guidance defeats the primary advantage of multi-agent systems: decentralization. A single point of failure would compromise the entire team. Conversely, completely agent-wise guidance without team-level guarantees risks leaving coupled requirements unmet. The core tension is therefore how to let each agent operate autonomously while still providing provable guarantees on team-level constraints.

DeGG-Flow: A Framework for Decoupled Generative Guidance

To address this tension, the authors introduce DeGG-Flow, a general framework for multi-agent flow matching with decoupled generative guidance. Its central insight is to represent the generative process as a control-affine dynamical system and adapt disentangled control from multi-agent physical-space control into the generative process. This adaptation yields two classes of decoupled guidance: shared-entangled (SE) guidance for requirements whose satisfaction depends on multiple agents together, and private-entangled (PE) guidance for requirements associated with each individual agent that can be affected by its neighbors. For each class, the framework establishes feasibility conditions and finite-horizon convergence guarantees, and derives a Wasserstein bound that characterizes the distributional deviation induced by the guidance.

The guidance is applied directly during generation, without collecting new training data, retraining the learned generative model, or repairing generated objects afterward. Each agent computes its own guidance input using only locally available information and the nominal flow‑matching vector field, while the team-level guarantees emerge from the coordinated application of agent‑wise constraints.

Shared‑Entangled Guidance for Coupled Task Requirements

SE guidance is designed for scenarios where multiple agents contribute to shared goals or requirements. A typical example is multi‑robot collaborative policy generation: a team of robots must jointly manipulate a plank to construct a bridge over a spatial gap. In such settings, the task success and safety of the team depend on the coordinated actions of all robots, yet each robot should generate its policy independently.

The guidance input for agent i is derived from a factor graph that encodes which agents participate in which shared requirements. For each factor a in the graph, a continuously differentiable function q_a(τ, z_𝒮a|χ) represents the shared requirement, where q_a ≤ 0 means the requirement is satisfied. A time‑varying upper bound β_a(τ|χ, z(0)) is introduced and initialized from the initial generative state, then tightened toward a prescribed terminal value—specifically, choosing β_a(1|χ, z(0)) = 0 ensures the hard requirement is satisfied by the end of the generative process.

The agent‑wise constraint takes the form ∇_ziV_a(τ, z_𝒮a|χ)⊤(f_i^θ(τ, z|χ) + g_i(τ, z|χ)u_i) + ζ_a,i(τ, z|χ) ≤ 0, where the term ζ_a,i captures the time‑derivative of the bound and a class‑𝒦_∞ function α_a that shapes the constraint. Critically, when multiple SE factors are simultaneously active at an agent, their local guidance conditions may compete for the same input. The framework resolves this competition through an adaptive constraint allocation approach: effective directions are constructed to decompose competing requirements, and allocation coefficients w_a,i are set proportionally to the squared norm of each effective direction. This ensures that each active factor receives a fair share of the guidance budget, and the modification is inactive when the effective directions are sufficiently aligned.

Under the assumption that each agent encounters at most two simultaneously active SE factors, and that the effective direction for each active factor is non‑zero when its local constraint violation is positive, a unique guidance input exists that satisfies all active SE inequalities. This guidance can be obtained by solving a quadratic program with linear constraints, and the existence result is constructive: the minimizer of u_i^⊤H_iu_i subject to (ℓ_a,i)⊤u_i + b_a,i ≤ 0 for all active factors a admits a unique solution when H_i ≻ 0.

Finite‑horizon convergence is guaranteed by Theorem 3.1: if every agent along the trajectory satisfies its agent‑wise SE constraint for all τ ∈ [0,1], and the terminal bound β_a(1|χ, z(0)) = 0, then the shared requirement q_a(1, z_𝒮a(1)|χ) ≤ 0, i.e., the final generated state satisfies the coupled requirement.

Private‑Entangled Guidance for Individual Affordance Requirements

PE guidance addresses settings where each agent has its own requirement, and whether that requirement is satisfied can depend on the states of neighboring agents. A representative task is multi‑object scene generation: each object must satisfy affordance requirements (such as unobstructed access regions), and these requirements may be affected by the positions of nearby objects. Unlike SE guidance, which deals with shared factors, PE guidance assigns each agent a single inequality constraint based on its private violation function V_i^PE(τ, z|χ) ≥ 0.

The total violation is aggregated as V^PE(z|χ) = ∑_i∈𝒩 V_i^PE(z|χ), so that V^PE = 0 means all private requirements are satisfied. The guidance coefficient is chosen as α^PE(s) = c^PEs^ρPE with 0 < ρ^PE < 1; this subadditive form enables finite‑horizon convergence. The time‑varying weight ω^PE(τ) modulates the required decrease of V^PE over the generative horizon.

The agent‑wise constraint takes the form ∇_ziV^PE(τ, z|χ)⊤(f_i^θ(τ, z|χ) + g_i(τ, z|χ)u_i) + ζ_i^PE(τ, z|χ) ≤ 0, where ζ_i^PE = ∂_τV_i^PE(τ, z|χ) + ω^PE(τ)α^PE(V_i^PE(τ, z|χ)). If the local constraint violation b_i^PE ≤ 0, the nominal input u_i = 0 already satisfies the constraint. When b_i^PE > 0, a feasible guidance input exists provided the effective direction ℓ_i^PE ≠ 0, and this guidance can again be obtained by solving a quadratic program with a single linear constraint.

Theorem 3.3 establishes finite‑horizon convergence for PE guidance: if every agent satisfies its agent‑wise PE constraint for all τ ∈ [0,1], and the integrated guidance strength c^PE(1−ρ^PE)∫₀¹ ω^PE(s) ds ≥ (V^PE(z(0)|χ))^1−ρPE, then V^PE(z(1)|χ) = 0, meaning all private requirements are satisfied at the end of the generative process.

Bounding the Distributional Deviation

Any modification to the nominal vector field inevitably shifts the generated distribution away from the unguided one. To characterize this deviation theoretically, the framework derives a Wasserstein‑2 bound. Consider nominal and guided joint generative state trajectories initialized from the same random sample ξ ∼ p₀. The guided dynamics add a joint guidance correction Γ(τ|ξ), which stacks the per‑agent guidance inputs g_i(τ, X_gui(τ)|χ)u_i(τ). Under suitable Lipschitz assumptions on the nominal vector field and integrability conditions on the guidance correction, the 2‑Wasserstein distance between the guided and nominal distributions at τ = 1 satisfies

W₂(μ₁, ν₁) ≤ ∫₀¹ exp(∫ₛ¹ L(r) dr) ⋅ √𝒵ᴺ ‖Γ(s|ξ)‖² p₀(ξ) dξ ds

where L(τ) is the integrated Lipschitz constant of the nominal vector field and Γ(s|ξ) is the guidance correction at step s for initial sample ξ. An additional decoder‑Lipschitz constant L_D extends the bound to the output space: W₂(D̂_N(X̂_nom(1)|χ), D̂_N(X̂_gui(1)|χ)) ≤ L_DW₂(μ₁, ν₁). Empirically, the distributional deviation between the nominal and guided vector fields is evaluated in the application suites, and the observed deviations remain within theoretically predicted bounds.

Application 1: Multi-Robot Policy Generation with Shared Requirements

The first application evaluates SE guidance on a collaborative robot task inspired by disaster response and search‑and‑rescue scenarios. A team of robots faces a spatial gap that cannot be crossed directly; they must collaboratively use a plank to construct a traversable bridge. The generated object is a motion policy specifying robot contact‑point velocities on the plank. The condition χ contains the current robot and plank states and environment information. Only the first few actions of each generated policy are executed before a new policy is generated—a common practice in model‑predictive fashion.

Two SE factors encode the coupled requirements: q_task(z|χ) evaluates task progress at the plank pose reached after executed actions, and q_safe(z|χ) evaluates safety along these actions. The factor graph has two factors, each involving all robots in the team. The nominal vector field is a GNN trained on teams of size N ∈ {4, 5, 6}, and evaluation includes 50 manipulation trials for each N ∈ {4, 5, 6, 7, 8}, with N = 7, 8 representing team sizes unseen during training.

Results, summarized in Table 1, show that the nominal (unguided) policy fails in 32 out of 250 trials across all team sizes. The guided policy, which applies SE guidance during the generative process, successfully constructs a traversable bridge in all 250 trials, including the 100 trials with team sizes N = 7 and N = 8 that were never seen during training. This demonstrates that SE guidance can recover failures of the nominal policy while preserving successful collaboration at team sizes beyond those used during training. The emergent coarse‑to‑fine routing pattern revealed in the supplementary analysis further confirms that the semantic‑generic partition learned by the GNN aligns with the diffusion process's denoising order.

Application 2: Multi-Object Scene Generation with Affordance Requirements

The second application evaluates PE guidance on multi‑object scene generation. A useful scene must satisfy not only geometric validity—collision avoidance and workspace containment—but also how individual objects are intended to be accessed or used. Each object has its own affordance requirements, and whether those requirements are satisfied can depend on the positions of neighboring objects.

Task 2 generates a tabletop arrangement of N objects. For object i, its pose is g_Oi = (R_i, p_i) ∈ SE(2), and the decoder maps the joint generative state to the set of object poses. The condition χ specifies the selected objects from a library, their geometry, the tabletop, and their affordance requirements. For each object, one or more nearby regions are designated as affordance regions that should be unobstructed when the object is used. The private violation function for object i is defined as V_i^PE(z|χ) = ½∑_k max{r_ik(z|χ) + δ^PE, 0}², where r_ik ≤ 0 means the k‑th geometric requirement is satisfied (such as object separation, unobstructed affordance regions, and containment within the tabletop), and δ^PE > 0 is a buffer.

The nominal vector field is trained on scenes with N ∈ {4, 5, 6}, and evaluation includes 50 cases for each N ∈ {4, 5, 6, 7, 8}. Three conditions are compared: (1) Natural Nominal, which evaluates scenes generated under default affordance requirements; (2) Private Nominal, which evaluates the same nominal scenes after the desired access or use of an object changes; and (3) Guided, which starts from the same initial samples and incorporates PE guidance based on the changed requirements.

Table 2 reports success rates across all conditions. Under the default requirements (N = 4, 5, 6), the Natural Nominal achieves 88–100% success depending on team size. When the desired affordance requirements are changed (Private Nominal), success rates drop significantly—down to 72% at N = 5 and as low as 16% at N = 8, reflecting the increased difficulty of satisfying modified access requirements with an unguided generator. The Guided condition, by contrast, satisfies all affordance requirements in all 250 cases across all team sizes, including the 100 cases with team sizes N = 7 and N = 8 unseen during training. A visual example in Figure 3 shows an N = 7 scene: the nominal generator produces a scene satisfying a right‑handed laptop use requirement, and PE guidance adapts the scene to satisfy a left‑handed use requirement without any retraining of the generative model. This adaptability highlights the framework's ability to accommodate new constraints at generation time.

Broader Impact and Future Directions

DeGG-Flow opens several avenues for future research. The finite‑horizon guarantees established in this work assume continuous‑time dynamics; extending these guarantees to discrete‑time implementation is a natural next step. Large‑scale multi‑agent systems in real‑world settings, where communication delays and partial observability are present, also warrant investigation. Additionally, the current framework assumes that the nominal vector field is either pretrained or learned offline; integrating the guidance mechanism with online policy adaptation could further enhance practical applicability.

More broadly, DeGG-Flow demonstrates that a pretrained generative model can accommodate hard constraints unseen during training, without the need to collect new data or retrain the model. This capability is particularly valuable in domains where requirements evolve after deployment—such as robotics operating in changing environments, or generative design tools used by engineers with shifting specifications. By providing formal guarantees on coupled multi‑agent requirements through agent‑wise guidance, DeGG-Flow brings the field closer to generative models that are not only expressive but also trustworthy in safety‑critical and constraint‑heavy applications.