Building Org Charts One Role at a Time: How MAGIC Learns the Right Granularity for Multi-Agent Systems
In a multi-agent system, the collaboration topology—the graph of who talks to whom, who delegates to whom, and who reports to whom—matters as much as the individual agents themselves. A task that requires deep research benefits from a group of specialized agents working in parallel; a task that requires a one-line code fix does not. Yet most existing approaches to generating these topologies commit to a single granularity: either every node is an individual agent, or every node is a predefined group. They make the same organizational choice for every subtask in a query, even when different subtasks demand different levels of collaboration.
A team of researchers has proposed a solution they call MAGIC: a framework that selects granularity locally for each functional role, building mixed-granularity agent graphs where some roles are executed by a single agent and others by reusable groups of agents. The paper, submitted on 22 September 2026, demonstrates that this flexibility is not merely a convenience but a measurable advantage, outperforming state-of-the-art baselines across eight benchmarks while reducing inference cost.
The Fixed-Granularity Mismatch
The problem is intuitive but underappreciated. Consider an API-update task: investigating upstream changes and testing compatibility may warrant a dedicated search group and testing group, while implementing the actual patch may need only a single programmer agent. An all-atomic system assembles collaboration node by node, without reusable group priors and with longer construction sequences. An all-group system treats each group as an indivisible unit, imposing unnecessary multi-agent overhead on subtasks that a single agent handles more efficiently.
The researchers formalize this as a "fixed-granularity mismatch." They empirically demonstrate the problem through controlled experiments: on the MMLU-Pro benchmark, a GAA configuration (group for decomposer, atomic for solver and verifier) outperforms both the all-atomic and all-group endpoints by 4.5 accuracy points. On TAT-QA, the preferred configuration flips to AAG. The grouped role that works best differs by task, and mixed assignments outperform both fixed-granularity endpoints while consuming fewer tokens than the all-group configuration.
The Organization Space: A Mixed-Granularity Formulation
MAGIC frames topology generation as constructing a mixed-granularity agent graph from a fixed role library. Each role in the library admits one or both realizations: an atomic agent that executes directly, or a group that is itself a fixed internal agent graph with the same external interface. A task-specific organization is then a collection of units, each assigned a role and a granularity choice, with directed dependencies specifying information flow between them.
Crucially, the all-atomic and all-group organizations are special cases within this mixed space. The formulation does not force a choice between granularity levels; it allows the policy to select the appropriate level for each role independently. The construction process must also respect structural constraints: roles must have admissible realizations, interfaces must be compatible, and the expanded graph must be acyclic.
Incremental Construction with an Edge-Aware GRU
MAGIC builds the graph one unit at a time. At each construction step, a task-conditioned policy selects three things simultaneously: a functional role, whether that role is realized as an atomic agent or a group, and which existing units the new unit depends on.
The policy's encoder observes the public query and the partially constructed graph. A frozen text encoder embeds the query and role realizations. Node features combine realization semantics with granularity choice, construction position, and degree embeddings. An Edge-aware GRU processes units in construction order, combining each unit's features with the mean representation of its actual predecessors and the query embedding. The recurrent state captures construction history, while the predecessor aggregation captures graph dependencies.
Task-conditioned attention pools the node states, and a projection combines the pooled representation with the query and construction progress to form a decision context. From this context, three conditioned heads independently parameterize the action: a categorical distribution over roles (including a stop action), a categorical distribution over the chosen role's realization (atomic or group), and independent Bernoulli decisions selecting admissible predecessors from existing units.
Dense-Reward Shaping Without Distorting the Objective
Training such a construction policy presents a challenge. The terminal task reward—the final answer score—is available only after the entire graph has been executed, which is a late signal for decisions made at the beginning of construction. Potential-based reward shaping addresses this by providing intermediate feedback from partial-graph evaluations while preserving the cumulative terminal reward.
The mechanism works as follows. A fixed probe set of other training queries is shared across all trajectories. Each time a unit is added, the partial graph before and after the addition is evaluated on the probe set, and the paired utility difference contributes to a graph potential. This potential is augmented with terms penalizing structural complexity (extra nodes and edges beyond allowances) and role repetition (multiple units assigned the same role). The shaped reward at each step is the task score at termination plus the change in potential, ensuring that the total return across an episode equals the final task score. This return-preservation property is proven formally and means that shaping redistributes learning signals without adding a lasting bias to the objective.
The practical effect is significant: without dense rewards, only 32.81% of intermediate construction actions receive non-zero advantage signals on MMLU-Pro. With dense rewards, this rises to 83.13%. The policy learns not just from final outcomes but from the incremental value of each organizational decision.
Direct Policy Optimization, No Demonstration Corpus Needed
Unlike Search-then-SFT approaches that first collect successful construction trajectories through search and verification and then fine-tune a generator to imitate them, MAGIC optimizes the construction policy directly from trajectories sampled under the current policy. There is no pre-collected corpus of successful graphs. The policy generates proposals, they are executed, and the returns are used to update the policy through normalized advantage estimation with KL and entropy regularization against a frozen reference policy from the start of training.
This direct optimization route has two advantages. First, it avoids the bottleneck of verifying all possible configurations, which grows exponentially with the number of dual-granularity roles. Second, it allows the policy to discover organizational patterns that might not be captured in any pre-collected demonstration set. The paper compares both routes under matched API cost budgets and shows that direct policy optimization achieves comparable or better performance with lower inference cost.
Results Across Eight Benchmarks
MAGIC is evaluated across eight benchmarks spanning question answering (MMLU-Pro, TAT-QA), mathematical reasoning (GSM8K), code generation (HumanEval), and other task types. The system outperforms state-of-the-art topology generators, including both atomic-agent methods that predict edges over specified agents and group-level methods that connect predefined collaborative groups.
The performance-cost trade-off is particularly striking. Because MAGIC can use a single atomic agent for subtasks that require no collaboration, the constructed graphs tend to be shorter and cheaper to execute than all-atomic alternatives that must assemble collaboration node by node. At the same time, the ability to instantiate groups when needed means MAGIC captures reusable collaboration priors that all-atomic systems lack. The result is better accuracy at lower cost, not just better accuracy at equal cost.
What This Means for Multi-Agent Design
The paper's contributions extend beyond a specific system. The mixed-granularity perspective reframes how researchers think about multi-agent organization design. Rather than choosing between fine-grained and coarse-grained topologies as competing paradigms, the paper argues that the right answer depends on the specific role and task, and that a single policy should be able to make that judgment locally.
The dense-reward mechanism offers a general lesson for combinatorial construction problems: when the terminal reward is sparse and late, providing intermediate signals from partial evaluations can dramatically improve learning efficiency—provided those signals are shaped to preserve the terminal objective rather than introducing new optimization targets.
Limitations and Open Directions
Several constraints bound the findings. The role library is fixed and predefined; the system can select among roles it has been given but cannot invent new ones. The group internal topology is also fixed, meaning the system chooses whether to instantiate a group but not how that group should be internally organized. The probe set is shared across all trajectories and construction steps, which simplifies implementation but may not capture every query's specific utility landscape. The paper acknowledges that role and edge choices further enlarge the construction space beyond the dual-granularity roles analyzed in the coverage bounds.
The analysis also assumes that the frozen LLM executors produce consistent scores across evaluations, which may not hold perfectly in practice due to the stochastic nature of LLM outputs. The training process requires multiple trajectory samples per query with partial-graph evaluations on shared probes, which imposes computational overhead during training even if inference is efficient.
Nevertheless, the empirical evidence across eight benchmarks, the controlled granularity experiments showing task-dependent preferences, the analytical coverage bounds, and the dense-reward ablation studies together make a compelling case. In multi-agent systems, as in organizations, the most effective structure is not the one-size-fits-all approach but the one that adapts its granularity to the demands of each subtask.
Read the paper on arXiv