Relations compose through the entity they share—Alice is Bob's mother and Bob is Carol's father, so Alice is Carol's grandmother—and the order matters, since a father's mother is not a mother's father; verifying a composed relation means closing the loop. This paper demonstrates that if each entity's coordinate frame may be rotated independently, a cascade of structural consequences follows. In the pure-gauge sector, relations that share no cycle cannot interact at all. The only gauge-invariant score of a triangle is a class function of its holonomy H = Tij Tjk Tki—a Wilson loop. And the only equivariant update of an edge is U = Ψ(H) Tij with Ψ conjugation-equivariant. Edge attention by Wilson loops is therefore not one design among many but the unique form the symmetry permits.
Three research communities hold pieces of this form without recognizing the underlying unity. The commuting-operator embeddings of knowledge graphs constitute its abelian, commuting sector. AlphaFold's triangle update and the Edge Transformer exhibit the shape of an edge updated through the third vertex, but without the symmetry constraints that make the form necessary. Lattice QCD builds the same construction equivariantly on lattices. This paper instantiates the equivariant sub-family on general relational graphs as holonomy attention, a mechanism that requires no training and relies on composition alone.
Why Relations Close Loops When Frames Float Independent
Standard attention mechanisms compute a score between two entities and use that score to update one or both. The score typically depends on the relative orientation of the two entities' feature vectors. This paper argues that when each entity carries its own frame—its own local coordinate system that can rotate independently—the space of permissible scores and updates shrinks dramatically. The reason is that any score or update must be equivariant under independent frame rotations. If the score changes under a rotation that does not change the underlying relation, the mechanism is not truly capturing the relation.
The paper begins by formalizing entities as points in a space each equipped with a frame. A relation between two entities is represented by a transformation Tij that maps from entity j's frame to entity i's frame. When frames float independently, the system has per-entity frame freedom. The paper shows that this freedom alone implies three things, starting with the simplest: relations that do not share a cycle cannot influence one another in the pure-gauge sector. If two relations involve disjoint sets of entities, there is no loop to close, no shared entity through which a signal can propagate. Hence their scores are decoupled.
This decoupling has a concrete consequence for graph-structured data. Consider a graph where edges are labeled by transformations. If an edge exists between i and j, and another between k and l, and the vertex sets {i, j} and {k, l} are disjoint, then no gauge-invariant pathway connects the two edges. Any attempt to pass information from one to the other would require breaking the gauge symmetry, which the framework does not allow. The result is a modular decomposition: the graph falls into independent components that can be analyzed separately.
The Pure-Gauge Consequence: Cycles Are the Only Paths to Interaction
Building on the decoupling of disjoint relations, the paper examines what happens when relations do share vertices. The minimal shared structure is a triangle: three entities i, j, k with edges Tij, Tjk, Tki. Because each entity's frame can rotate independently, the three transformations are not fixed relative to one another. Their product Tij Tjk Tki however is invariant under simultaneous rotation of all three frames. This product is the holonomy H, also known as a Wilson loop in gauge-theoretic language.
The paper proves that H is the only gauge-invariant quantity that can be formed from a single triangle. Any score assigned to the triangle must be a class function of H—meaning it depends on H only through invariants such as its trace, determinant, or eigenvalues. This is a strong restriction: it means that no matter how the individual frames are reoriented, the triangle's score remains unchanged. This invariance is what makes the Wilson loop the natural object for measuring composed relations.
The result generalizes to larger cycles. Any closed loop of transformations yields a holonomy that is gauge-invariant. Open paths, by contrast, are not invariant; their scores change under frame rotations. Hence, in this framework, only closed loops—cycles—provide stable, frame-independent information. This explains why the paper's abstract states that relations sharing no cycle cannot interact.
Holonomy Is the Only Gauge-Invariant Triangle Score
With the holonomy H established as the fundamental gauge-invariant object, the paper turns to the question of how to score a triangle. Given H = Tij Tjk Tki, what score should be assigned? The answer is that the score must be a class function of H. In practice, this means extracting invariants from H.
One natural invariant is the trace tr(H), which sums the diagonal entries and is invariant under conjugation. Another is the determinant det(H), which measures whether the holonomy preserves volume. The eigenvalues of H carry information about the holonomy's rotational nature—whether it is a pure rotation, a rotation with scaling, or something else. Any triangle score that respects the gauge symmetry must be expressible in terms of these invariants.
The paper notes that commuting-operator embeddings used in knowledge graphs fall into the abelian sector, where all transformations commute. In that sector, H = Tij Tjk Tki simplifies because the order of multiplication does not matter. The holonomy reduces to the product of the three transformations, and its invariants are limited. This is why commuting scores are capped at 0.76 in the experiments reported—the second selection rule mentioned in the abstract arises from the restricted expressivity of the abelian sector.
Non-abelian theories, where transformations do not commute, allow richer holonomies. The non-commutativity means H can encode relative rotation angles that are not accessible in the commuting case. This is part of why holonomy attention can achieve an AUC of 0.99 for flagging corrupted triangles, while commuting scores stagnate at 0.76.
The Unique Equivariant Edge Update: Wilson Loop in Disguise
Beyond scoring triangles, the paper asks what happens to an individual edge when the triangle's information is incorporated. Given an edge Tij and the holonomy H of the triangle containing i, j, k, how should Tij be updated? The paper proves that the only equivariant update has the form U = Ψ(H) Tij, where Ψ is a conjugation-equivariant map.
Conjugation equivariance means that if the holonomy is conjugated by some transformation—representing a change of frame—the updated edge transforms in the same way. In other words, if H → G H G−1 for some frame change G, then Ψ(H) → G Ψ(H) G−1, and the full updated edge U transforms as G U G−1. This ensures that the edge update is consistent with the entity frame freedom.
The form U = Ψ(H) Tij is remarkably restrictive. It says that an edge can only be updated by first computing the holonomy of a triangle involving that edge, then applying a conjugation-equivariant function to that holonomy, and finally multiplying by the original edge transformation. No other construction satisfies the symmetry requirement.
This result explains the connection to existing models. AlphaFold's triangle update uses a similar shape—an edge updated through the third vertex—but without the gauge constraints that force the Wilson loop form. The Edge Transformer likewise updates an edge by attending to a third entity, but again without the symmetry guarantees. Lattice QCD builds the same construction equivariantly on lattices, where the Wilson loop is a first-class object. Holonomy attention ties these together by showing that the Wilson loop form is not a design choice but a symmetry requirement.
From Three Communities to One Framework: Holonomy Attention
The paper's central contribution is to synthesize the three strands—commuting-operator embeddings, AlphaFold-style triangle updates, and lattice QCD—into a single framework called holonomy attention. The framework operates on general relational graphs, where each edge carries a transformation and each entity carries a frame. The attention mechanism proceeds by computing holonomies over triangles and using them to update edges via the form U = Ψ(H) Tij.
Because the mechanism is derived from symmetry rather than trained from data, it requires no labeled training examples. The parameters of Ψ can be initialized once and then left fixed, or they can be set to the identity, in which case the update simply multiplies the edge by the holonomy. The paper evaluates the no-training setting and finds that composition alone is sufficient to infer unobserved S4 relations on sparse graphs at 0.88 accuracy, with up to a fifth of edges corrupted. This places holonomy attention at or slightly below specialist synchronization algorithms, which are designed specifically for the same task.
Every commutative architecture, by contrast, is capped at 0.083 accuracy on the same task. The gap is stark: 0.88 versus 0.083. The reason is that commutative architectures cannot capture the non-abelian holonomy that encodes relative rotation order. They are limited to the abelian sector where all transformations commute, and the second selection rule caps their scores at 0.76 for corrupted triangle detection.
Holonomy attention's corrupted triangle detection achieves AUC 0.99, far above the commuting scores' 0.76. This improvement comes from the holonomy's ability to encode non-commutative structure. When a triangle is corrupted—say, one of its edges is replaced by a random or incorrect transformation—the holonomy H changes in a way that is detectable through its invariants. The AUC 0.99 reflects the high distinguishability of holonomy distributions between clean and corrupted triangles.
Additionally, holonomy attention separates clusters without labels. This unsupervised clustering ability stems from the fact that holonomies produce frame-independent signatures for each connected component of the graph. Components with different relational structures yield different holonomy invariants, allowing a simple clustering algorithm to group vertices by their holonomy signatures. No label information is needed.
No-Training Relation Inference on Sparse Graphs
The experimental evaluation focuses on inferring unobserved S4 relations—relations from the symmetric group on four elements—on sparse graphs. The task is to recover missing or hidden relations from a partially observed graph. The graph is sparse, meaning most entity pairs have no direct edge. A fifth of the edges are corrupted, meaning some observed edges carry incorrect transformations.
Holonomy attention achieves 0.88 accuracy in this setting. The baseline specialist synchronization algorithms, which are designed for the same problem, perform at a comparable level, and the paper notes that holonomy attention is "at or slightly below" these specialists. This is notable because holonomy attention has no training component; it relies entirely on the composition of observed transformations.
Commutative architectures, which enforce that all edge transformations commute, max out at 0.083 accuracy. This abysmal performance underscores the importance of non-commutative structure for the task. The second selection rule that caps commuting scores at 0.76 for triangle corruption detection further limits their usefulness.
The paper also reports AUC results for corrupted triangle detection. Holonomy attention reaches AUC 0.99, while commuting scores are capped at 0.76. The 0.76 ceiling arises because the abelian sector cannot distinguish all corruption patterns; some corrupted triangles produce holonomies that are indistinguishable from clean ones when commutativity is enforced.
Cluster Separation Without Labels
One of the more striking results is that holonomy attention separates clusters without any label information. Given a relational graph, the algorithm computes holonomy invariants for each triangle and aggregates them at the vertex level. Vertices that participate in similar relational patterns end up with similar holonomy signatures, naturally grouping them into clusters.
This capability has practical implications. In a knowledge graph, for instance, vertices might represent entities and edges might represent typed relations. Even without predefined types, the holonomy signatures reveal structural clusters—groups of entities that play similar roles in the relation network. A developer could use this to discover latent categories or to inform subsequent supervised learning.
The clustering works because the holonomy H = Tij Tjk Tki is a frame-independent summary of the triangle's relational structure. Two triangles that are related by a global frame rotation have the same H, and two triangles with the same relational composition (up to frame rotation) also share H. This invariance is what enables clustering without labels.
Implications for Relational Reasoning
The paper's main theoretical result is that edge attention by Wilson loops is the unique form permitted by the symmetry of per-entity frame freedom. This means that any model that updates edges by attending to other entities—and many do—is either implicitly or explicitly approximating the holonomy attention form. AlphaFold's triangle update and the Edge Transformer both have the right shape, but they operate without the gauge constraints that make the form necessary. Lattice QCD already respects the constraints, as it is built on lattices where gauge symmetry is fundamental.
For practitioners, the key takeaway is that if a model handles relations between entities that each have their own frame of reference, and if the model's updates and scores are to be independent of those frames, then the Wilson loop construction is not optional. It is forced by the symmetry. Models that ignore this—by, for instance, simply summing edge features or using commuting embeddings—are limited to the abelian sector and will fail to capture non-commutative relational structure.
The no-training results show that composition alone, when framed correctly, can yield substantial performance. Achieving 0.88 accuracy on S4 relation inference without any learned parameters is a strong result, especially compared to the 0.083 ceiling of commutative architectures. The high AUC 0.99 for corrupted triangle detection suggests that holonomy-based scoring could be used as a diagnostic or as a component in larger systems that need to detect inconsistent or corrupted relations.
The cluster separation without labels opens a path toward unsupervised discovery of relational structure. In settings where labeling is expensive or unavailable, holonomy attention provides a way to group entities by their relational role. This could be combined with downstream tasks such as classification or regression.
Limitations and Trade-offs
The paper acknowledges several limitations. The no-training setting, while elegant, does not achieve the very top performance; specialist synchronization algorithms still edge it out slightly. Adding learned parameters to Ψ could close the gap, but the paper's point is that the bare form already captures a large fraction of the achievable performance. The trade-off is between theoretical purity (no training, symmetry-derived form) and raw accuracy.
Another limitation is the reliance on triangle holonomies. Larger cycles could in principle be considered, but the paper focuses on the minimal structure—a single triangle—because it is the building block for all larger loops. Extending the framework to longer cycles would increase computational cost without a clear gain in expressive power for the tasks considered.
The task of inferring S4 relations on sparse graphs is specific; the paper does not claim that holonomy attention is optimal for all relation-inference tasks. Other relation types, larger graphs, or different corruption patterns could yield different results. The 0.88 accuracy and AUC 0.99 should be understood as specific to the experimental setup described.
Finally, the framework requires that each entity carry a frame. In applications where entities naturally have associated coordinate systems—such as protein structures in AlphaFold, or where frames can be defined from relative poses—this is not a burden. But in settings where entities are frame-less, additional work is needed to assign or learn frames before the framework can be applied.
What It Means in Practice
For a working developer, holonomy attention provides a new building block for relational models. If your entities have frames—and many do, even if implicitly, through their feature representations—then the Wilson loop form is the symmetry-allowed way to score triangles and update edges. You can implement it by computing the product Tij Tjk Tki for every triangle, extracting invariants such as tr(H) or det(H), and using those to modulate edge updates via a conjugation-equivariant map.
The fact that commuting architectures are capped at 0.083 on the S4 relation inference task suggests that simply adding more layers or parameters will not overcome the abelian limitation. Non-commutative structure is essential. Developers working on knowledge graph embeddings, protein structure prediction, or any domain with directed or ordered relations should consider whether their models respect the gauge symmetry or inadvertently collapse it.
The unsupervised cluster separation capability means that holonomy attention can be used as a preliminary step even before labels are available. Compute holonomy signatures, run a clustering algorithm such as k-means on the resulting vertex embeddings, and you obtain groups that reflect the underlying relational structure. This can inform data exploration, feature engineering, or the decision to introduce supervised learning.
Finally, the synthesis of three previously separate strands—commuting embeddings, AlphaFold-style updates, lattice QCD constructions—into a single framework suggests that there is a common mathematical core underneath many existing models. Recognizing this core can guide model design: if a model's updates and scores can be written as a class function of a holonomy times the edge transformation, it is gauge-equivariant by construction. If not, it may be violating the symmetry and paying a price in limited expressivity.
Holonomy attention is not a panacea, but it is the unique form the symmetry permits. Its results—0.88 accuracy without training, AUC 0.99 for corrupted triangle detection, cluster separation without labels—show that the symmetry constraint is not unduly restrictive; rather, it carves out a large and performance-rich subspace of the space of possible models. For developers building relational models where entities have independent frames, holonomy attention is the natural place to start.