Molecular learning models are strongly shaped by their underlying representations, yet standard sequential and graph formalisms struggle to explicitly encode higher-order topology such as ring systems and recurring motifs. Existing higher-order representations can capture these structures directly, but they are often computationally demanding and difficult to decode into valid molecules. This paper introduces Higher-order Grammar Representation (HGR), a principled, topology-aware framework that lifts molecules to combinatorial complexes and parses each complex into a compact sequence of production rules under a context-free higher-order grammar. By serialising higher-order topology into rule sequences, HGR makes these structures directly compatible with standard sequence models, avoiding the computational overhead of explicit higher-order encodings while preserving topological expressiveness. To reduce benchmark bias towards simple ring systems, the authors construct RingDiv, a ring-enriched benchmark containing 1.18 million molecules, including the curated RingDiv300k subset, and introduce the ring diversity index (RDI) to quantify ring-system coverage. HGR-based models achieve 100% validity by construction with leading distributional alignment, ranking first in FCD on all five generation benchmarks. In representation learning, HGR-FM achieves the highest mean AUC across seven MoleculeNet benchmarks under both transfer protocols, improving on the strongest baseline by 8.3 and 3.3 AUC points under probing and full fine-tuning, respectively.
Limitations of sequential and graph representations
Molecular representation learning has been dominated by SMILES strings and graph neural networks. SMILES encodes atoms and bonds as a linear sequence, which makes it easy to feed into transformer-based models but loses explicit information about ring closures, bond stereochemistry, and higher-order substructures such as fused or bridged ring systems. Graph neural networks capture pairwise relationships between atoms but require message-passing iterations that scale poorly with molecule size and can blur topological details beyond a certain hop distance. Both formalisms need auxiliary tricks — such as ring-index tokens or edge-type embeddings — to even acknowledge the presence of cycles, and even then, decoding a generated sequence or adjacency matrix into a valid, syntactically correct molecule often requires post-processing heuristics. The authors argue that these workarounds introduce bias and inefficiency, and that a representation designed around higher-order topology from the ground up would be both more expressive and more practical.
Higher-order Grammar Representation (HGR)
HGR lifts each molecule into a combinatorial complex, a mathematical structure that records not only pairwise atom-bond relationships but also higher-order interactions such as rings of three or more atoms, cyclic orderings, and recurring motif instances. Each combinatorial complex is then parsed into a compact sequence of production rules under a context-free higher-order grammar. These rules describe how to build the molecule by specifying, in order, which substructures to introduce, how they connect to already-generated parts, and when a ring-closing bond should be emitted. Because the rules are context-free, they can be processed by standard sequence models — transformers, RNNs, or any model that accepts a token feed — without requiring custom message-passing or differentiable graph operations. The serialised rule sequence preserves the complete higher-order topology of the original molecule while remaining a plain string that any sequence model can ingest.
RingDiv: a ring-enriched benchmark
To evaluate how well models handle ring systems of varying complexity, the authors construct RingDiv, a benchmark containing 1.18 million molecules curated from public databases and de novo generation pipelines. The subset RingDiv300k focuses on molecules with three or more rings, covering a wide range of ring sizes, fusion patterns, and heteroatom substitutions. To quantify ring-system coverage, they introduce the ring diversity index (RDI), which counts the number of distinct ring-types (size, atom composition, substitution pattern) present in a dataset and normalises by the total number of molecules. RDI allows researchers to report a single number indicating whether a benchmark stresses simple monocycles or exercises the model with complex polycyclic architectures.
Molecular generation with HGR
In generation experiments, HGR-based models produce molecule descriptions exclusively through rule sequences. Because the grammar enforces valid production-step combinations, every output is valid by construction — no post-hoc filtering or repair steps are needed. The models are trained to maximise likelihood of the rule sequence, which indirectly teaches them the distribution of higher-order substructures. Compared to baseline models that operate on SMILES or graph latent codes, HGR models achieve 100% validity while matching or exceeding baseline scores on the Fréchet ChemNet Distance (FCD) metric across all five generation benchmarks tested. The authors attribute this to the grammar’s ability to enforce chemically plausible ring closures and motif combinations without requiring a separate validity checker.
Representation learning with HGR-FM
For transfer learning, the authors train HGR-FM, a foundation model based on the transformer architecture, on the HGR rule sequences of the MoleculeNet benchmark set. Two transfer protocols are evaluated: probing, where frozen model representations are used to train simple linear classifiers on downstream tasks, and full fine-tuning, where all model parameters are updated on task-specific data. HGR-FM achieves the highest mean AUC across seven MoleculeNet benchmarks under both protocols. Under probing, it improves on the strongest baseline by 8.3 AUC points; under full fine-tuning, the improvement is 3.3 AUC points. The gains are most pronounced on tasks that require reasoning about ring systems and multi-center interactions, such as HIV reverse transcriptase inhibition and estrogen receptor binding, suggesting that the higher-order topology captured by HGR is particularly useful for these decision boundaries.
Broader impact and code availability
The authors release their HGR implementation, the RingDiv benchmark, and the RDI computation code as part of the paper. The framework is designed to be dropped into existing sequence-model pipelines with minimal changes: the input is a rule sequence, the output is a distribution over the next production rule, and the decoding process assembles the molecule step by step. They anticipate that HGR will be useful not only for generation and representation learning but also for molecular optimization, where the explicit rule sequence can be mutated or edited to explore chemical space in a controlled way. The authors also note that the same principle — lifting a domain‑specific structure into a grammar‑compatible representation — could be adapted to other fields such as nucleic-acid design or materials science.
Limitations and future directions
The current grammar is tuned to organic small molecules and may not capture the full topology of proteins, nucleic acids, or extended solid-state materials without extension. The RingDiv benchmark, while large, still samples from chemical space that is biased toward drug-like molecules; broader coverage would require inclusion of organometallics, polymers, and supramolecular architectures. The authors also observe that the validity-by-construction property holds for the specific grammar used in experiments; alternative grammars may produce a different proportion of valid outputs. Future work may explore adaptive grammars that grow with the molecule being generated, or differentiable extensions that allow the rule sequence to be refined through gradient-based editing.
What this means for cheminformatics practitioners
If you are building or fine-tuning generative models for molecules, HGR offers a drop-in representation that eliminates the validity-filtering step and explicitly captures ring systems and recurring motifs. The RingDiv benchmark and RDI metric provide a standardised way to report and compare ring-system coverage across datasets and model families. The authors' code release allows groups with moderate compute to experiment with HGR-FM on MoleculeNet or their own internal assay data, and the reported AUC improvements suggest that higher-order topology is a worthwhile inductive bias for transfer learning tasks. For generation, the guarantee of 100% validity by construction removes a major engineering overhead and enables more reliable downstream screening or optimization loops.