Designing RNA sequences that reliably fold into specific secondary structures is essential for understanding their biological functions but remains a challenging computational problem. RNA, or ribonucleic acid, stores genetic information and performs catalytic roles within cells. Each RNA molecule's function depends on its three-dimensional shape, which emerges from its secondary structure—a pattern of base-pairing between nucleotides. Determining the nucleotide sequence that produces a desired folding pattern, known as RNA inverse design, inverse folding, or RNA design, has applications in synthetic biology, gene regulation studies, and the creation of RNA-based therapeutics. Traditional computational approaches rely on energy minimization, stochastic sampling, or gradient-based optimization. These methods often struggle with large sequence spaces, rugged fitness landscapes, and the need to balance local base-pairing constraints with global structural correctness.
CocoRNA introduces a cooperative multi-agent reinforcement learning framework that reframes RNA inverse design as a sequential decision problem. Instead of treating the full sequence as a single decision, the method decomposes the task into smaller sub-problems, each addressed by an individual agent. Multiple agents collaborate to build the sequence nucleotide by nucleotide or in structured blocks. This decomposition reduces the effective search space each agent must navigate, which in turn improves exploration and makes the learning problem more tractable. During training, a centralized critic observes the complete RNA structure and provides global feedback to the agents. This critic does not participate in sequence construction directly but supplies a learning signal that aligns individual agent actions with the overall goal of achieving the target secondary structure. The decentralized agents act independently during execution, but their training leverages global information, creating a bridge between local decision-making and structural outcomes.
The architecture relies on reinforcement learning terminology. An agent selects an action—adding a specific nucleotide to the growing sequence—and receives a reward based on how well the partial structure progresses toward the target. The centralized critic evaluates the final folded structure against the desired secondary structure and computes a global reward signal. This signal is then backpropagated through the agents, adjusting their policy parameters. Because the critic has access to the full structure, it can penalize actions that seem locally reasonable but lead to global misfolding, and it can reward actions that contribute to correct base-pairing patterns even if their benefits manifest later in the sequence. This credit assignment problem is central to multi-agent RL, and CocoRNA addresses it through the centralized critic architecture.
Exploration is another focus. In standard RNA design, a greedy approach that always selects the nucleotide with the lowest free energy may become trapped in local optima. CocoRNA's multi-agent setup introduces stochasticity and diversity among agents. Different agents may propose different nucleotides at the same position, and the collaborative mechanism averages or selects among these proposals based on the learned policy. This diversity helps the search traverse broader regions of the sequence space and avoids premature convergence to suboptimal designs. The agents also communicate through the shared reward signal, indirectly coordinating their choices without explicit message passing.
The training procedure proceeds in phases. Initially, agents receive random sequences and learn to produce structures that satisfy basic base-pairing constraints. As training progresses, the target secondary structure becomes more specific, and the critic's expectations tighten. This curriculum-like schedule shapes the agents' policies toward high-fidelity designs. The authors report that after training, the learned policies generalize to unseen secondary structures without further fine-tuning. This generalization is a key advantage, as it means a single set of trained agents can design sequences for a wide variety of target structures, rather than requiring a new model for each new design task.
Experimentation uses the Rfam dataset, a collection of non-coding RNA families with curated secondary structures. The benchmark compares CocoRNA against established RNA design methods such as RNAinverse, INFO-RNA, and others. According to the reported results, CocoRNA achieves a higher success rate—defined as the proportion of designed sequences that fold into the target structure within a specified distance threshold—across a range of structure complexities. Additionally, the design speed, measured in computational time per sequence, is notably faster than competing methods. These improvements stem from the reduced per-agent complexity and the guided optimization provided by the centralized critic.
Beyond the Rfam benchmark, the authors conduct further experiments on other biological sequence design tasks. These include designing transcription factor binding sites, optimizing ribozyme sequences, and generating synthetic promoters. In each case, CocoRNA demonstrates comparable or superior performance to task-specific baselines. The authors attribute this broad effectiveness to the general nature of the reinforcement learning framework: the same agent architecture and training procedure can be adapted to different design objectives by simply changing the reward function and target structure specification. This adaptability suggests that CocoRNA could serve as a foundation for a new class of computational design tools across molecular biology.
One design choice worth noting is the granularity of the sub-problems assigned to agents. The paper describes two variants: one where agents add nucleotides sequentially, and another where agents handle blocks of contiguous nucleotides. The sequential variant offers finer-grained control and is better suited for structures with intricate long-range interactions. The block-wise variant reduces the number of decision steps and can be more efficient for simpler or more modular structures. The choice between variants depends on the target secondary structure's complexity and the available computational budget. Both variants share the same centralized critic architecture, so the core innovation—the cooperative multi-agent setup with global structural feedback—remains consistent.
Limitations mentioned by the authors include the dependence on accurate structural predictions from the folding model used during evaluation. If the predicted secondary structure deviates significantly from the true in vivo structure, the learned policies may produce sequences that do not function as intended in biological contexts. Additionally, the method's performance on extremely large genomes or highly repetitive sequences has not been extensively studied. These limitations are typical of computational design methods that rely on in silico folding predictions and should be considered when applying CocoRNA to real-world engineering projects.
For a working developer or researcher interested in building on this work, the code release accompanying the paper provides pre-trained agent policies, the centralized critic model, and tools for evaluating designed sequences against target structures. The framework is implemented in a popular deep learning library, and the authors have released scripts for adapting the reward function to custom design objectives. A practical starting point would be to instantiate the agent architecture for a new RNA family, provide a set of target secondary structures, and run the training loop with the centralized critic. The reported generalization ability means that even a moderately sized training set of structures should yield policies that perform well on novel targets.
CocoRNA's contribution lies in framing RNA inverse design as a cooperative multi-agent reinforcement learning problem and demonstrating that this framing yields improved success rates and faster design cycles on standard benchmarks. By decomposing the design task and leveraging a centralized critic for global guidance, the method navigates the rugged fitness landscapes that challenge traditional approaches. The reported generalization to unseen structures further enhances its practical utility, as it reduces the need for task-specific retraining. While the method inherits the typical limitations of in silico structure prediction, its broad applicability to various biological sequence design tasks indicates a promising direction for computational molecular engineering.