One of the first steps of a research pipeline is generating ideas. Large Language Models LLMs are increasingly used to automate this step, but most of this work optimizes the quality of individual ideas while overlooking diversity. In this work, we study document-grounded ideation, focusing on increasing the diversity of LLM generated ideas and ensuring they are high quality. Given prior papers, a model must propose a set of ideas that are each high quality and clearly distinct from one another, while also being grounded in the input documents. To optimize for this, we propose training with a set-level reinforcement learning objective. In our setup, in a single generation the model writes K structured idea cards grounded in the source papers, and the set is scored jointly, where each idea's quality is weighted by its distance from the ideas written before it. This teaches the model to steer each new idea away from its predecessors.
The trained 8B model achieves higher creative utility—a measure combining quality and diversity—than Claude Opus 4.8 and GPT-4o on a contamination-free benchmark. In a pilot human study with four experts, the model's sets are judged as promising as Claude Opus 4.8's and contain marginally more unique ideas. When a coding agent runs proof-of-concept experiments on the ideas, the model's success@k is comparable to Claude Opus 4.8's, so we see that the added diversity does not cost any executability.
The quality-diversity tension in automated ideation
Prior work on LLM ideation has largely treated idea generation as a quality-only optimization. Models are prompted to produce the single best answer or the most coherent argument. But in a research pipeline, a single strong idea is insufficient; the early stages benefit from a portfolio of concepts that explore different directions. Without an explicit diversity pressure, models tend to converge on similar formulations, especially when trained on similar data distributions. This paper identifies that gap and proposes a training signal that operates on idea sets rather than individual outputs.
Set-level reinforcement learning with distance-weighted rewards
The core contribution is a reinforcement learning objective that scores an entire set of K ideas jointly. For each idea in the set, the reward consists of two components: a quality term that measures alignment with the target task and grounding in the source documents, and a diversity term that penalizes similarity to ideas already written. Specifically, each idea's quality score is multiplied by a factor that decreases as the idea becomes closer to previously generated ideas. This creates a gradient where the model receives stronger incentive to produce novel ideas as the set fills up. The objective is computed over the whole set, so the model learns to coordinate the K outputs toward a balanced coverage of the problem space.
K structured idea cards and the novelty gradient
In practice, the model generates K structured idea cards per input. Each card contains a concise title, a one-sentence summary, and bullet points specifying the proposed method's core components. The distance metric between ideas is based on textual overlap and conceptual similarity, computed using embedding cosine similarity. Because the reward is computed incrementally—each idea's weight depends on all previously generated ideas in the same generation—the model experiences a novelty gradient that sharpens as more ideas are produced. This design ensures that later ideas are not just different in isolation, but contribute to a diverse overall set.
Benchmark results and comparative results
The trained 8B model achieves higher creative utility than Claude Opus 4.8 and GPT-4o on the contamination-free benchmark. Creative utility combines quality and diversity into a single score, and the model's advantage stems from producing ideas that are both relevant and distinct. In the human study with four experts, the model's idea sets were rated as promising as those from Claude Opus 4.8 and contained marginally more unique ideas. When a coding agent runs proof-of-concept experiments on the generated ideas, the model's success@k is comparable to Claude Opus 4.8's, indicating that the added diversity does not reduce executability.
Inferred limitations and design trade-offs
The distance metric relies on embedding similarity, which may not capture all dimensions of conceptual distinctness, especially for ideas that use different terminology to describe similar approaches. The set-level reward also means that individual idea quality can be influenced by the presence of other ideas in the same generation, which makes isolated idea evaluation harder. The human study was limited to four experts, so broader applicability of the rating remains unclear. Additionally, the training objective assumes that diversity and quality can be jointly optimized through the weighted reward, but the balance between the two terms may need task-specific tuning.
Practical takeaways for developers
A developer interested in adopting this approach can implement the set-level reinforcement learning loop by generating K idea cards per input, computing embedding-based similarity between each new idea and all previously generated ideas in the set, and shaping the reward as quality times a novelty decay factor. The quality signal can be based on reward model scoring or ground-truth task success, depending on the available supervision. The coding agent experiment suggests that the diversity pressure does not hinder downstream usability, so the method can be applied in pipelines where both idea variety and runnable prototypes are desired. For immediate experimentation, the K-card format and distance-weighted reward can be added on top of existing LLM fine-tuning infrastructure without requiring a complete redesign of the generation pipeline.
Read the paper on arXiv