Can You Build a Foundation Model for the Mouse Brain?
Large-scale neural recording now captures hundreds of thousands of neurons across dozens of brain regions in hundreds of animals. The data is there. The question is whether this scale can be exploited to learn general-purpose neural representations that transfer across animals, tasks, and brain regions, the way language models learn representations that transfer across downstream NLP tasks.
The analogy is tempting but misleading. Language is discrete, densely annotated, and largely shared across speakers. Neural data is continuous, sparsely annotated, variable across animals, and recorded under heterogeneous conditions. A representation learned on one mouse's visual cortex may say nothing about another mouse's motor cortex, not because the underlying computation differs but because the recording geometry, electrode placement, and behavioral state differ. Whether pretraining can overcome this variability and extract shared neural structure is an open question that the field has not had the tools to answer systematically.
BrainWideBench, from a multi-institution collaboration spanning Penn, Mila, Columbia, Stanford, Allen Institute, and many others within the International Brain Laboratory, provides those tools. It is a benchmark for evaluating across-animal transfer on multi-region neural recordings, built on the IBL Brainwide Map dataset of 600+ hours of single-spike recordings spanning 276 brain regions from 139 mice. The benchmark organizes evaluation around three complementary task suites that probe distinct properties of learned representations, and it standardizes pretraining-evaluation splits, quality control, and metrics so that methods can be compared on equal footing.
The Dataset: Brainwide Map at Scale
The IBL Brainwide Map is one of the largest coordinated neuroscience datasets in existence. It records neural activity and behavioral measurements from mice performing a sensory-guided decision-making task. High-density electrophysiology captures individual neurons at single-spike resolution across hundreds of brain regions simultaneously, spanning cortex, thalamus, hippocampus, and subcortical areas. The dataset covers 276 anatomical regions, giving it the brainwide coverage that single-lab recordings typically lack.
BrainWideBench curates this dataset with standardized train/test splits and quality-control metadata at the session, probe, and unit level. The pretraining set includes 126 mice. The held-out evaluation set includes 13 animals, selected to be disjoint from pretraining. Quality control is not an afterthought. The benchmark provides session-level, probe-level, and unit-level QC flags so that researchers can study how data quality shapes pretraining and model robustness. This is important because neural recordings are noisy, electrode drift changes which neurons are captured across sessions, and not every recording session produces usable data.
Three Task Suites, Three Views of Representation Quality
The benchmark organizes downstream evaluation into three complementary task families, each probing a different property of what a learned representation captures.
Task Suite 1, Behavior Prediction, asks whether learned representations support decoding of behavioral variables from neural activity. This includes frame-level tasks (decoding wheel speed, paw position, licking rate, whisker motion energy at each time step) and sequence-level tasks (decoding the stimulus shown, the animal's choice, and the trial outcome over entire trial sequences). The evaluation paradigm uses a linear probe on top of frozen pretrained features, so the metric reflects what the representation itself captures about behavior, not what a downstream model can learn to extract.
Task Suite 2, Neural Activity Prediction, asks whether learned representations can predict masked or future neural activity. Given observations of some neurons at some time points, can the model predict the activity of other neurons or the same neurons at future time points? This tests whether the representation captures the dynamical structure of neural population activity, the patterns of co-variation and temporal evolution that define how the brain computes. The metric is fraction of deviance explained under a Poisson model (D-squared), which accounts for the sparsity and non-Gaussian statistics of spike trains.
Task Suite 3, Neuron Identity Prediction, asks whether the model can predict the anatomical brain region of a neuron from its recorded activity. This is not a decoding task in the traditional sense. It tests whether the representation captures biologically meaningful organization, whether neurons in different brain regions have distinct activity signatures that the model can learn to distinguish. The metric is classification accuracy for region labels, evaluated under both within-animal and across-animal transfer settings.
The three suites are complementary because they probe different aspects of representation quality. A representation that decodes behavior well may not predict neural dynamics well, because behavioral decoding can rely on a small number of neurons that are highly correlated with the behavioral variable, while neural prediction requires capturing the full population structure. A representation that predicts neural dynamics well may not identify brain regions well, because dynamics can be shared across regions while identity depends on region-specific patterns. The benchmark is designed so that no single suite tells the whole story.
The Baseline Models
The benchmark evaluates a diverse set of baselines spanning single-session models and large-scale pretraining approaches. Single-session baselines include linear probes, multi-layer perceptrons, convolutional neural networks, gated recurrent units, autoencoders, LFADS (latent factor analysis via dynamical systems), CEBRA, and POYO. Pretrained baselines include POYO+, POSSM, NDT (neural data transformer), NDT2, NEDS, MtM, LOLCAT, NEMO, and NuCLR.
These models represent different architectural choices (transformers vs. state-space models vs. recurrent networks), different pretraining objectives (masked prediction, contrastive learning, next-token prediction), and different scales of pretraining data. The benchmark evaluates them under common QC standards and provides all metadata needed for reproducibility.
What the Results Show
The central finding is that pretraining improves performance over matched single-session baselines, but current methods exhibit heterogeneity in transfer capabilities. Gains depend strongly on the alignment between pretraining objectives and downstream tasks. No single approach performs uniformly well across all three suites, and most methods are designed to address only a subset of them.
This heterogeneity is the paper's most important result. A method that excels at behavior prediction may underperform at neural activity prediction. A method that captures anatomical organization may not decode behavior well. The alignment between what the model was trained to do and what the downstream task requires is the primary determinant of transfer quality.
The benchmark also reveals that zero-shot generalization to unseen animals is harder than finetuning on downstream objectives. Models that perform well when finetuned on a held-out animal's data often perform poorly when asked to generalize without any finetuning. This suggests that current pretraining methods do not learn fully animal-invariant representations, which is the core aspiration of the foundation model analogy.
The evaluation across 13 held-out animals also shows that performance varies substantially across animals, even within the same task suite. Some animals are easier to decode or predict than others, likely reflecting differences in recording quality, brain coverage, and behavioral performance. The standardized splits and QC metadata allow researchers to disentangle these effects, but the heterogeneity itself is a finding: neural data is not as homogeneous as language data, and foundation models for neuroscience will need to handle this variability.
Why This Matters for the Foundation Model Ambition
The paper frames BrainWideBench as a step toward general-purpose models of the mouse brain. The current results show that this goal remains distant. Pretraining helps, but the gains are task-specific and transfer is limited. The field does not yet have a neural foundation model that generalizes across behavior, dynamics, and anatomy simultaneously.
This is not a failure of the benchmark. It is a honest measurement of where the field stands. The benchmark provides the standardized evaluation infrastructure needed to track progress, identify which pretraining objectives produce the most general representations, and determine whether scaling data and model size produces the same kind of predictable improvements seen in language and vision.
The quality control metadata is particularly valuable for this purpose. Neural recordings vary in quality, and a model that appears to transfer well on high-quality sessions may fail on noisy ones. By making QC flags available, BrainWideBench allows researchers to study this question directly, rather than treating data quality as a nuisance variable to be averaged over.
What This Means for Neuroscience and ML Researchers
For machine learning researchers, BrainWideBench provides a new domain for studying transfer learning, pretraining, and scaling. Neural data has properties that distinguish it from language and images: it is sparse, non-Gaussian, high-dimensional, recorded under variable conditions, and organized by biological structure that no one fully understands. A foundation model that works here would demonstrate that the pretraining-transfer paradigm generalizes beyond the domains where it was developed.
For neuroscience researchers, the benchmark provides a standardized way to evaluate whether computational models capture general principles of neural computation rather than overfitting to specific animals, sessions, or tasks. The three task suites probe different aspects of what a model learns, and the across-animal transfer setting tests whether those learned properties generalize.
The benchmark is released with code, standardized splits, and QC metadata. The code is available on GitHub. The combination of a large, curated dataset, standardized evaluation, and diverse baselines makes it possible for new methods to be compared fairly against existing work, which is exactly the infrastructure that drives progress in the foundation model era.
The honest conclusion is that learning representations that jointly generalize across behavior, dynamics, and anatomy remains an open challenge. BrainWideBench does not solve it. It makes it measurable, which is the necessary first step.
Read the paper on arXiv