Why single-cell foundation models have underdelivered and what drug discovery needs instead

Most foundation models have produced striking results. In protein biology, AlphaFold predicts protein folding structure with startling accuracy.¹ In weather forecasting, machine learning systems are now beating scientific computing systems that were painstakingly built over the course of decades.^2,3 In language, large models have shown that scale paired with the right structure can produce broad generalisation.⁴

The wide variety in successful foundation models encouraged researchers to target single-cell foundation models.

If a sentence can be represented as a sequence of words, perhaps a cell can be represented as a set of genes. Scientists hypothesised that if you train a transformer across millions of cells, it may learn a useful representation of cellular state. That premise produced ambitious models. scGPT was trained on more than 33 million cells;⁵ Geneformer was pretrained on about 30 million human single-cell transcriptomes;⁶ and scFoundation was pretrained on more than 50 million profiles.⁷

If a sentence can be represented as a sequence of words, perhaps a cell can be represented as a set of genes. 

Their performance on drug discovery-relevant questions, however, has been more uneven than anticipated. In perturbation prediction, a Nature Methods benchmark reported that deep-learning methods, including single-cell foundation models, did not outperform simple linear baselines.⁸

In zero-shot evaluation, another study found that Geneformer and scGPT failed to consistently beat simpler baselines and struggled with batch effects.⁹ The conclusion is not that these models lack value but that their current strengths must be separated from stronger claims about mechanistic generalisation. A model that helps organise cell states, annotate cell types or integrate datasets may still fall short when asked to predict the effect of an unseen perturbation in an unfamiliar biological context.

What foundation models quietly assume

The case for single-cell foundation models rests on the assumption that cellular data contains a learnable structure – like the structure found in language, proteins or weather. Thus, if you collect enough observations and make the architecture expressive enough, the model should learn a representation that generalises beyond the examples it has seen.

That assumption holds when two conditions are met. The first is a ‘compact grammar’: a set of constraints that shrinks the effective search space of a domain. Language has syntax, semantics and recurring patterns of word use. Proteins have physical constraints on stable folding. Weather has fluid dynamics and thermodynamics. In each case the data is not merely large; it is shaped by regularities that constrain the possible outcomes.

The second is coverage: the density of observations relative to the meaningful space being modelled. A dataset of millions of cells sounds large. The question is whether it is dense relative to the biological space the model is expected to cover.

Coverage is not the same as scale

Single-cell transcriptomics does not offer the same kind of compression as language, proteins, or weather. A cell’s state reflects genotype, tissue, disease, environment, developmental history, time, sampling conditions and perturbation context – any of these can change the biology being measured.

The adult human body contains about 30 trillion cells.¹⁰ Public single-cell resources are large – CZ CELLxGENE reported more than 93 million unique cells as of October 2024¹¹ – but those cells are spread across datasets, tissues, donors, species, disease states, protocols and levels of curation. Relative to the full space of human cellular variation, the sampling is sparse. There is also sparsity within each measured cell: single-cell RNA-seq profiles contain many zero counts – some from genes that are genuinely silent and some from transcripts that were present but not captured. The result is sparse sampling of a vast state space, with incomplete measurement at each point.

A cell’s state reflects genotype, tissue, disease, environment, developmental history, time, sampling conditions and perturbation context – any of these can change the biology being measured.

There is a deeper asymmetry. Proteins became tractable because folding compresses an astronomical sequence space into a finite set of forms and evolution sampled them close to completion over the course of four billion years. Language has a saturation point, where additional text eventually teaches the model less. Cellular state space has no comparable point. Each new cohort, tissue, perturbation and disease context expands the relevant space. Scale stays useful but incomplete: more data does not automatically make deployment in-distribution.

Why cell types are not the grammar

The obvious rescue is to say that cell types are the grammar – a finite catalogue of identities that compresses the space the way folds compress proteins. Cell type labels are essential for organising experiments and interpreting data, but they are not hard physical constraints. As dynamic processes such as differentiation, activation, stress response and disease progression are measured more precisely, many cellular identities appear as continua and trajectories rather than fixed bins. The categories stay useful, but they do not provide the stable grammar that physics provides in protein folding or weather.

This matters because a model can perform well on cell-type annotation or integration and still fail at mechanism. Annotating a state is not the same as predicting how that state responds to a perturbation. Integrating datasets is not the same as identifying which target will move a disease-relevant pathway without unacceptable toxicity. The closer a task sits to organising known biology, the more these models can help. The closer it sits to predicting unseen biology, the more sparse coverage begins to matter.

Drug discovery is an out-of-distribution problem

This is the heart of it. Drug discovery rarely asks whether a model can recognise a pattern it has already seen in a slightly different form. It asks whether the model can support a decision in a context that is biologically, clinically or experimentally new.

A new target may sit in a pathway studied in one tissue but not another. A perturbation may behave differently across cell states. A patient subgroup may carry a disease mechanism underrepresented in public data. These are not edge cases; they are the substance of discovery.

Drug discovery rarely asks whether a model can recognise a pattern it has already seen in a slightly different form. It asks whether the model can support a decision in a context that is biologically, clinically or experimentally new.

It is also why benchmark design matters so much. When a model is evaluated with data splits that keep training and test examples close together, its performance can look stronger than it will be in deployment. The right question is not only whether the model does well on held-out data, but whether that held-out data reflects the kind of novelty the model will actually face.

From expression matrices to biological relationships

The architectural implication follows directly. When the relevant structure cannot be learnt implicitly from dense coverage, more of it must be represented explicitly. For drug discovery, that structure is largely relational. Biology is not only a matrix of expression values; it is a system of typed, directional, context-specific relationships. A drug inhibits a target. A target participates in a pathway. A pathway influences a phenotype in a tissue context. A resistance pathway becomes relevant when another node is blocked.

This is closer to how discovery decisions are actually made. Is a target safe to inhibit in hepatocytes? Is an observed expression change causal, compensatory or incidental? Which pathway might compensate if a node is blocked? Will two interventions reinforce each other or undermine one another? These questions need context, directionality and evidence.

Biomedical knowledge graphs are one way to represent that structure and public resources already show the approach is feasible at scale. PrimeKG integrates more than four million relationships across biological scales.¹² SPOKE contains about 27 million nodes and 53 million edges across biomedical entity types.¹³ But size is not the point. The point is whether relationships are represented in a way that supports reasoning.

An edge should not simply assert that one entity is associated with another. It should carry the evidence behind the relationship: the experimental system, direction of effect, tissue or cell type, perturbation readout, binding measurement where relevant, literature provenance and confidence. When relationships carry evidence and context, a model can reason over mechanism rather than mere correlation. It can ask whether a claim holds in the relevant setting, whether the evidence comes from a suitable model system and whether the direction of effect aligns with the intervention.

Structure is the bottleneck

None of this argues against single-cell foundation models. Rather, it argues for assigning them the correct role. They can organise cellular atlases, support annotation and surface patterns and improve representation learning. The risk is treating those capabilities as proof that the same model can automatically solve target discovery, patient stratification or perturbation response in genuinely new biology.

For teams evaluating AI systems in drug discovery, three questions help:

• Does the benchmark resemble the decision being made?
• Is the model interpolating or generalising?
• Does the representation support mechanistic reasoning, or only compress correlations?

I do not think the future is a choice between single-cell foundation models and mechanistic knowledge representations.

The stronger direction is to combine them: use foundation models where they are effective at learning representations and use structured biological knowledge where decisions require mechanism, context and evidence.

The lesson is not that foundation models fail in biology. It is that they succeed when their assumptions match the structure of the domain. Single-cell state space is vast, sparse and shifting. Drug discovery questions are mechanistic, relational and often out-of-distribution by design. Larger models may help at the margins, but the deeper requirement is structure. Drug discovery needs AI that can reason over the organisation of biology, not only compress its measurements.

References

1. Jumper J, et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021).
2. Lam R, et al. Learning skillful medium-range global weather forecasting. Science 382, 1416–1421 (2023).
3. Hersbach H, et al. The ERA5 global reanalysis. Quarterly Journal of the Royal Meteorological Society 146, 1999–2049 (2020).
4. Brown TB, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems 33, 1877–1901 (2020).
5. Cui H, et al. scGPT: toward building a foundation model for single-cell multi-omics using generative AI. Nature Methods 21, 1470–1480 (2024).
6. Theodoris CV, et al. Transfer learning enables predictions in network biology. Nature 618, 616–624 (2023).
7. Hao M, et al. Large-scale foundation model on single-cell transcriptomics. Nature Methods 21, 1481–1491 (2024).
8. Ahlmann-Eltze C, Huber W, Anders S. Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines. Nature Methods 22, 1657–1661 (2025).
9. Kedzierska KZ, Crawford L, Amini AP, Lu AX. Zero-shot evaluation reveals limitations of single-cell foundation models. Genome Biology 26, 101 (2025).
10. Sender R, Fuchs S, Milo R. Revised estimates for the number of human and bacteria cells in the body. PLoS Biology 14, e1002533 (2016).
11. CZI Cell Science Program; Abdulla, S. et al. CZ CELLxGENE Discover: a single-cell data platform for scalable exploration, analysis and modeling of aggregated data. Nucleic Acids Research 53, D886–D900 (2025).
12. Chandak P, Huang K, Zitnik M. Building a knowledge graph to enable precision medicine. Scientific Data 10, 67 (2023).
13. Morris JH, et al. The scalable precision medicine open knowledge engine (SPOKE): a massive knowledge graph of biomedical information. Bioinformatics 39, btad080 (2023).