Virtual cell model rankings just got a major upgrade

Shift Bioscience, a biotechnology company focused on the biology of cell rejuvenation and therapies for age-related diseases, has announced significant progress in optimising how virtual cell models are ranked for gene discovery. The company announced results from a new study that introduces novel metrics and baselines designed to better assess the performance of these models. This enabled more effective development and prioritisation in their rejuvenation target discovery pipeline.

The power and challenge of virtual cell models

Virtual cell models trained on single-cell RNA (scRNA) datasets present a transformative approach to studying phenotypic changes caused by gene perturbations. By simulating the up- or down-regulation of genes, these models can compress what would traditionally take centuries of laboratory work into mere months of virtual experiments. This capability is crucial for expanding target discovery programs and streamlining the transition to costly wet lab research.

However, despite their promise, benchmarking studies have highlighted that prominent virtual cell models have often been outperformed by a simple dataset mean when evaluated using performance metrics. This underperformance has hindered the full potential of virtual models in accelerating gene discovery.

Refining metrics to reveal true model performance

Led by Lucas Paulo de Lima Camillo, Shift’s Head of Machine Learning, the study explored both virtual and real-world data to uncover how experimental factors – such as control bias and weak perturbations – can distort the true evaluation of model performance using standard metrics.

The team developed a refined ranking system incorporating several key pre-processing steps aimed at focusing on biologically meaningful changes:

• Differentially expressed gene (DEG)-weighted score metrics
• Negative and positive baseline calibrations
• DEG-aware optimisation objectives

These innovations help  better highlight genuinely predictive models and ensure that poorly performing ones are identified and excluded from target identification programs.

Accelerating therapeutic discovery with more reliable models

“In this research, our team has shown that by focusing on the development of new metrics and baselines, we can more easily identify models that demonstrate strong predictability,” said de Lima Camillo. “The paper provides foundational data which will enable us to develop more powerful, biologically-useful perturbation models, ultimately accelerating our therapeutic pipeline and helping us to uncover new targets for rejuvenation therapeutics.”

With this enhanced ranking framework, Shift Bioscience is set to accelerate its discovery of rejuvenation targets, advancing the development of therapies aimed at combating age-driven diseases.