The predictive validity crisis: Pharma’s productivity paradox – Part I

The pharmaceutical industry faces a paradox that challenges conventional wisdom about technological progress. Despite revolutionary advances in molecular biology, genomics and computational power, drug discovery has become dramatically less efficient over the past seven decades. The numbers are stark: the average pharmaceutical company spent 100 times less per FDA-approved drug in 1950 than in 2010, adjusted for inflation. This increase occurred despite DNA sequencing becoming 10^10 times more efficient and X-ray crystallography improving by 10^4 times.

Dr Jack Scannell, an R&D productivity researcher, argues that this paradox arises from a fundamental misunderstanding of what drives pharmaceutical innovation. The answer is not technological capability but a concept called ‘predictive validity’ – the degree to which preclinical models accurately predict human therapeutic outcomes.

The accidental fall of predictive validity

In retrospect, the remarkable productivity of early drug discovery can be explained by predictive validity.

Some in vitro and animal models proved surprisingly good at predicting human outcomes, particularly for anti-infectives, blood pressure drugs and treatments for excess stomach acid. Between the 1950s and 1970s, low regulatory hurdles meant researchers could move quickly from lab tests to human trials. The ‘design, make, test’ loop was fast. In fact, some drugs were tested for efficacy in humans with minimal pre-clinical study. For example, every major class of antidepressants was discovered by administering compounds to people and observing effects. No antidepressant class has ever been discovered through model systems without prior knowledge of similar compounds’ impact in humans.

In this fast ‘design, make, test’ loop, humans – the ultimate target – served as their own model system far earlier in the R&D process than is the case today. Predictive validity was essentially perfect, since researchers were testing the very system they aimed to treat. As Scannell puts it, “people are a pretty good model of people.”

However, this approach could not continue. As ethical standards tightened and regulatory requirements increased, more up-front work was required before human trials, which also became far more costly. In addition, the pre-clinical models that genuinely predicted human outcomes yielded effective drugs and so rendered themselves economically redundant. The world no longer needed endless new antibiotics, blood pressure medications or stomach ulcer drugs.

Today, for major untreated diseases, we cannot for ethical reasons conduct risky trials in humans. However, the preclinical model systems still in use routinely fail to accurately predict human efficacy, particularly in Alzheimer’s disease, cancer, and many psychiatric and neurological conditions.

The false positive trap

The mathematics of drug discovery reveals why poor preclinical models are so damaging. Given that the vast majority of randomly selected molecules or targets are unlikely to yield effective treatments, screening systems must have high specificity to be useful.

Poor models essentially become ‘false positive-generating devices,’ identifying compounds that appear promising in preclinical testing but fail in human trials. The faster and more efficiently these poor models are run – through high-throughput screening, combinatorial chemistry or AI-driven approaches – the faster false positives are generated, which then tend to fail at great expense in human trials.

The AI amplification problem

Current applications of artificial intelligence in drug discovery often fall into this trap. Some early AI-first firms essentially automated the process of running fundamentally non-predictive models faster. They selected model systems that were optimal for machine learning, but poorly suited to capturing human pathophysiology. As Scannell observes, this pattern has repeated across multiple technological waves over several decades, from computer-aided drug design to high-throughput screening and combinatorial chemistry.