The predictive validity crisis: Pharma’s productivity paradox – Part II

The predictive validity crisis is not insurmountable, but overcoming it requires fundamental changes in how the industry approaches disease model selection and outcome measurement. Companies like Vertex, Regeneron and AstraZeneca offer examples of how this can be done.

Vertex and Regeneron have focused on what Dr Jack Scannell describes as “problem choice”. These companies prioritise diseases with exceptionally strong human genetic validation, often monogenic disorders where the therapeutic mechanism is well-established. When biological causation is clear, it becomes far easier to design model systems that are likely to predict human outcomes.

Vertex’s work in cystic fibrosis is a classic success story. The disease, caused by defective cystic fibrosis transmembrane conductance regulator (CFTR) chloride channel function, differs from health via a single, well-defined molecular defect. When Vertex restored chloride channel activity in patient-derived cells under laboratory conditions, it gave strong confidence the therapy would benefit patients – assuming the treatment wasn’t toxic. Crucially, the same chloride channels involved in lung pathology also regulated salt in sweat. This biological link enabled a practical measurement: reductions in sweat chloride within days predicted long-term improvements in lung function. Vertex aligned its model system (patient cells), therapeutic mechanism (CFTR correction) and measurement approach (sweat chloride testing) into a coherent predictive system.

AstraZeneca has taken a different approach, described as the ‘Five Rs’: right target, right tissue, right safety, right patient and right commercial potential. While broader in scope than model selection, it can also be viewed through the lens of predictive validity. By systematically addressing each domain, AstraZeneca reduces the errors and bias that disconnect decision criteria from real clinical outcomes.

Bad outcome measures depress predictive validity

In some cases, it may be possible to produce a reasonable pre-clinical model of human disease, but we struggle to measure human disease in a way that allows the model to be predictive. An apocryphal example comes from a drug trial for a progressive neurological disorder. In this case, drug efficacy was assessed by comparing the distance that treated vs. untreated patients could walk in six minutes. One neurologist notes they could not tell which patients were on drug or placebo, but the six-minute walk results seemed to depend on the time of day the test was performed.

Patients who arrived early could park close to the clinic and performed well on six-minute walk tests, while those arriving later had to park 500 meters away and struggled to complete the test. This is an extreme example, but noisy or artifactual measurements plague many therapeutic areas, particularly in neurological and psychiatric diseases.

Poor measurement of human disease can force even effective treatments into impossibly large, expensive trials to demonstrate statistically significant benefits. A drug may work, but noise in measurement systems can mask its effects unless trials involve thousands of patients over many years.

Beyond traditional model selection

Current model selection in the industry often depends more on historical accident than scientific rationale. As Scannell bluntly states, “the main reason that a particular preclinical model is chosen is because either it’s the one the CRO has, or it’s the one your PhD supervisor used.” This approach has left the industry using models that don’t work precisely because they don’t work – unsuccessful models do not render themselves commercially redundant through success and so remain in use indefinitely.

A systems thinking approach starts with the endpoint and works backwards: what do we want to achieve in patients? How can we measure that outcome reliably? What model systems can predict success? This reversal of conventional thinking – from endpoint to model – can identify tractable problems with current scientific tools.

The innovation paradox in algorithmic approaches

The industry’s approach to computational innovation often reflects the same problems plaguing model selection. Companies frequently apply sophisticated algorithms to fundamentally flawed datasets, creating an illusion of progress while reinforcing existing limitations. The focus shifts to coding efficiency and mathematical sophistication at the expense of data quality and biological relevance.

True algorithmic innovation in drug discovery requires aligning computational approaches with problems where the underlying biology is well-understood and measurable. Methods such as causal inference Bayesian approaches and advanced machine learning techniques can provide genuine insights – but only when applied to high-quality, predictive datasets.

Regulatory and cultural barriers

Interestingly, regulatory agencies may be less conservative about model requirements than pharmaceutical companies assume. In cancer, for instance, animal models are notoriously poor predictors of human outcomes, yet companies often invest heavily in generating positive preclinical data not because regulators require it, but because clinicians and patients expect it before enrolling in trials.

This creates a perverse situation where companies spend millions on non-predictive studies to satisfy stakeholders who know the models don’t predict human outcomes. Breaking the cycle requires cultural change alongside scientific innovation.

The path forwards

 Solving the predictive validity crisis demands integrated thinking: choosing targets with strong human validation, using models that reflect human biology, applying outcome measures that capture meaningful clinical change and matching algorithms to high-quality datasets. Success won’t come from simply improving individual components – faster screening, better chemistry, more sophisticated AI – but from designing coherent systems where each element reinforces the others’ predictive power. The companies that master this integration, following Vertex’s example, may finally reverse the decades-long decline in pharmaceutical productivity.

The industry’s future depends not on accelerating existing approaches, but on rebuilding the fundamental systems that generate predictive insights about human therapeutic response. Only then can technological advances translate into genuine medical progress.

Missed Part I? Read it here.