The Opportunity Cost of Misdirected Talent: An ARPA Counterfactual

Speak to any UK researcher, and you might be surprised about our strength of opinion on buckets. Since April, all research funding has been allocated three ways: curiosity-drive, strategic needs, and supporting innovative companies. Bucket 1 is responsive, distributed, investigator-led, designed to push research beyond the known. Bucket 2 is targeted, strategic, directed.

This is not new - Polanyi's Republic of Science shows similar debates in the UK stretching back to (at least) 1946, where it was the vice-chancellors who were driving:

In the view of the Vice-Chancellors, therefore, the universities may properly be expected not only individually to make proper use of the resources entrusted to them, but collectively to devise and execute policies calculated to serve the national interest. And in that task, both individually and collectively, they will be glad to have a greater measure of guidance from the Government than, until quite recent days, they have been accustomed to receive.

Today, it is the government that wants to see return on investment, and ambitious, curiosity-science that is being cut. While this is a UK-centric funding drive, it is mirrored in the USA and represents a wider capital allocation story.

What might the impact be of a more directed approach to innovation? I already see behavioural change in how colleagues describe the intended impact of their research; far less 'foundational knowledge' and far more 'application-informed' research. I'm being asked to assess what is most likely to succeed... and STFC-area early-career fellowship discovery-science applications are quietly becoming EPSRC-applicable instrumentation development; out is the awe, and in is the impact.

This goes far beyond abstract strategic direction; funding changes will reshape the facilities that we carry forward, but will have clear human impact on the research community.

Reams have been written about the future impact, but I was struck by a parallel from the 1950's, in the establishment of two widely copied funding approaches whose own fortunes have risen and fallen over the decades: The National Science Foundation (Bucket 1) and the Advanced Research Projects Agency (Bucket 2 - and the model for the UK's own ARIA).

What can we learn from the benefits - and risks - of directing research by studying the most copied funder structures in history?

Innovating in innovation: NSF and ARPA

In 1945, Vannevar Bush delivered his report to the president, claiming 'basic research is the pacemaker of technological progress'; this document set the founding model for the NSF in 1950. At it's core the Bush model implied that the frontier of knowledge is best known by the people standing closest to the research; In Stokes' framing, this rewards so-called Niels Bohr type researchers: a quest for fundamental understanding, without a-priori consideration of application.

In 1958, ARPA was formed. From the business school researchers who distil the essence of innovation to translate - with mixed success - to industry, to the policy makers building replica agencies (ARPA-E, IARPA, and ARIA) with different macro-goals, people love the ARPA model.

At the centre are the programme directors (PDs), who are borrowed - often from academia. These are the unit of research activity, and the revolution of the ARPA model is the relentless focus on fixed-term secondments over institution building; the nexus of funding, responsibility, and empowerment lies in a single individual who points in a direction and shouts 'go!'.

Lots has been said about the ARPA model. But over it's lifetime, I'm not sure that the counterfactual has been robustly questioned: would innovators have produced more and better research outputs if ARPA had not existed? And what can this tell us about buckets of research funding?

The Gradient and the Pin

All universities have a research strategy, but when I am on a hiring panel, I am faced with a more prosaic choice: do we bet on the most promising person, or do we recruit to a more narrowly defined pre-defined growth area? There's never an easy answer - in essence, we are choosing between our own Bucket 1 and Bucket 2, with our expert panel having to agree on the Pareto front of curiosity and strategy. And our conversation always reaches 'but how will they fund their research?'

Traditional funding agencies use responsive funding to allow the community to hill-climb; the Bush approach implies that the frontier of knowledge has a gradient, and the people standing closest are the only ones positioned to know which direction is worth climbing.

ARPA flips this on its head - PDs put an (ambitious) pin in the map, and fund whoever can be persuaded to work towards this, regardless of whether this direction is most promising or whether the person funded was best placed to do the work. ARPA seeks Pasteur's quadrant researchers, who can deliver both fundamentals and application.

We have our two distinct models; explore and exploit, bucket 1 and bucket 2, NSF and ARPA. The question is, does redirecting researchers cost you the destination, or does it cost the advantage of the researcher already being at the frontier?

We can't assume that a researcher redirected to Pasteur's quadrant automatically becomes more useful or productive; they may in fact lose the advantage of domain knowledge that made them valuable. The enthusiasm for Pasteur's quadrant risks treating all redirects as productive, where it can just as easily be resource misallocation.

This isn't an unambiguous defence of the NSF model; delegating responsibility to peer review can just as easily turn into 'giving money to smart people', lacking procedural accountability in a different way.

The Value in Not Knowing You've Won

Many ARPA-funded researchers left no public record. However, we do know about a specific group: the scientists that advised ARPA were the JASONs. This group of - initially all male - academics over-indexed on theoretical and nuclear physics, and went on to include 11 Nobel Prize winners. It is fair to assume that they would have been successful with NSF funding, and did not just include those involved in defence research. In other words, their inclusion was a clear redirection of some of the most valuable research resource in the USA.

While we will never know the totality of their output, we do know that defence spending typically provides a 1-2x ROI compared with 8x for public R&D; it is suggestive that pound-for-pound, the long-run ROI of most researchers would have been higher if NSF-funded than ARPA-funded.

But of course, what about the moonshot successes unique to the ARPA model?

An irony of ARPA is that some of the most common civilian successes - the internet or seismology underpinning the theory of tectonic plates - survived because the flat autonomous structure of ARPA didn't know what projects to protect or to kill. In essence, they were civilization-scale wins despite having emerged from human-computer symbiosis (not nuclear command-and-control) and nuclear anti-proliferation respectively; ARPA's structure allowed them to succeed.

However, other well-known projects such as the Grand Challenge (autonomous vehicles) and the Red Balloon Challenge (social engineering) were PR exercises dressed up as research funding: the former cost far more to run than it awarded to the winners, and the latter used capability that had long been available - neither meaningfully contributed to the development of the fundamental capabilities in the fields. Here, it was ARPA's structure that allowed these to proceed.

The Principal Problem

What links these serendipitous wins to misaligned behaviours? The most intriguing aspect of all directed research - and specifically ARPA - is the principal-agent question. Resources flowed via the Pentagon (the principal), who expected defence innovation while (mostly) accepting ARPA's unique role in high-risk and early stage work. ARPA (the agent) provided a lightweight structure for delivery of innovation. But nothing comes for free - all principals expect return on investment, and in domains where work is hard to assess but results are easy to measure, the agent is implicitly incentivised to reduce risk and deliver results.

For defence-aligned research, the fundamental 'Pentagonian Bargain' that the organisation can maintain autonomy and secure resources holds when stakes are low, and breaks under pressure. This is particularly clear from the TIA programme, with a deep civil liberties failure a 'dishonest misuse of DARPA'.

This issue is far broader than ARPA; all directed research suffers from principal-agent problems, and this is not a criticism specific to the ARPA model. In fact, the programme-officer model can survive and translate to a positive-sum principal, such as at the Gates Foundation. But the principal's own incentives dominate the agent's autonomy in practice - and it is precisely this that can lead to a hidden drive towards misalignment.

And while a clearer public-service remit for research has been used in derivatives of the ARPA structure, most notably in ARIA, this has not provided immunity against criticism of their investment decisions. Accusations of poor return to stakeholder will persist, precisely because the process of high-risk research is impossible to evaluate, but the cost is all too tangible.

The Counterfactual We Can't Run

The opportunity cost of redirected science - be it ARPA or Bucket 2 - is invisible, because the paths not taken cannot be observed. Despite over 60 years of operation, we do not have a mechanism to approximate this cost.

What is clearer is that the return on awe - the Bucket 1/NSF model - and return on investment lie on different axes. My ROI-based critique actually misses that their research often produced neither; would researchers have otherwise studied the psychic abilities of a magician with taxpayer's funding?

We now have enough ARPA derivatives with different principals and risk appetite - ARPA-E, ARPA-H, ARIA, Gates-style - to understand which aspects of this experiment in innovation are valuable. This isn't a novel thought - the IFP's Caleb Watney has been pushing this question for years.

And the Buckets? Writing a recent grant proposal I found myself having to frame a study as foundational, or to define an application focus. This felt uncomfortable, because I did not know whether I was protecting the taxpayer or replaying the 'Grand Challenge', prioritising legible impact today over the understanding that will matter in twenty years.

I do not have a mechanism to measure my own opportunity cost of short-term impact focus over long-term curiosity focus, but neither does my institution or UKRI. And with sixty years of the most copied funding models in history, nobody has built a counterfactual tracker.

While - unlike Polanyi - I do not necessarily believe strategic direction is always wrong, any funder (government, industry or neo-lab) that redirects talent without attempting to price what this gives up must accept that they are running an uncontrolled experiment and calling it a strategy.

Bounding What You Can't Predict

So, what can we learn from curiosity-driven vs strategically directed research?

It might tempting to try and build a system that predicts optimal pin placement - better strategic direction. But that's wrong for AI-for-materials and a category error applied to one-off research.

Instead, I think we must shift our mindset: there is no value in attempting to optimise the direction of exceptional research talent, but there is systemic value in shifting the median direction of research. My own day-job is built on the premise that you must design recipes to improve distributions, not the hero device - a single champion result is statistically indistinguishable from a lucky draw.

This will look like avoiding making the worst allocation of research talent - the Red Balloons with empty impact, the Bohr-to-Pasteur costs where researchers are ill-equipped to make this transition, the TIA-type principal capture. For UKRI, this counterintuitively means applying far greater critical appraisal to Bucket 2 than Bucket 1; misdirection has a real cost.

In other words, the goal is not to 'gamble well' with funding, but to bound the tail risk of misdirection. In this way the right question is not 'how do we fund the next Pasteur', but 'how do we stop funding the next Red Balloon.' Shifting the median direction of research beats chasing the outliers, because the outliers are already at the productive frontier.