Is R&D Getting Harder, or Are Companies Just Getting Worse At It?
27 March, 2017 / Articles
We know innovation drives corporate growth. As Strategy& reported in its 2015 survey of 1,757 executives, “innovation today is a key driver of organic growth for all companies — regardless of sector or geography.” According to that report, the top 1,000 R&D spenders invested $680 billion in R&D that year, up 5% from the prior year. Historically, R&D has been viewed as the engine of national economic growth as well.
Despite the importance of innovation to companies, as well as to the broader economy, despite the 250% rise in the number of scientists and engineers engaged in R&D, and despite all the experts dedicated to helping companies innovate, the money companies spend on R&D is producing fewer and fewer results. In fact, my research shows the returns to companies’ R&D spending have declined 65% over the past three decades.
Not coincidentally, this decline in companies’ research quotient or RQ (a metric I’ve developed that measures R&D productivity, or how much output they get for their innovation inputs) mimics the decline in U.S. GDP growth over the past 30 years.
One possible explanation is that R&D has gotten harder. This is a theory advanced by Stanford economist Chad Jones. Jones proposed that two mechanisms were behind this: first, a “fishing out” (or cherry-picking) effect — the notion that the most obvious ideas are discovered first, so that the quality of remaining ideas is degrading over time. If you think about recent innovations such as personal computers, the internet, and smart phones, you might be skeptical about this idea, but we’ll examine it more concretely in a moment. The second mechanism is diminishing returns to research labor — the idea that adding more researchers decreases the number of innovations per worker, because it increases the likelihood that researchers are duplicating one another’s efforts. Both of these ideas seem plausible. In fact, Northwestern economist Robert Gordon, in The Rise and Fall of American Growth, makes similar arguments. The dismal outcome if Jones and Gordon are correct is that growth will decline to zero (other than for population growth).
I have a more optimistic explanation, which is that companies have gotten worse at R&D. While companies getting worse may not sound more optimistic than R&D getting harder, if it’s true, and if companies can restore their former RQs, then theory tells us the economy should enjoy perpetual growth as long as R&D investment continues.
So the big question now is whether Jones and Gordon are correct that R&D has gotten harder, or am I correct that companies have gotten worse at it. While the 65% decline in RQ suggests I’m right that companies have gotten worse at R&D, it will also look like companies have gotten worse if it in fact R&D has gotten harder.
How could we test this? One thought is that if R&D has truly gotten harder, it should have gotten harder for everyone. In other words, not only will average RQ decrease each year, but maximum RQ will decrease as well. Thus if I take the best company in each year (the one with the highest RQ), and compare it to the best company the following year, then the later companies will tend to have lower RQs than earlier companies.
That’s not what I found when I examined 40 years of financial data for all publicly traded U.S. firms. I found instead that maximum RQ was actually increasing over time! When you think about all the marvelous companies that have been created as part of the internet economy, that seems plausible, but there is still reason to be skeptical when you’re swimming against the tide. So the next thing I checked was whether the same pattern held if instead of looking at all public companies, I restricted attention to a particular sector, e.g., manufacturing or services. I found that maximum RQ was increasing within sectors as well. I then looked at coarse definitions of industry, such as Measuring Equipment (Standard Industrial Classification 38), then successively more narrow definitions, such as Surgical, Medical, And Dental Instruments (SIC 384), then Dental Equipment (SIC 3843). What I found was that as I looked more narrowly, maximum RQ did decrease over time (see the chart below). Thus, Jones’s theory might hold at the industry level.
The implications of the pattern are actually pretty exciting. What the pattern suggests is that while opportunities within industries decline over time, as they do, companies respond by creating new industries with greater technological opportunity. Once I saw this pattern it was easy to think of examples. In fact, many of these examples are referenced in the current debate on disruption. Some common examples are the death of the typewriter and its replacement by personal computers, and the death of landlines and their replacement by cell phones. While there are numerous other examples, what is true in the two cases that came to mind, is that the market for the new technology was actually much broader than that for the technology it replaced.
As an example, personal computers enjoy an installed base in the U.S. of 310 million machines, while the installed base of electric typewriters was only 10 million machines at its peak in 1978. While we can’t say whether this pattern of increasing maximum RQ will continue forever, it has persisted over the 40 years for which we have data.
So the good news is that while industries may be doomed, companies don’t have to be. They can move into industries with greater opportunity, while exiting industries with declining opportunity. That news also provides an important lesson in improving RQ: companies likely have to diversify to avoid diminishing opportunities in their own industry. This general pattern is at least a century old, and in fact was the genesis for industrial R&D. This genesis is captured in vivid case histories of DuPont, General Motors, and Standard Oil in Harvard business historian Alfred Chandler’s influential book, Strategy and Structure.
To summarize where this leaves us, it appears the decline in companies’ (and the economy’s) ability to drive growth from R&D stems from the fact that companies have gotten worse at innovation, rather than because innovation has gotten harder. This is great news, because the problem of companies getting worse is fixable, whereas the problem of innovation getting harder isn’t. The challenge, of course, is knowing what to fix and how to fix it.
The science man and innovator, Fernando Fischmann, founder of Crystal Lagoons, recommends this article.
