What Big Companies Get Wrong About Innovation Metrics
19 May, 2015 / Articles
The fear of getting Netflix-ed or Uber-ized is spurring big companies to dial up their investment in innovation. Companies as diverse as AIG, Disney, and Intuit have been building innovation teams, launching “accelerator” programs to attract promising startups, and giving employees seed funding to test out new ideas with real customers.
But as investment increases, many companies are struggling with a challenging question: how do you know whether your chosen innovation strategy is actually bearing fruit?
Beneath that question is a very real worry. If this new crop of Chief Innovation Officers, company-bred venture capitalists, and creative catalysts can’t prove that they’re moving the needle on things that actually matter to their employer, their jobs will almost certainly evaporate.
When my publication, Innovation Leader, surveyed 198 senior innovation executives late last year, in partnership with the consulting firm Innosight, we found them using two different kinds of measures.
One type is what we call “activity” metrics, which show that you’re busy stoking the boilers of innovation. Some examples: how many employees have been trained in “lean startup” methodology, or how many new product ideas are currently being researched. Two-thirds of our respondents, for instance, said that they were tracking the number of projects in their development pipeline.
The other type of metric, an “impact” metric, shows that the ship is actually going somewhere. Market share, cost reductions, and profit margins of new businesses are examples of impact metrics. The most widely used metric among our respondents, for instance, was revenue being generated by new products or services over the course of their first few years in the market. Sixty-nine percent of respondents said they’re gathering that data.
The five most commonly used metrics were:
• Revenue generated by new products
• Number of projects in the innovation pipeline
• Stage-gate specific metrics, i.e. projects moving from one
• stage to the next
• P&L impact or other financial impact
• Number of ideas generated
• The perfect innovation metric is elusive, and interviews with executives after the survey highlighted five ways that most measurement efforts go wrong.
Alignment can take a while. The first was the difficulty in creating a set of metrics that are aligned with what senior leadership actually cares about. It may be entry into a new geographic market, or the creation of a service revenue stream in a predominantly product-driven company. But innovation executives who had been in the role for two or more years almost universally said that they have moved away from more generic activity measures — like how many people had participated in a company crowdsourcing initiative — and toward more specific impact measures that matter to the CEO or COO.
Patience is a rarity. In businesses with billion-dollar bottom lines, it’s nearly impossible to create significant ripples in a year or two. Many of the startups now disrupting major industries like hospitality or media had four or five years to figure out what opportunity was actually worth pursuing, and build a working mechanism for attracting customers. After all, their survival hinged on figuring out how to make things click. (Netflix, for instance, got started in 1997, and introduced streaming a decade later.)
Large public companies can be more capricious. “The challenge everyone will face is patience,” says Adam Yaeger, Innovation Director at $9 billion insurance firm Assurant. “It takes a lot longer to hit that hockey stick [of growth] when you are building something from scratch,” rather than trying to layer incremental revenue onto an existing business. No matter what innovation measures a company decides to track, two or three years of data collection may not be enough to determine the real impact.
Failure isn’t fun to measure. Investors in new ventures acknowledge that most will fail, and corporate innovation leaders realize that most of their explorations and tests will lead to dead ends. But most established business units in a company have evolved to avoid failure, not “celebrate it,” as many Chief Innovation Officers like to exhort. Tracking dollars put into start-up partnerships that go kaput, or a new Apple Watch app that gets downloaded by two dozen people, doesn’t make you a superstar in most organizations. Core to the work of innovation, however, are edgy explorations and high-risk tests.
Hershey executive Deborah Arcoleo says she is interested in developing metrics related to what her team at the Advanced Innovation Center of Excellence has learned from its collaborations and experiments — one of which focuses on 3D printing with chocolate. “As you conduct learning experiments, the amount that you know goes up so the amount that you don’t know goes down,” Arcoleo says. “Then you feel more comfortable starting to invest a little bit more.” Putting a number on that understanding of new markets and technologies — whether it makes sense for one’s employer to enter them, and how — is difficult.
The vision thing. It can be easy to count the number of innovation workshops held at company outposts around the world, or tally revenues from add-on services developed by an innovation group and then handed over to the sales force. But white-sheet-of-paper innovation efforts are fundamentally different from growing and optimizing existing lines of business. Apple founder Steve Jobs famously didn’t believe in focus grouping new products, and when Starbucks founder Howard Schultz wanted to create a totally novel coffee-buying and drinking experience, he didn’t order up market research. The result? Products like the iPad and Starbucks’ new Reserve Roastery and Tasting Room in Seattle, a prototype for future Starbucks flagship stores.
Companies that aspire to be true innovators need to remember the role of vision and gut instinct, says Jason Berns, senior director of innovation at the athletic apparel company Under Armour. “Is there good revenue potential for what you’re working on, will it affect the business? That’s a group assessment,” Berns says. “How big could this be? Somebody needs to have a vision.” As an example, Berns brings up the company’s SpeedForm running shoes, which have interiors made using some of the same technology that produces bras, eliminating the interior seams that cause discomfort.
Measuring too much. There’s a danger that measurement sucks up resources better devoted to cultivating and testing new ideas. “A lot of large companies are highly analytical,” says Mona Vernon, vice president of the data innovation lab at Thomson Reuters, the financial media company. “They find real comfort with reports and governance. Which means that you can spend all your energy measuring stuff if you’re not careful.”
The writer E.B. White once compared attempts to analyze humor to dissecting a frog in a biology class: “Few people are interested, and the frog dies of it.” While the corporate impulse for dashboards and scorecards can’t be eliminated entirely — nor should it — there may be an inverse correlation between the intensity of a company’s obsession with measuring innovation and the breakthroughs it generates.
If they hope for their roles to endure and have an impact on their employer, corporate innovation executives need to develop a set of metrics that demonstrate a return-on-investment to senior leadership and business unit leaders. But they also need to develop the relationships — and at times fight — to have the resources and white space that allow them to incubate high-risk, potentially high-impact projects.
