Can You Predict a Startup’s Success Based on the Concept Alone?
24 September, 2015 / Articles
It’s easy to make fun of bad startup ideas – Airbnb for toilets? – but it’s not so easy to pick the good ones ahead of time. Just ask venture capitalists, the vast majority of whom lose money. The difficulty of separating good ideas from bad is part of why angel investors end up investing based largely on the founding team.
So does the initial idea matter at all to a startup’s success?
New research helps answer this question and reinforces just how hard it is to pick a startup based on the idea alone. The paper, by Erin Scott of the National University of Singapore, Pian Shu of Harvard Business School, and Roman Lubynsky of MIT, finds that the perceived quality of a startup idea predicts success in some sectors, but not in others. If you’re investing in startups in life sciences or energy, for instance, the initial idea seems to matter more than if you’re investing in software or consumer products.
The researchers studied a dataset of 652 ventures from MIT’s Venture Mentoring Service (VMS), which connects founding teams with mentors. What makes the data interesting is the way in which mentors select which teams to work with. Potential mentors “receive an objective, standardized summary of the proposed venture, composed by a VMS staff member,” and only very limited information about the team. “On the strength of the summary alone, without meeting the entrepreneurs, mentors must decide whether they want to work with a venture,” the researchers write. In other words, their decision depends almost entirely on their opinion of the potential startup’s idea.
The researchers measured initial mentor interest in response to the summary of the venture’s idea, and then compared that measure to the venture’s eventual outcome. The more initial interest from mentors, the greater the chance that the venture was successfully commercialized and that it raised venture capital or angel investment. Overall, good ideas – as judged by the mentors — had a greater chance of succeeding.
But how much more likely is a seemingly good idea to succeed, compared to an average one? Overall, 22% of the ventures were successfully commercialized. (Commercialization was measured by whether the venture launched a product or service, with evidence of repeated sales.) Ventures that were a standard deviation above average in terms of mentor interest were about 26% more likely to be successfully commercialized. That’s a meaningful and statistically significant increase, but it also suggests that predicting startup success based on idea alone is very difficult. The initial idea helps predict success, but plenty of other factors must matter.
Could the link between mentor interest and success just mean that mentoring is valuable? The researchers controlled for that possibility. First off, VMS is set up such that every venture has access to mentoring, even those based on less popular ideas. Second, when the researchers controlled for how much mentoring a venture received, the link between interest in the initial idea and eventual success remained.
Notably, the relationship between mentor interest in the idea and eventual commercialization was largely driven by highly rated ideas. An average idea wasn’t all that much more likely to be commercialized than a below-average one. But a highly rated idea was significantly more likely to be commercialized. This makes sense, as the venture capital world is driven by big successes. Separating good ideas from average ones may be even more important than separating the average from the bad. (The relationship between mentor interest and commercialization held even after the very most popular ventures were removed, so this effect wasn’t just driven by a few great ideas.)
When the researchers broke up the data by sector, things looked different. They grouped R&D-intensive sectors like life sciences, hardware, and energy together, and less R&D-intensive ones like software and consumer products together. For R&D-intensive ventures, the relationship between mentor interest and commercialization was even stronger, particularly when the idea was based on academic research or intellectual property. But for software and consumer goods the relationship was no longer statistically significant.
Think of it this way: if the venture “idea” includes patent-protected technology in an industry with high entry costs, it’s going to be easier to determine that the venture has commercial potential. For web and mobile ventures, which are less likely to have intellectual property, and where entry costs are lower, it’s harder to know up front whether a venture will have a real, sustainable competitive advantage.
None of this means that good ideas don’t matter to startups. But this research does reinforce the idea that it’s difficult to pick a good idea early on in a startup’s life, particularly in less R&D-intensive industries. Investors may therefore be justified in betting on other factors, like the quality of the founding team or early traction with customers.
No doubt some investors will choose to ignore all this, believing they are unusually capable of picking out good startup ideas. Some of them may even be justified in their self-confidence. But a final word of warning to angel investors and VCs who see themselves as particularly expert in this regard. The researchers checked to see if “expert” mentors were any better at picking ideas than the group overall. They looked at mentors with experience in the venture’s industry, as well as mentors with a PhD. Neither group was any better at predicting which ideas would succeed.
