How we attempted to measure startup success

Posted by: John Tozzi on December 29

Michael Arrington at Techcrunch raised an eyebrow today at our list of startups that have received the most venture capital over the last four quarters, saying that search engine Cuil doesn’t belong on any list of most successful start-ups. Fair enough to quibble. Let’s just be clear about how we reported this.

Our goal was to tap “the collective judgment of the venture capital community,” professional investors whose business is to bet on the companies the think have the best shot to succeed. It was the only empirical way we could think of to cull a list of successful startups. We got our data from the MoneyTree Report by the National Venture Capital Association and PricewaterhouseCoopers. As we explained, we looked at the seed- and early-stage companies that received the most venture capital investment over the last four quarters available (Q4 of 2007 through Q3 of 2008).

Cuil raised $33.25 million in that period, and whatever you think of the company or their product, the amount raised indicates that investors judged them worthy of significant backing, as some TechCrunch commenters noted. Obviously, that’s different from them getting traction or becoming profitable. But they made the list under the criteria we used, which we made clear from the start.

It’s worth noting that many of the companies on our list are developing drugs or cleantech products that are years away from going to market and by no means guaranteed commercial success. Cuil has a product on the market. If we excluded them, would we be criticized for leaving them off the list while including drug discovery and development startups?

As I said, we picked what we thought was the most empirical way of measuring startup success, and it’s fair to disagree. (Among other things, using VC investments as the bar excludes the many successful bootstrapped startups.) We even invited readers to take issue with how we selected the companies and offer their own measures of startup success. So quibble away.