• Startup & Investor Resources
  • Recent Posts
  • Search
  • Contact Me

Instigator Blog

Lean Analytics, startups, angel investing, product management and more!

Defining Success

July 7, 2011 by Ben Yoskovitz

The Lean Startup model is all about running experiments, learning and making informed decisions on what to do next. I’m clearly a believer in the model, but there are some parts of it that are quite nuanced and challenging to figure out. One of those is with respect to experimental design.

A good experiment needs a few things including a hypothesis (that’s testable) and a sizeable enough group of testers. It also needs a definition of success. A good experiment either fails or succeeds. In order to know if an experiment has failed or succeeded, you have to have a measurement for success (anything else is failure). Without a clearly defined target / definition of success, you’ll likely end up in a wishy washy middle ground. You may be running an experiment and collecting metrics, but you can’t really interpret the results without something to compare them to. The missing definition of success leads too easily to a positive justification of the results, after the fact.

So what can you do?

Before starting an experiment, define success. Put a line in the sand. If you cross it, you’ve succeeded. If not, you’ve failed. Remember: Success in this case doesn’t mean you stop learning and iterating, it just means you move to the next experiment and the next phase of your startup.

Defining success is very hard.

There may be market or competitive comparables that you can use. There are a fair number of published numbers for certain things that lots of startups deal with: engagement, conversion rates, etc. But it’s definitely hard to put a stake in the ground and know that if you get there, you’ve succeeded (at least in the experiment you’re running.)

From what I’ve seen, most startups don’t do this very well. They’re not sure how to define or measure success, they base a great deal on gut and intuition. I’m all for gut and intuition – you need both in droves to be a successful entrepreneur – but they’re also great liars. Entrepreneurs are great at fooling themselves (and sometimes we have to in order to keep going!) A pre-defined measure of success can’t lie. It might be wrong, but it turns very gray issues into more black-and-white ones.

There’s very little in a startup that’s black-and-white, but you owe it to yourself before you start anything (whether it’s early validation of your idea, testing the value of a specific feature, implementing a business model, etc.) to come up with a definition of success.

Filed Under: Customer Development

Want more content like this?

Signup for free and you'll get new content as soon as it's available. Thanks!

Ben Yoskovitz

Founding Partner at Highline Beta, a hybrid venture studio and VC firm that works with large, ambitious companies to identify new areas of opportunity through internal and external innovation.

Previously I was VP Product at VarageSale and GoInstant (acq. $CRM), and Founding Partner at Year One Labs.

Angel investments include: Breather, Spoiler Alert, SendWithUs and others.

My bio »

Buy Lean Analytics

Lean Analytics

"Lean Analytics is the missing piece of Lean Startup!" - Dan Martell, founder Clarity

Get the book at leananalyticsbook.com

Get updates

I've moved to Focused Chaos a newsletter focused on startups, investing and more. Please visit there to subscribe and get weekly content.

  • Startup & Investor Resources
  • Recent Posts
  • Search
  • Contact Me
Views expressed here are mine and mine alone.

Copyright © 2023 · Built on the Genesis Framework