The initial launch of your Minimum Viable Product (MVP) is a big step. It means you’re putting something into people’s hands to try. Keep in mind that this isn’t so much a launch as it is the beginning of a process. But you’re out there, and people are using your product.

Now you face one of the toughest challenges for startups: What to build next?

Chances are you have a laundry list of features you want to build and think you need, but you were at least reasonably rigorous in your elimination process and didn’t build them for the MVP. Now a certain amount of panic sets in because you know the MVP isn’t great and you skipped a lot of features you think are important and feedback is starting to roll in too! You have a priority list, but customers are also telling you what they want; aside from some bugs they’ve found, they’re giving you a list of features and telling you to build them. And the customer is always right … right?

Not necessarily.

Here’s a great post titled, Why Your Customer Feedback is Useless. It’s a must read. Go ahead and check it out, but come back. Or just finish this and then go read it…either way, stick around ‘kay?

Prioritizing feature development after the release of your first MVP is very hard. We’re facing that with Localmind, which is out in the wild (in limited release) and there’s a flood of activity – customer feedback, complaints, kudos, our own brainstorming, advisory input, etc. Plus we now have to monitor multiple sources of information from Twitter, Facebook, Uservoice, email, etc.

So we’ve going through a few exercises and thought processes to help us prioritize feature development without losing complete control over the process.

For starters, we try and always remember: Build -> Measure -> Learn.

If we build something, we better be able to measure the impact and learn from it. If we can’t, we have to really question why we’re building it, because it’s going to be hard to know if we’re right, wrong or somewhere in the middle.

Lenny (Localmind’s founder) has done an amazing job of putting together a couple dashboards tracking key metrics for judging the performance and success of Localmind. He’s using Geckoboard and Mixpanel; both are worth checking out. Having the metrics upfront grounds us in a certain amount of reality and reason. Even as we see users joining and excitement growing, we can pull back and look at the key metrics we identified and say, “Hold up, hype is all well and good, and tons of fun, but this particular metric isn’t where we want it to be.”

We put a lot of weight on our metrics (along with the qualitative feedback we’re getting) to help us prioritize feature development.

Next, we’ve broken up our high level priorities into two major categories:

1. Retention / Engagement
2. Virality

Every feature we look at building has to fit into one of those two categories. If it doesn’t, we’re not going to tackle it in the near future. And those two categories are prioritized too: Retention > Virality (at least for now). I’d say this should be the case for any web application (B2B or B2C) … if tons of people show up, but they don’t stay and they don’t come back, why bother having them show up in the first place? It’s possible to get them back through re-marketing to them, but it’s not easy.

One of the challenges with many B2C applications is that retention increases when you have more users (which typically requires virality.) It’s the chicken and egg problem. I think this drives a lot of startups to focus on virality first, aiming for tons of users, hoping that the volume alone will make up for a product that’s weak on retention. Possible, but very risky.

This simple categorization lets us ask a clear question, “Does the proposed feature improve retention or virality?” If “yes” (or at least, “we think so!”) then add it to the list, and we’ll continue to prioritize from there.

Further prioritization takes place inside each category. There could be 20 feature ideas that we think will increase retention. Which one should be built first?

Here are some criteria you can use (ranked in order of importance):

• Why? - There needs to be a reason why you think something is going to improve retention or virality. For Localmind we think a lot about how to delight users, how to implement surprise. Gamification is all the rage these days as a way of increasing retention (and virality), although I think it’s easy to say, “Let’s slap some points on there and people will love it!” It’s much harder than that. Asking the question “why” means you can put a hypothesis on paper about a specific feature and test directly against that, qualitatively and quantitatively.
• Overall Hypothesis Validation – When starting your startup, you’ve got hypotheses written down that you’re trying to validate. Any feature you build has to be driving towards the validation or invalidation of those hypotheses.
• Measurability – The feature’s impact has to be measurable. I’ve already mentioned this above.
• Development Time – Time is one of a startup’s biggest enemies. The relative development time of each feature has to be compared.
• Complexity for User - Complexity kills so many things, including a user’s experience with a web application. Especially early on. You know a proposed feature is getting too complex when you string a bunch of sentences together that all start with, “And it’d be great if it did…” “And” in this case is the enemy of success.
• Risk - There’s almost always some form of risk when building new features. There’s a certain amount of technical risk in terms of how it might impact the codebase. And there’s user risk in terms of how people might respond. There’s also risk in terms of how a new feature drives future development, potentially setting you on a path you don’t want to pursue.
• Innovation - Building functionality is typically quite easy. There’s no shortage of other web apps to steal ideas from either. And that’s OK. But it’s worth looking at how innovative a new feature really is, whether it stands out creatively and has the potential to make people say, “Damn, that’s smart.”
• User Input - Your users are important. Their feedback can be important. But relying on them is risky. In some cases, it’s better to take advantage of them for their own good. Even in a case where an overwhelming number of users want something it may still not be the right thing. Doesn’t mean you ignore them, and certainly providing quick, positive feedback and customer service is absolutely essential, but don’t automatically prioritize the features they say they want.

Distractions literally kill startups. They may do it slowly, you may not even notice it’s happening, but it happens all the time. Feature development in and of itself can be a distraction if it’s not being done in a prioritized way. Building stuff for the sake of building it is startup suicide. So having a rigorous and honest prioritization process for feature development is critical to controlling distraction and eliminating waste.