Are You Testing the Effectiveness of Your Website?

by Ben Yoskovitz

I recently attended an event in Montreal where Albert Lai spoke. Albert’s a serial entrepreneur with a number of successes (and failures) under his belt. Great speaker. One of the things he said really stuck in my mind (which I’m paraphrasing), “It’s all about the market. Don’t do things based on your own personal whims of what you think is right or wrong, good or bad. Listen to the market. It’s always right. And you’re typically wrong.”

It’s important advice - especially for startups - because we often live in a bubble building our web apps till our heart’s content, thinking we’ve just invented the next Google. It’s not that entrepreneurs don’t seek out advice, but whether we listen to it is another story. One of the biggest hindrances to taking advice (even our own) is time. We’re rushing to launch, building as quickly as we can to get a head start on the competition, and we don’t necessarily have time to sit back, get good feedback, assess it and adjust accordingly.

Nowhere is this clearer than in people’s websites. Most people will agree on the importance of design. There are exceptions (think: Plenty of Fish) which in my mind just prove the rule. Where most websites fail is in their organization and navigation. And in their missions. What’s the goal of your website? If you don’t have a clear answer to that you need one as soon as possible.

Everyone will “test” design out — just ask a bunch of people, “Does this new design look good?” But information architecture, navigation and the effectiveness of a website in meeting your goals … people just don’t test those things enough.

The Sooner You Start Testing Your Website The Better

The minute your website goes live, you should start testing whether it’s working or not. And one of the best ways is with heatmaps. A heatmap shows you where people are clicking on an individual page. I use a service called crazyegg. It’s a simple way to very quickly get a ton of insight into how people are using your website.

In the spirit of openness, I’ve included two screenshots below. One is for Standout Jobs’ home page and the other is for our employer sign-up page. This is only 4 days worth of data, and we haven’t changed anything significantly because of these results, but we will be making changes soon.

Standout Jobs Home Page Heatmap

standout jobs home page heatmap

Standout Jobs Employer Sign-Up Page Heatmap

standout jobs signup page heatmap

What Do You Think?

Instead of analyzing these results on my own, I thought it’d be interesting to see what everyone else thought. The heatmaps don’t show number of page views or clicks, but I think the sample size of data is big enough to start drawing conclusions. So what are yours?

February 11th, 2008
More in Marketing
What next?
Related posts:
  • There are many notable examples of bad design that works. Craiglist is another one. A lot of people will judge a site by its look; it’s the human nature. Would you trust a car seller dressed like a vagrant? Now that being said, if the vagrant looking seller is of exceptional help, you might return and talk to your friends about him.

    Some sites have amazing design but have very bad usability. You don’t know what to do. My mantra is to think Apple. Look at the iPod. How simpler can it get? Even my mom can use it. The design? Nothing really amazing about it…
  • It's often not easy to listen. Some design seems wrong, but prove to be very effective. Once I moved a link below the fold to provide more information on the action, CTR went down the drain. I then provided the link at the top and after the detail where it belonged, CTR went higher than without the details, while most people used the top link. Big fonts, user want always bigger, while we try to show them as much information as we can. We want useful interface, while user want usable interface. It is hard.

    Your interface seems to have that covered. There is one thing puzzling me in your heat map. On the home page the “Job seekers, click here.” link seems strangely popular for its location. In a way it may just be a good thing telling you candidates read through the presentation before deciding to click.
  • What do I think? What you have shown is amazing. I did not know that such a thing existed. With technology at your command and a willingness to test, you cannot go wrong. In an earlier post we had discusses the business start ups and it was mentioned that after the business has been launched, it is all about innovation, constant, unrelenting. You have got it right. All the very best to you Ben.
  • Juan
    That's one good looking tool!
    Do you think it's really better than Analytics' overlay view?
    I'll give it a try.
  • Juan
    Besides the visual appearance, I just noticed one point in which crazyeggs could be better than Analytics' overlay view: if a page happens to have two elements linking to the same page (such as a big button and a simple link), Analytics shows the same result on both (stats are based on URLs), whereas crazyeggs will tell you the difference, as it uses actual click coordinates, if I understand correctly.
  • MCF
    Bad design that works is not an excuse for bad design.

    Three easy/immediate changes to improve the design:

    (1) Move the sign-up/tour box above features/benefits and below the headers
    (2) Move the sign-up form to right below the features/benefits
    (3) Center the menu links to stay consistent

    The key to making design work is A/B testing lots of combinations over time. Heatmaps are pretty but if you want some real advice I'd need to see the actual quantitative information behind it all.

    That's all you get for free, email me if you want some real advice (seriously).
  • Those heat maps are very cool. I've never seen them before.

    I think Albert Lai's advice is critical for success. It reminds of of how Paul Graham says that a startup needs to not spend too much time coming up with the perfect idea and just launch. Then once the product is out, let the market decide what direction to take it in (or what your product/market is).

    It also reminds me of how 37 Signals says that less time is good at launch, and that it forces you to become more useful/valuable quicker.

    As for design tips, I used your thought and I thought the sign-up process was smooth. My suggestion would be to think about how a new user can easily explore and stumble upon a couple of jobs quickly. When I first used the site, I maybe clicked on two or three sites then left. I'm being super-picky here and only reporting on my individual experience.
  • hey ben,

    i know you guys have launched, but there is one guerilla marketing tactic that is used often for starting web apps, which is making up a fake website, with a different url, with a different layout, and a different way to present the service, and then see if people actually signup. actually provides objective data on what works and what doesn't
  • I think one very significant point to notice at this stage is the surpreme popularity of "candidates" rather than "employers". I think this shows a trend of having a higher base of job seekers. You might want to consider turning your focus here a little more since a majority of the marketing and useability is currently heavy on the employer side. I know you need a balance of both in your community and you need to do what you can to attract more employers but remember the importance of your existince visitors. :-)
  • I've learned that when creating a website, we do it as if our audience is a 3-year old child. Large buttons, simple directions, and a clear end goal. As long as we TELL them what to do, it all works out quite well. From your own heat maps, it looks like people are actually taking the time to check you out before they post, which is good, because it means they're interested!
  • There is no doubt that testing in an early stage can save time and money in the long run.

    Good post!
  • I'm based in Scotland and apart from setting up a couple of web based franchises (the good kind) i'm also creating a thesis on the design of websites.

    I can't agree with you more about testing. The complications of the consumer's website navigation can only be tamed through testing (typically A-B split).

    I highlighted the following five points:

    1. Perceived ease of use
    2. Perceived usefulness
    3. Perceived reputation (brand equity etc)
    4. Personalization (home page, products etc)
    5. Security (SSL etc)

    This is my first time on your site and I'm sure to be back!

    Thanks

    Alex Barton
  • Thank you for all the feedback.

    @Juan: The heatmap from crazyegg shows you all clicks. So if you have two "links" to the same place, those links will show up separately on the heatmap, so you can tell which one is more effective.

    @Rebecca: We definitely want to provide candidates with a great experience, and it makes sense that they would be a higher percentage of our traffic (since there are more candidates than companies) but our business is based on acquiring companies as customers - which is why the focus is there. It doesn't mean we ignore candidates, but I don't think the changes we'll make are going to emphasize / focus more on them.
  • Always! If you don't read what others have to say you will find yourself standing outside the crowded path, where people work and live. You need to listen the market , think of your goals and ways to get there otherwise you won't get anywhere.
  • Heatmaps look much better than google analytics and perform the same function. I've never heard of heatmaps before, I think I'll try it. Thanks for this informative post. I'll be back!
  • Great post.

    I'm glad you mentioned testing after launching a Web site and not just before while in test. I have made many more
    changes/improvements to my site after launching last week than I did in the several weeks I tested before going live. Testing will continue to be important as the site continues to grow.
  • @D. Hoch: Certainly testing internally beforehand makes sense, but you can never replace real live users and putting your feet to the fire with them.
  • I think that people forget many times that often a website with the most flash and nifty HTML can often be confusing and hard to navigate to a visitor. When I go to sites, I look for clean, simple, and un- complicated sites that are not hard to work my way around. This is really important for companies just starting a web site as well - they may think that a site with as much color and pizazz as possible will only get them much more attention - wrong! Also, Im glad you mentioned testing the web site - the only thing more frustrating than a badly designed web site is when you cant even navigate the site map. Thanks!
  • The idea of 'testing' a site design is probably one of the most overlooked aspects of ecommerce. Andy Jenkins over at StomperNet has repeated stated that the flow and design of a sales page or order page can severely cripple your sales. In other words, if you are not testing all aspects of the visual presentation of your site you are potentially losing a lot of money. And like was mentioned in the post, some of the better moves are not logical. For example, moving the 'hacker safe' type logos to the top improved sales. I would have avoided that because I personally would think the word 'hacker' would scare off customers but in his testing it helps with sales.
  • Of course its very neccessary to test a websites design.We do this since a long period to rise our visitors on the page.It works.
  • There is no doubt that testing in an early stage can save time and moneyan many nerves.
  • I agree on gathering feedback completely, but in my experience, it's not quite that easy.

    Does anyone have any good resources on the best way to gather unbiased feedback? In other words, people are using your site (not counting friends & family) and are forming opinions that will help you make your site better. How can you properly solicit this feedback from them?

    Any ideas?
  • @The College Review Guy: Good question. I think one way might be to pop up a survey and gather feedback. I believe there's a company called iPerceptions (http://www.iperceptions.com) that does this quite well.
  • Great tool - thanks for the input!
  • i use the heatmap in cnjunction with google analytics and the result is perfect. There are lot of information that a webmaster can understand from the map...and then with analytics he has the possibility to analize them deeply!
  • Ben,

    Great post. Thanks especially for the tips on HeatMaps and CrazyEgg.

    We find with our technology clients (and I'd bet it's true for any B2B company) that there are two reasons why websites commonly fail to deliver the desired results:

    1. They are developed without thorough consideration of the target customers, their business needs and their decision-making process.

    2. They are designed by people who are not marketers and who do not understand persuasion, communication and selling, and are not given any guidelines to follow.

    I'd be interested to hear what you and others think of this.

    Best,

    Susan
  • @Susan: I wound tend to agree on why sites fail. Sites that are not designed with the customer in mind, and driving that customer to take specific actions, will be in trouble.

    Thanks for the comment.
  • Wow.. I agree that feedback from other bloggers is really important. Is it okay if I ask your feedback about my blog Ben? It is new ;0
blog comments powered by Disqus
Co-Founder of Standout Jobs.
Entrepreneur and Opportunity Seeker!
About Me · Email Me · Twitter