One of the challenges of early product design is creating the initial product hypothesis on what the product actually is. Internally, products tend to be more easily described as a bundle of features, but it’s difficult to convert that into an actual description. Marketing and initial product design is often the realm in which everyone has an opinion, and it’s hard to judge which one is more valid. Should this button be red or blue? Is this the right message for the user, or should we phrase it another way? The answers to the questions, however, can be very important and fundamental decisions that have profound impact on the product. As Jason Fried says, copywriting is interface design.
The natural questions that follows is, how can we take the diverse opinions and create a constructive process with which to answer them? It’s important to be able to evaluate ideas and make decisions to break the stalemate:
#1. Implement a test
One way to answer these questions is simply by testing it. Set up a landing page or bucket a certain group of users into a test group, and measure the results against the key metrics that you’re trying to drive. Doing very small variations high up on the funnel can have great results.
Pros/Cons
With this approach, it can be tempting to design by numbers. However, if you’re not getting the qualitative feedback directly from the users, you may be missing important data about the decision. For example, a correlative increase in click-through might actually indicate that your copy is misleading rather than better converting. Another consideration for this approach is how hard running the test is. Even though ideally everyone should be able to set up a test, this can create an additional hurdle for non-technical team members to advance their ideas.
Tools
If you’re focused on marketing copy or testing a simple value proposition, one free solution to this is Google Website Optimizer which lets you test completely different pages or do multivariate tests of multiple page elements. Optimizely is another useful tool geared toward providing non-technical ways to test landing pages. If you’re looking for deeper analysis like funnels, there are many tools like Kissmetrics, Mixpanel and Kontagent available.
#2. Conduct live user testing
A second approach is to speak directly to potential customers and show them the interface or ask them questions to test the different hypotheses. A version of this might range from from conducting a survey, posting on your community forums, screencasting with Skype, or all the way to visiting someone at their home or place of work.
Pros/Cons
The benefits of live usertesting is that it allows you to unearth information and responses through interaction with users. Watching and engaging with users can be insightful. However, generating live usertests can be quite expensive in terms of time or money to determine the answers to qualitative questions. In addition, you run into the problem of sample bias. Depending on the nature of your business, the user sets which you receive may not necessarily be representative of your audience.
Tools
There’s some simple ways to put this into practice. The first version is taking your customer email list and emailing them. A similar option is posting a Craigslist ad that leads to a Wufoo survey form. Present a quick message about your product and what type of user you’d like to see respond (with a time commitment), and then push them into a Wufoo survey which asks qualifying questions to determine whether or not they’re a candidate for you. After that, you’ve got a list of potential customers to interview in-person or on the phone.
The second version that I use and recommend is Usertesting.com. They provide a great service where you select the type of user you’d like to interview, provide a list of tasks and essay questions, and connect you with the user for a small fee. In exchange, you receive a short essay answers to these questions and a video screencast of your user completing the tasks and verbally answering the questions in their native browser environment. This is super important for consumer web because you can see the experience of your product on their browser in their home environment. The benefit of this approach is that it’s incredibly easy. However, the drawback is that it’s a less interactive audience and these users are potentially less representative of your actual userbase.
#3. Create a customer advocate group
The third way that I’d suggest answering these questions is by creating a customer advocate group. Ideally, this is a group which is composed of the top customers or potential customers of your product which everyone in the company agrees is the real customer. For a B2B company with 20% of the customers providing 80% of the revenue, this would be the council of your 20%. For a consumer company, this may be some of your highest ARPU users or top posters.
Pros/Cons
Creating a customer advocate group can have the benefit of removing the problem of the vocal minority. These are the engaged users who may complain the loudest, but aren’t necessarily representative of your top users. It can also be a great way to establish and cultivate great relationships with your top customers.
The idea of a customer advocate group is particularly powerful because it dispels the argument that “This isn’t representative of our customer base” when the answers surfaced aren’t convenient to the product direction. By creating this group, you have a very targeted and accessible set of users to run these qualitative questions through. Once this group is established, it should be pretty easy to get them on the phone, email them a few paragraphs, or even throw together a few Balsamiq mockups to show them. I think creating this is the preferred and ideal method, but it takes more work on the front end to set up.
One potential downside to this method is that customer advocate groups do need to be maintained. You should constantly be asking, Is this group currently representative of the customer we are targeting? As your product or customer group evolves, it’s important to regularly revisit this question and redefine or augment the group.
Tools
If you have an existing customer base, find a way to get to the set of top emails that you agree as a team represent your customers to contact them. A great way to do this is to ask for an interview or even create a more formal program for them to be involved. If you don’t have an existing customer list, scour your network, Google Blogsearch or Twitter and look for the most knowledgable and potentially representative group of users for your product.
5 Ways NOT To Make Product Design Decisions
I’ll close by thinking through five ways not to make product design decisions.
- Don’t copy what competitor X is doing, but make the effort to understand what or why they’re doing it.
- Don’t make decisions in a silo. Have conversations with your customers and have a clear reason why you’re doing it.
- Beware of the loudest voice in the room, because the strongest opinion might not be the right one.
- Avoid creating “maybe useful” product features.
- Don’t forget to talk to your support or community staff. Pay attention to those who are closest to the action.
Thanks for reading! 🙂