List of questions to ask website usability testers
I talked about why usability testing is so important for website owners, and why it is also important to test those internal entry pages, and not strictly the homepage. But once you have your usability testers held captive, here are some questions you can ask them after their perusal through your money website.
You can either ask testers for written answers or just ask them verbally. You will likely get more honest replies with written answers, however, because of how much time this would take, you will need to severly cut back the number of questions you ask. And written answers would need to be done with a “while you wait” mentality because the longer your tester delays in writing them, the less accurate they will be, both in what they say and with how much they actually remember after the fact.
And of course, remain neutral. They won’t want to answer very thruthfully if you respond to one of their answers with an “Are you crazy?” or “How could you possible have thought that?”, no matter how well you know the person! Always remain neutral, listen to the answers carefully, and do your best not to lead your testers onto the answers you want? For example “Could you tell this site was for teenage girls?” would likely elicit a yes, even if the person thought it was for moms with six or more kids. Asking “who do you think the intended audience is?” would definitely get you the answers you were looking for.
And don’t talk too much tech or geek. If you use your mother-in-law as a tester, would she know what a “browser” is or know what a “javascript style menu” is? Take care that you don’t go too far over your tester’s heads with terminology that is familiar to those in the web industry, but definitely not many in the mainstream public.
After he or she is done surfing around, ask some questions…
Have you visited this site before?
Previous familiarization with a site can skew first impressions.
What do you think the purpose of this site is? (ie. selling, informing, entertainment, etc)
If they think it is a selling site, but it is actually a content information site, question what made them think the purpose was different than it really is.
Who do you think the intended audience is?
You know your targeted demographic, but perhaps they noticed clues that would leave them to believe a completely different audience was intended.
Could you find what you were looking for?
You want to know if everything was there the user expected or if there was something he or she thought they’d find but didn’t
Was it easy to get to the home page from the page you started on?
If the user expresses trouble getting to the home page, reassess your navigation structure or find out where they expected to find a link to the home page but didn’t.
Was there something missing you were expecting to see?
For prompting, you can ask about more text, more images, a FAQ, a question answered, etc.
Could you tell what the page was about?
If they sound confused, ask specifically what they thought it was about, and what those indicators were.
Was anything too obtrusive?
Particularly important if you use pop-up or pop-under ads, IntelliTXT type advertising products, an in-your-face style of ad placement, use of flash, etc.
Was anything too well hidden?
If you noticed he or she seemed to be hunting around the site, prompt with this.
Problems or kudos on the color scheme?
Too flashy? Too bland? Just right?
Easy to read (both font style and size)?
Was the font size too large or small? Was the chosen font difficult to read or in a color that made the text not as readable as it could have been?
How did you find the layout of the site?
Was everything organized well and set out as expected? If the layout is not a usual style, question that experience as well.
How intuitive and helpful is the navigation system?
If you are using any kind of javascript or floating menus, this question is crucial to find out if there were any problems from a visitor’s perspective. Oftentimes there are.
Did you notice… (advertising, newsletter signup, video, search box… etc)
Any other elements of the site or design you need to ask?
What would encourage you to return to this site in the future?
Was there something that could have been added to increase the return visitor rate?
Name your three favorite things about the site, and your three least favorite
This usually can bring up the unexpected things about your site that either endear people to it, or make them more likely to bounce quickly.
If you could change one thing on the site, whether it is major or minor, what would be at the top of the to do list?
You will get all kinds of responses, however it can help you gauge what people see as the most important things that hindered their experience. But if multiple people all say how annoying a single element is, such as the auto-play video you placed on the home page is, chances are good you should probably remove it the same night!
Obviously you can change these questions to suit your particular site. I will also be doing a new entry on usability testing specifically on ecommerce sites, to get you read for the holiday season. Look for that by the end of this month!
Note: Anyone contacting me (for consulting or anything else) through the Contact Form recently, please resend. I guess my funky Outlook spam filter thought anything with the “JenniferSlegg.com contact form” in the subject line was pretty darn suspicious and junked them all, even with the protection level set to low. Ah, the joys of Vista! And mini-rant: Why can’t I simply whitelist a particular subject line on the built-in spam fighter?
Subscribe to my RSS feedPress my buttons!
These icons link to social bookmarking sites where readers can share and discover new web pages.
September 24, 2007 at 7:46 am
Those are great questions, Jennifer. What is your opinion of using the 5 second test for usability testers? For those who are not familiar, a five second test is when you show a user a page for five seconds, take it away and ask them, “What does this page do?” Reason being: that’s about what users give a page, five seconds, before they decide that they are in the right place or not.
September 24, 2007 at 9:36 am
The 5-second test is very similar to the ideas set out in Blink by Malcolm Gladwell about how decisions about something can be made within the first few seconds of viewing. However, there are definitely some limitations.
I could look at a site for five seconds and say “Um, I think that’s some kind of sports site.” Meanwhile, another person could look at that same page and say “Oh, this is a fantasy football league site that tracks your team’s stats and points.” They are both valid first impressions, however if you went based on my impressions, you’d be making a lot of needless changes to a site that someone in the appropriate demographic and audience could nail right off the bat without any changes.
Now, put me in front of that same site for a proper usability site, I could probably figure out what it did, the intended audience and answer all kinds of usability questions, because many usability issues are applicable to all kinds of sites and markets.
I am not saying the five second test isn’t useful, it is for certain kinds of broadbased sites. But for more niche sites, you need to make sure you have the intended audience, which is much harder to do, unless you go into a sports bar on a Monday Night Football night and offer a free beer to anyone who takes the five second test for a fantasy football site
However, many other markets would be much harder.
September 24, 2007 at 9:52 am
That’s a great argument, Jennifer. Probably one of the best I’ve heard in the 5 second test debate. I think you are right, you have to decide if the 5 second test is appropriate based on the topic of the site. But if it is a site about something generic (i.e. ISPs), I think it can be very useful.