I could look at a site for five seconds and say “Um, I think that’s some kind of sports site.” Meanwhile, another person could look at that same page and say “Oh, this is a fantasy football league site that tracks your team’s stats and points.” They are both valid first impressions, however if you went based on my impressions, you’d be making a lot of needless changes to a site that someone in the appropriate demographic and audience could nail right off the bat without any changes.

Now, put me in front of that same site for a proper usability site, I could probably figure out what it did, the intended audience and answer all kinds of usability questions, because many usability issues are applicable to all kinds of sites and markets.

I am not saying the five second test isn’t useful, it is for certain kinds of broadbased sites. But for more niche sites, you need to make sure you have the intended audience, which is much harder to do, unless you go into a sports bar on a Monday Night Football night and offer a free beer to anyone who takes the five second test for a fantasy football site However, many other markets would be much harder.