[thelist] survey says...

Adam Fahy mailme at myobligatory.com
Sat Dec 28 10:09:00 CST 2002


> Lachlan Cannon wrote:

> "50.
>
> Do use a "large" number of users (e.g., over 100) for performance tests,
> because of:..."
>
> All very well for them to say. Not all of us have that much money/time.
>
> "55.   Do ensure that important system (Web site) sounds are louder for
> older users."
>
> Or let them decide exactly how loud they need them.
>
> "56.  Do use TFT screens rather than CRT screens to elicit the fastest
> possible reading performance from users (older users benefit even more
> than younger users)."
>
> So we're somehow meant to reach out over the web and replace people's
> monitors? I wish someone would do that for mine.

Questions 42-53 were regarding usability testing; questions 54-56 were regarding the users /in usability tests/.

I tend to dislike articles written as a list of "DOs and DON'Ts," because--ironically in this case--the format is a design constraint which limits the writer's ability to explain each idea clearly, specifically ("don't do what donny don't does;" huh?).  It's a poor choice for the subject because a) it retards comprehension, and b) the idea of what is "usable" is rarely concrete enough to state without qualifiers: "depending on the site's purpose..." "for /most cases/, you should..." "users /tend to/ prefer..."  You can see this shining through in some of the contradictory "DOs" from the Web Site Design Issues section (whitespace vs fluidity vs screen resolution).

There were additional points given as a follow-up response to the article, past the list of references (everyone read the references, right?).  I'll reprint:


<<
The Ergonomic Pragmatist

Great list Bob. But let me share a few ideas and concerns.

1. I agree that 98% of browsers can show flash without a download. But without usability training 98% of developers will use flash in ways that makes the site worse.

2. Web applications are different. But for most cases do NOT try to mimic a windows applications. Users will expect to be in a browser and have the back button work.

3. Give it up Bob. It is too late to move the scroll bar to the left.

4. Be careful about making decisions just because the users said they preferred it. There may be hidden performance costs (as in #35 where redundant links will take space that could be used for useful content). Also, one of my favorite quotes of the year... "Because someone likes something doesn't mean it will move merchandise." Carolyn Burke - Sr. eCommerce Manager at Royal Bank

5. Obviously testing with 5 users was a bad idea. But 100s will be impractical for all but remote summative tests (which are much less valuable then formative tests early in the design process). Be practical and run formative tests with 12-30 subjects.

6. Automated testing does let you run lots of users in remote locations. It works well for summative testing. But for formative testing I recommend in person studies where you can watch users, notice their reactions, and talk freely about the design.
>>






More information about the thelist mailing list