[thelist] RE: Stats - Was ( Do women view web pages differently from men?)

[Luther, Ron]

I agree it's been an interesting thread.

Personally, I would not be surprised to find gender, age, ethnic, national,
religeous, socio-economic, or brand-of-beer-preference differences ... or
differences for some sites anyway ... I suspect there may be a number of
'other' factors confounding specific web usage testing results.

But a couple of folks have raised issues on statistics ... how many people
do I need to ask for representative results ... is this list representative
... etc.

Got a definitive answer for you ..... "It Depends."

Do you need to defend that your site is "accessible" to some group of
individuals in a court of law against a lawsuit against you?  If you did I
might recommend one sample design to help you do that.  In this case the
size of the sample isn't nearly as important as being able to document (a)
that the sample is truely "random" (and therefore your results and
extrapolations are statistically valid), and (b) that your testing
methodology is "unbiased" and accurate.

Are you just curious if there is a gender-based difference to information
retrieval on your site?  If you are then I would possibly recommend an
'acceptance-sampling' design that would progressively increase in precision
to allow you to answer that question.


A couple quick things to mention and then I'll stop before I start
lecturing:

(1) "Statistical validity" isn't always worth the cost.  As has been pointed
out in the past, getting a handful of your friends to review a site can have
immense value to you as a designer .... without adhering to strict
statistical procedures.

(2) There is a technical difference between a "representative" sample and a
"random" sample.  They are not the same.

(3) 'Self-recruited' samples ... [like posting on evolt and asking for a
site review] ... are pretty much never "random" or "representative".   But I
think anyone on this list knows that they are VERY valuable.


Short answer - Statistical validity has it's place and can be a very
valuable tool ... but this does not mean that other methods do not have
value as well.


RonL (former stat guy)