[thelist] Page Testing

[Joel D Canfield]

> I've heard of automatic testing applications and scripts before, but I
> have no clue how they're supposed to work - I just don't see how they
> could. Testing is about not only seeing how a web app will work under
> normal conditions, but also under "what happens if the user does...
> this!" conditions. Users are unpredictable - I just don't see how a
> computer can emulate that. But then again, maybe you're 
> talking about a different method of testing.

Exactly. There are two (at least) kinds of testing: regression testing
(do the changes we've made break existing functionality?) and exception
testing (are there holes where a user could break our toys if we don't
cover our backsides?)

Automated tools (I used to manage our install of Rational Robot) are for
regression testing primarily, but can include certain exception testing.
Load up data for expected and/or potential scenarios, and let the tool
do the grunt work, logging errors along the way.

For a smaller site, it's way overkill; we used it for a mind-bogglingly
large Java/browser app. But when I added tools to our intranet, I did
three levels of testing: 1) test it myself to make sure it worked as *I*
expected. 2) have experienced specialists in that department test it for
real-world use and known abuses ;) 3) have a group of target users test
it

I had the advantage of virtually always having expert users who were
telling *me* how the tools should work, and access to a QA team who
specialized in testing. But I guess the short version is that I always
tried to test with two groups: a carefully directed test ("do this, do
that, try this") and a random, free-form group who just banged away like
they would in real life.

joel