[thelist] Response.Write versus varHTML = varHTML & "..."

Fri Jan 11 01:52:37 CST 2002

I've just had a massive paradigm shift. One of those moments where the 
whole world just flips upside down.

Over the years I've developed a style of coding in ASP that I had come to 
rely on and helped me tackle problems. That is until I found out today that 
a major style of coding I was using is significantly "flawed". My 
philosophy had been put the branching logic at the top of the page, the 
html in the middle (with drop-ins for variables and 'chunks' of HTML) and 
the functions called by the branching logic at the bottom of the page (or 
if necessary in a shared library). This way I could calculate up front and 
then scatter bomb the HTML as necessary with the data collected or mangled.

e.g.

[%
  IF THEN END IF
  IF THEN END IF
  SELECT CASE blah
	CASE One
		ChunkofHTML = FunctionOne()
	CASE Two
		ChunkofHTML = FunctionTwo()
	CASE Three
		ChunkofHTML = FunctionThree()
	CASE ELSE Four
		ChunkofHTML = FunctionFour()
  END SELECT
  IF THEN END IF
%]
[HTML]
	..blah blah blah...
	[%VariableHere%]
	..blah blah blah...
	[%=ChunkofHTML %]
	..blah blah blah...
	[%VariableThere%]
	..blah blah blah...
[/HTML]
[%
  FunctionOne()
	FunctionOne = ..blah blah blah...
  End Function

  FunctionTwo()
	FunctionTwo = ..blah blah blah...
  End Function

  FunctionThree()
	FunctionThree = ..blah blah blah...
  End Function

  FunctionFour()
	FunctionFour = ..blah blah blah...
  End Function
%]

As I said this process worked great for me, until today. We are designing a 
nifty little tool that can (at times) pull up quite a large number of rows 
of data to be thrown to the screen. The above system works great when there 
are about 50-75 rows of data. However the page becomes unbearably slow 
(long processing time server side, watching with Task Manager and the 
server bouncing above 80% most of the time) when the number of rows gets 
over 100-150.

We did quite a bit of testing and it was not our DB connection and queries. 
We have a semi-optimized our queries. We have semi-optimized the database 
with indexes and data manglement such that a combination of 11 separate 
queries (each with several joins) pulling a cumulative of 1K rows of data 
from 2.2 Million records only takes about 5 seconds. (There is probably 
room for improvement here but this showed us that the DB and queries were 
not the bottleneck, for now.)

Plus Task/Performance Manager showed that DLLHost was consuming most of the 
CPU versus SQLServer.

So we looked at the ASP code itself.

To my horror, the following lines (and their numerous kin) proved to be the 
performance killer:

   Top of Some Loop
	subPageHTML = subPageHTML & "...blah blah blah"
	subPageHTML = subPageHTML & "...blah blah blah"
	subPageHTML = subPageHTML & "...blah blah blah"
   Bottom of Some Loop

It didn't make any difference how I pulled the data from the DB, either 
iterating over the rows with the Record Set open, or pulling the rows via 
getRows and iterating over the returned array. Timing data inserted into 
the script showed that over an 8 second period the above process proceeded 
to slow to a crawl very quickly.

e.g. First Second - Processed 115 Lines
      Second Second - Processed 87 Lines
      Third Second - Processed 55 Lines
      Fourth Second - Processed 27 Lines
      Fifth Second - Processed 17 Lines
      Sixth Second - Processed 17 Lines
      Seventh Second - Processed 17 Lines
      Eighth Second - Processed 17 Lines

Granted there was some over head processing due to the Timing Data, but 
this reflected very well the performance slow down we were seeing with real 
data as the rows returned increased.

It was obvious that the subPageHTML = subPageHTML & "" lines were killing 
performance. So I proceeded to hack up my ASP page and move code around. I 
had to push 90% of the logic/branching code to the "center" of the page in 
order to output the data in the correct location (replacing the Functions 
with Subs). If I was to replace the subPageHTML = subPageHTML & "" process 
with response.write & response.flush, then I needed to get the lines of 
code that called the former functions now subroutings to the location where 
I wanted the output.

So now I had this:

[%
  IF THEN END IF
  IF THEN END IF
%]
[HTML]
	..blah blah blah...
	[%VariableHere%]
	..blah blah blah...
[%
  SELECT CASE blah
	CASE One
		SubOne()
	CASE Two
		SubTwo()
	CASE Three
		SubThree()
	CASE ELSE Four
		SubFour()
  END SELECT
  IF THEN END IF
%]
	..blah blah blah...
	[%VariableThere%]
	..blah blah blah...
[/HTML]
[%
  SubOne()
	Response.Write ..blah blah blah...
	Response.Flush
  End Sub

  SubTwo()
	Response.Write ..blah blah blah...
	Response.Flush
  End Sub

  SubThree()
	Response.Write ..blah blah blah...
	Response.Flush
  End Sub

  SubFour()
	Response.Write ..blah blah blah...
	Response.Flush
  End Sub
%]

This of course completely threw off variables within the HTML that expected 
to have data higher up in the page. And creates a quandary; How do you 
organize the HTML and code when you are forced to throw data to the browser 
when it's created?

e.g. You want to toss a Message to the user near the top of the page of how 
many rows returned. But the row counter and HTML created from the DB data 
are now being calculated and outputted "after" that area of the page has 
already been sent off to the browser. It forces you to go to the DB more 
often than you were previously planning and break up functions that IMvHO 
should be related blocks of code since they share the same source of data 
or related logic. Granted, based upon the speed increase this extra trip 
does not subtract from the performance greatly - but it feels so unclean.

I even tried SpeedString ( 
http://www.intesoft.co.uk/speedstring/default.asp ) from InteSoft, but did 
not see the performance increase that we saw by moving to Response.Writes. 
With the above changes the processing time dropped to 2 seconds.

<digress>In fact, Netscape 6.2 appears to have a huge rendering problem 
with a set of three nested tables. The rendering time with Netscape is 
much, much longer than the time to calculate and send the data to the 
browser. The first two are framing for the third. The third table is the 
biggest, holding all the rows of data being returned. IE 5.x and 6.x have 
no problem rendering the output. Go figure. (I'll be testing with the 
latest Mozilla release tomorrow.)</digress>

So tomorrow I tackle my now fractured ASP world. As much as I hate to work 
with VBS (the ultimate brain dead language), this really really irks me.

For those that have read this far and are not laughing their arses off at 
my woes - any perls of wisdom outside of "drop ASP for solution XYZ"?
---
Anthony Baratta
President
Keyboard Jockeys

"Conformity is the refuge of the unimaginative."