[thechat] Memory or CPU?

[Luther, Ron]

Scott Dexter & Joel Canfield replied to my [kinda OT] question:


Hi Guys,

Thanks for the sanity check.  But it looks like it's gonna end up 
as a moot point ... <visions of Henry Ford /> ... I can have any 
laptop I want ... as long as its the "black one" that IT has decided 
to support this week.  ;-(

>>> data files on an ad-hoc analysis ... more memory or more CPU?

>Using what software??

Mostly a reporting package called 'Brio' pulling various bits out of 
an Oracle back-end. If I understand it correctly it likes to load 
big chunks of data into memory. [One really nice feature (for those 
of us second class citizens who don't have 'dba' or 'resource' level 
access) is that it lets you load an Excel or csv into memory and then 
'join' it to your db to run a query ... those often run me into memory 
or cycle limitations.]

>How large are the files?

Some of the base tables probably top 50M records. I don't pull it all 
mind you, but, (as below), I often wind up with full table scans ... 
and multiple table joins ... and multiple times a week I get both those 
ugly 'exceeded CPU usage - being logged off' messages, and the 'Windows 
low on memory - please close everything and buy a Mac' messages. (Usually 
after things have been running for an hour ... Grrrr!)

>How often? 

Depends on whats going on ... anywhere from 3x a week to 20x a day.

>(How ad-hoc?)

Good question! ... I usually need 'detail level' stuff (which sends me 
to the big-butt base tables) ... and I pretty much always need to search 
on non-indexed fields ... so I'm looking at full table scans on large 
tables -- to which I may need to 'right join' some other goodies. 
Basically just being a hog - trying to abuse the architecture to answer 
questions it wasn't designed to support ... and (not being in "IT" per 
se) not having the access to incrementally attack things by iteratively 
creating temp tables. It's a joy.

<<Wifey suggested spending a little more time playing with adding db 
directives to my oddball queries -- *cough*an excellent point, I should 
listen to her more*cough*!>>

[I guess I kinda put the equivalent of 'Darlington stripes' on any 
piece of computer gear I get close to.]   ;-)

Lot of odd 'data quality' queries ... If I 'black box' the data enough 
I can often *infer* questionable business practices or SAP processing 
glitches. Lot of 'smoke and mirrors' kinda stuff other folks just don't 
seem to think to look for. {We've migrated from 'snapshot' driven data 
to 'event' driven data ... IT still doesn't understand how critical 
business practices are to an 'event' driven datastream.}


>I'd tend to favor memory over raw CPU, but it depends on the type of
>processing. 

Yeah, talking through it helped ... the app I'm using is very memory 
intensive ... I need to push for the big memory.


Thanks!!

RonL.
(Playing 'catch-up' today after being out with a sinus infection last 
Friday.)

<anti-tip>
My wife printed the paperwork for me last night ... I think I'm gonna 
sign up to do 'docent' things at the Houston zoo!  ;-)  That will be 
fun!
</anti-tip>