[thelist] (no subject)

Techwatcher techwatcher at accesswriters.com
Fri May 24 09:35:05 CDT 2002


Hi All, and especially Jonathan_A_McPherson at rl.gov, this bright morning:

Re "flat" files:

I would just like to point out that APL or J (and any variants) file
systems would refer to all systems in which the database (or data) is
not nestable (or cannot be "enclosed") -- which means, in essence, you
must worry about the LENGTH of a variable -- as "flat." Back when I was
forced to convert a db I'd built in APL into SQL, your kind of db
packages were all "flat" to an APL'er.

If you "nest" a variable (within what you might think of as a cell,
line, column, or 2-d array, or whatever), it becomes a variable with a
size of 1. (I can't even correctly say "a length of 1," since in APL
length implies at at least 2 dimensions!)

One of the advantages of this, of course, is that your actual data can
take up exactly as much room as it needs, and no more. You don't have
to allocate to all fields the length the largest value will need. Does
MySQL, SQL, or any other DB Package have this feature yet? I might
consider (for the sake of conformity) switching to one if it did. Until
then, I'll keep my development & manipulation within APL (or J), and
convert the files (disclosing each element, building a "flat" file as
APL or J think of it) to read them into a standard db (MySQL included
with my Web host account) only when I have to.

Cheers --
Carol Stein
techwatcher at accesswriters.com

Databases management systems (DBMS's -- Oracle, SQL, MySQL, Postgres,
etc)
have certain features that you can't get with a flat file or XML. When
those
features become important, you use a DBMS. When they are not important,
you
can probably downgrade. Some of the obvious reasons to use a DBMS:
* Speed. Modern DBMS's build indices and have very advanced routines for
getting you data as quickly as possible.
* Management of relational entities. DBMS's are good at storing data
about
multiple, related entities.
* Transaction and liveness guarantees. DBMS's allow you to protect your
data
-- back it up, be able to roll back unwanted transactions, ensure that
a set
of operations on data executes atomically, etc.

I'm sure I'm missing a few important ones -- those were just the ones
that
came to mind. Others on the list will supply additions, I'm sure. (-:

For your quote file, is speed a problem when you use a flat file? Do you
have relationships between quotes you need to manage? Do you need a very
reliable transaction-based system for maintaining your quotes?

Ask yourself what important (for *your* app) features a DBMS is getting
you
that a flat file or XML wouldn't. Then ask yourself what it's costing
you
(in terms of maintaining the DBMS software, added code for your app,
etc).
Then ask yourself if those costs are worth the benefits you are getting.
That's a decision only you are qualified to make.

I shouldn't lump "flat file" and "XML" together -- to me, a "flat file"
implies storage of entities of a single type, whereas "XML" is a
structured
format that stores multiple types of entities and their hierarchical
relationship. In those contexts, XML can do what flat files cannot, but
it
is still best thought of as another kind of file format.

FWIW, I almost always use a DBMS, because I almost always find that it
does
something I need to do that I wouldn't be able to do as well on my own -
-
but that might not be the case for your app. DBMS's are wonderful
tools, but
they are not always necessary and sometimes can add needless complexity
to
your application.




More information about the thelist mailing list