[thelist] download time stamp

Daniel J. Cody djc at starkmedia.com
Mon Mar 26 10:05:48 CST 2001


Hi Liz - 

Ok, here is a quicky little shell script using Awk that should get you
started..

First, lets assume you're using apache, and your log file looks
something like this:(i used this example from the evolt browser archive)

63.x.x.x - - [22/Mar/2001:13:19:33 -0600] "GET
/ie/32bit/3.02/winnt40full/msie302mnt.exe HTTP/1.1" 200 9089024
"http://browsers.evolt.org/index.cfm/dir/ie/32bit/3.02/winnt40full/"
"Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"

in this line from the logfile, you can see a couple things. first is the
IP address of the person that was downloading the file, the next two are
unused. the third, [22/Mar/2001:13:19:33 -0600]
is the date/time stamp that we want to get. next up is the HTTP method
they used, GET, then the URI and the version of HTTP they're using, then
the webserver status code - 200 - then the size of the file they
dowloaded. stuff after that we don't really care about for this
excersize follows.

So, if I'm reading you correctly you want to get the datetime of a
certain downloaded filetype(.doc in your case, .exe above) and lets get
the filesize just for fun.

So, to get those three things out of the long ass line above, run this
from the command line in the same directory as the logfile:

awk '/.exe/ { print $4; print $7; print $10 }' arch.access.log

what that line does is search for all occurances of .exe in the file
arch.access.log and it prints the 4th, 7th, and 10th fields from any
line that has .exe in it(so we dont get all the junk we dont want). Now
if you look at the line from the access log above, you can see that the
4th field is the date and time, the 7th(due to some wierd characters, it
doesn't quite look like the 7th field, but it is trust me :) is the file
that was downloaded /ie/32bit/3.02/winnt40full/msie302mnt.exe in this
case, and the 10th is the filesize, 9089024.

So what that script gives us is something like this:

[22/Mar/2001:13:19:33
/ie/32bit/3.02/winnt40full/msie302mnt.exe
9089024

So, that will get what you need.. All you should have to change is the
string you're looking for(probably .doc instead of .exe) and the file
name of the access_log. You may have to also get some different fields
based on what kind of logformat you use, but you should be able to
figure it out :)

So, now that I re-read that and your post again, it sounds like you want
something that will 'time' how fast people are downloading? If so, thats
a whole different case, but something we should be able to do. If its
not, good! Hope that helped out a bit :)

Shout if you have any more questions :)

.djc.


liz.hincks at tufts.edu wrote:
> 
> Thanks for clarifying what I need to think about.  This is for a "controlled"
> group of users in  a password protected environment...they all have the same
> tools/software and even the same laptop!! There is about 45 of them.  But, the
> problem is they are all over the world.  We are trying to find a quick and
> dirty way to get these users to take a "timed" test.  So, to answer you
> questions...they'll probably download a word document by http. ...
> 
> Feel free to give me any and all info on this script I will need to write on
> the access_log.
> thanks for you help,
> Liz
> 
> Quoting "Daniel J. Cody" <djc at starkmedia.com>:
> 
> > Hey Liz -
> >
> > Downloaded through what protocol? HTTP, FTP? If its HTTP, the best way
> > you're going to do it without 3rd party software is to run a bad-ass
> > script on the access_log from your website that parses for
> > 'downloadable' files like .exe, .zip, .tar, etc.. if it finds one, that
> > 'download' should have a timestamp. you could write a shell script that
> > does a check every 5 minutes for .exe files in the access_log and emails
> > you everything it finds(you could even through an AWK script in there to
> > only get the info you wanted, but thats another topic). if its FTP, you
> > should have some sort of ftp log, but it depends on what software..




More information about the thelist mailing list