I have accepted recently to build a web robot which will search for certain data on certain specific websites. The client estimates that the maximal search will consist of on the order of 1800 searches apiece on 13 different websites. The web sites are databases and are intended to be searched by actual web surfers. My concern, however, is that most human surfers do not perform 1800 searches each separated by a few milliseconds. :) I asked the client about asking the 13 sites if they would mind but he is uncertain because if they do mind then what do we do? He had an idea to rotate through the 13 sites, performing 5 or 10 searches on each at a time. This would spread out the searches a bit. I still don't think this will look like a human user however. Our latest idea is to break the search into pieces and do a section at a time, separating the sections by a significant length of time, like at least an hour. Question 1: Does anyone have any thoughts on this? Do I need to be concerned that the sites will notice that they are being searched by a robot? Will they mind? They are not the sort of sites that would be popular among robots I don't think--they are somewhat industry-specific. Question 2: Does anyone know of a *reliable* host who actually has *good* customer support to host such a site? The client thinks he wants a Windows platform and we estimate that the traffic will be up to 5 Gigabytes a month, but the bulk of that will be download not upload. <tip type="New Clients" author="Hershel Robinson"> When dealing with a new client in a freelance relationship, always ask for some form of payment up front. It doesn't have to be before you actually begin to work and it doesn't have to be a huge amount, but it is important to get something paid early on. This insures the seriousness of and fiscal health of your new client before you waste a month (or two) of work. </tip> Thanks, Hershel PS: For those of you who remember my question a week or two ago about how to estimate a job, this is that big fish I wrote about and I hooked him for an excellent price, thanks to advice from evolters!