[thelist] Spiders and things

Christian Heilmann codepo8 at gmail.com
Mon Jul 17 11:01:17 CDT 2006


> Hi all,
>
> We may be starting an ad campaign soon. I am looking for a product/service
> that will crawl the web and, for example, return all sites that have the
> words "law offices" on the title of the home page. I am certain that someone
> knows about this.
>
> Also, do you have any advice, warnings, or suggestions? I have never done
> this before.

You could use the API of Yahoo Search to get the information and write
it to a file with PHP:

http://developer.yahoo.com/search/web/V1/webSearch.html

example:
http://api.search.yahoo.com/WebSearchService/V1/webSearch?appid=YahooDemo&query=law+offices&results=100

As to writing an own spider, that is a lot trickier :-)

-- 
Chris Heilmann
Book: http://www.beginningjavascript.com
Blog: http://www.wait-till-i.com
Writing: http://icant.co.uk/



More information about the thelist mailing list