[thelist] SE spidering spectra URL's
Peter Van Dijck
peter at vardus.com
Thu Nov 23 07:57:13 CST 2000
Am I correct to think that SE's don't spider URL's with querystrings? And
if so, in many sites, if we want them to be spidered we need to give it
url's without querystrings? (like wired.com)
The approach we're taking is this:
we are building a Spectra site with lots of daily articles, e.g. there will
be URL's.
However, we make a daily copy of these articles into normal url's.
Then, we will submit a page with links to these url's to the se's.
That shold work.
Am I missing a better way, or anything???
Thanks guys!!
Peter (sick and speccing)
More information about the thelist
mailing list