[thelist] Google fails to find sub-pages
John C Bullas
jcbullas at nildram.co.uk
Sun Feb 1 12:48:44 CST 2004
At 18:07 01/02/2004, you wrote
>Hello List!
>
>I wonder if anyone can shed some light on a problem that has been baffling
>me for weeks.
>
>I registered a simple little HTML site with search engines for a
>client. It's got index.htm and maybe 10 sub pages. The URL is
>www.studiostitches.com.au
>
>Google will find references only from the index page, but not from any of
>the other pages. Even if I use a specific site search it fails to find
>anything located on another page.
>
>I have even added each separate page URL to Google, but so far, to no avail.
no robots.txt, no simple hyperlinks to the sub pages, javascript menus, no
spidering = no Googling ....= answer
BRUTE FORCE you might try adding a "hidden" hyperlink to a list of
hyperlinks to all the pages you want finding
<meta name="ROBOTS" content="NOINDEX">
should be in the top of this "list" page so it doesn't get recorded itself
(assuming the spider tool adheres to the robots.txt standard)
does robots.txt do this function?
check out "spiders only this way" on my somewhat haphazard homepage of
several 10s of pages
http://www.fatblokeracing.org
link is bottom left
AND I know google uses it as I get the 404s from the Googlebot for the
pages I have deleted from the site since I last updated the spider file ;)
FB
More information about the thelist
mailing list