ST Alpha - "Just from the URLs I suspect most of the items on the old WH.gov robot list were just either search forms or text-only versions of pages." (Reading a Post)

SlickerTalk
Search Archives
The Leaderboard
The FAQ

Down to Post

Backboards:

Posts: 153

Just from the URLs I suspect most of the items on the old WH.gov robot list were just either search forms or text-only versions of pages.

Posted by Ender (aka Ender) Jan 21 '09, 07:17

The thought process being that those weren't actual site content, just forms and printer-friendly copies of other content that's already indexable.

Post a message top