Perlfect Solutions
 

[Perlfect-search] following redirects

BOCZ Peter perlfect-search@perlfect.com
Thu, 15 Jan 2004 13:13:23 +0100
Hi

Is it a feature that web crawler follows and indexes redirects to external
URLs? My conf.pl allows the startup site to be indexed only, but due to
redirections to external sites, they get indexed (while indexing external
content correclty stops here).
I wish it could be configured by the administrator and search page shows the
source URL, not the destination.

Peter