[Perlfect-search] following redirectsBOCZ Peter perlfect-search@perlfect.comThu, 15 Jan 2004 13:13:23 +0100
Hi Is it a feature that web crawler follows and indexes redirects to external URLs? My conf.pl allows the startup site to be indexed only, but due to redirections to external sites, they get indexed (while indexing external content correclty stops here). I wish it could be configured by the administrator and search page shows the source URL, not the destination. Peter
|
| Copyright Perlfect Solutions 1998-2008 - Privacy Policy |