On Thursday 15 January 2004 13:13, BOCZ Peter wrote:

> Is it a feature that web crawler follows and indexes redirects to
> external URLs?

No, it's a bug. The last lines in the get_url() function in tools.pl have a 
comment about that. The problem is that the check_accept_url() call should 
be done again after this line, but it isn't:
$url = $response->base;

You could try that and report if it helps.

Regards
 Daniel

-- 
http://www.danielnaber.de