[Perlfect-search] following redirects
Daniel Naber firstname.lastname@example.org
Thu, 15 Jan 2004 20:33:08 +0100
On Thursday 15 January 2004 13:13, BOCZ Peter wrote:
> Is it a feature that web crawler follows and indexes redirects to
> external URLs?
No, it's a bug. The last lines in the get_url() function in tools.pl have a
comment about that. The problem is that the check_accept_url() call should
be done again after this line, but it isn't:
$url = $response->base;
You could try that and report if it helps.