Perlfect Solutions
 

[Perlfect-search] indexer.pl getting killed

Robin Gould robin@rdg.inuk.com
Thu, 12 Oct 2000 19:18:54 +0100
Hi

Thanks for that - I'm waiting to get Telnet access back to try that.

This seems like a 'tweak' - at this level of indexing (400 pages) should the
indexer not just breeze through it though - any idea how much memory it
might use so that I can compare this with what my ISP is allowing?

Robin

> -----Original Message-----
> From: perlfect-search-admin@perlfect.com
> [mailto:perlfect-search-admin@perlfect.com]On Behalf Of giorgos
> Sent: 12 October 2000 13:05
> To: perlfect-search@perlfect.com
> Subject: Re: [Perlfect-search] indexer.pl getting killed
>
>
> robin,
>
> please try the following... there is a line in indexer.pl:
>
> * foreach my $term_id (keys %tf_db) {
>
> substitute the aboce with:
>
> * while (my $term_id = each %tf_db) {
>
> this should reduce the amount of memory used a bit... but i am not sure
> if the problem lies there.
>
> let us know if this works for you.
>
> regards
> giorgos
>
> Robin Gould wrote:
> >
> > Hi
> >
> > Thanks for that - made no difference I'm afraid though -
> MINLENGTH was set
> > at 3 and I really don't want to go any higher than that anyway.
> >
> > Is this a 'typical' problem with ISPs or the first time you've
> heard of it -
> > just trying to get a feel for the problem. There seem to tbe people who
> > happily index >3,000 pages without problems, so I'm a bit confused why I
> > should be having problems with <400.
> >
> > Robin
> >
> > > -----Original Message-----
> > > From: perlfect-search-admin@perlfect.com
> > > [mailto:perlfect-search-admin@perlfect.com]On Behalf Of Daniel Naber
> > > Sent: 11 October 2000 23:06
> > > To: perlfect-search@perlfect.com
> > > Subject: Re: [Perlfect-search] indexer.pl getting killed
> > >
> > >
> > > On Thursday 12 October 2000 00:03, you wrote:
> > >
> > > > (varies). According to them 'the problem is related to security and
> > > > burden limits. It means that your script is either using an
> very large
> > > > amount of memory and/or processor time.' and is killed to
> protect other
> > > > users on the server.
> > >
> > > Depends on what you define as "very large amount". Basically,
> > > building the
> > > index for a full text requires quite some memory, don't think
> you can do
> > > ynthing about that.
> > >
> > > > I have stemming set to 0, number set to 0 - is there anything else I
> > > > might try to overcome this problem?
> > >
> > > You could try to set STEMCHARS to 7 or so... and increase MINLENGTH.
> > >
> > > Regards
> > >  Daniel
> > > _______________________________________________
> > > perlfect-search mailing list
> > > perlfect-search@perlfect.com
> > > To unsubscribe, set other personal options or view the list
> > > archives please visit:
> > > 
> > > 
> >
> > _______________________________________________
> > perlfect-search mailing list
> > perlfect-search@perlfect.com
> > To unsubscribe, set other personal options or view the list
> archives please visit:
> > 
> > 
>
> --
> Perlfect Solutions
> http://perlfect.com
> _______________________________________________
> perlfect-search mailing list
> perlfect-search@perlfect.com
> To unsubscribe, set other personal options or view the list
> archives please visit:
> 
>