Perlfect Solutions
 

[Perlfect-search] indexer.pl getting killed

Robin Gould robin@rdg.inuk.com
Sun, 29 Oct 2000 18:37:58 -0000
Since nothing seemed to be working, I deleted all the files in the 'data'
directory and all worked OK. The question is, why are the data files now
only a quarter the size they were yet the search still works fine. What was
the excess data - was it important and should i be concerned?

Robin



> -----Original Message-----
> From: perlfect-search-admin@perlfect.com
> [mailto:perlfect-search-admin@perlfect.com]On Behalf Of Robin Gould
> Sent: 12 October 2000 19:19
> To: perlfect-search@perlfect.com
> Subject: RE: [Perlfect-search] indexer.pl getting killed
>
>
> Hi
>
> Thanks for that - I'm waiting to get Telnet access back to try that.
>
> This seems like a 'tweak' - at this level of indexing (400 pages)
> should the
> indexer not just breeze through it though - any idea how much memory it
> might use so that I can compare this with what my ISP is allowing?
>
> Robin
>
> > -----Original Message-----
> > From: perlfect-search-admin@perlfect.com
> > [mailto:perlfect-search-admin@perlfect.com]On Behalf Of giorgos
> > Sent: 12 October 2000 13:05
> > To: perlfect-search@perlfect.com
> > Subject: Re: [Perlfect-search] indexer.pl getting killed
> >
> >
> > robin,
> >
> > please try the following... there is a line in indexer.pl:
> >
> > * foreach my $term_id (keys %tf_db) {
> >
> > substitute the aboce with:
> >
> > * while (my $term_id = each %tf_db) {
> >
> > this should reduce the amount of memory used a bit... but i am not sure
> > if the problem lies there.
> >
> > let us know if this works for you.
> >
> > regards
> > giorgos
> >
> > Robin Gould wrote:
> > >
> > > Hi
> > >
> > > Thanks for that - made no difference I'm afraid though -
> > MINLENGTH was set
> > > at 3 and I really don't want to go any higher than that anyway.
> > >
> > > Is this a 'typical' problem with ISPs or the first time you've
> > heard of it -
> > > just trying to get a feel for the problem. There seem to tbe
> people who
> > > happily index >3,000 pages without problems, so I'm a bit
> confused why I
> > > should be having problems with <400.
> > >
> > > Robin
> > >
> > > > -----Original Message-----
> > > > From: perlfect-search-admin@perlfect.com
> > > > [mailto:perlfect-search-admin@perlfect.com]On Behalf Of Daniel Naber
> > > > Sent: 11 October 2000 23:06
> > > > To: perlfect-search@perlfect.com
> > > > Subject: Re: [Perlfect-search] indexer.pl getting killed
> > > >
> > > >
> > > > On Thursday 12 October 2000 00:03, you wrote:
> > > >
> > > > > (varies). According to them 'the problem is related to
> security and
> > > > > burden limits. It means that your script is either using an
> > very large
> > > > > amount of memory and/or processor time.' and is killed to
> > protect other
> > > > > users on the server.
> > > >
> > > > Depends on what you define as "very large amount". Basically,
> > > > building the
> > > > index for a full text requires quite some memory, don't think
> > you can do
> > > > ynthing about that.
> > > >
> > > > > I have stemming set to 0, number set to 0 - is there
> anything else I
> > > > > might try to overcome this problem?
> > > >
> > > > You could try to set STEMCHARS to 7 or so... and increase MINLENGTH.
> > > >
> > > > Regards
> > > >  Daniel
> > > > _______________________________________________
> > > > perlfect-search mailing list
> > > > perlfect-search@perlfect.com
> > > > To unsubscribe, set other personal options or view the list
> > > > archives please visit:
> > > > 
> > > > 
> > >
> > > _______________________________________________
> > > perlfect-search mailing list
> > > perlfect-search@perlfect.com
> > > To unsubscribe, set other personal options or view the list
> > archives please visit:
> > > 
> > > 
> >
> > --
> > Perlfect Solutions
> > http://perlfect.com
> > _______________________________________________
> > perlfect-search mailing list
> > perlfect-search@perlfect.com
> > To unsubscribe, set other personal options or view the list
> > archives please visit:
> > 
> > 
>
> _______________________________________________
> perlfect-search mailing list
> perlfect-search@perlfect.com
> To unsubscribe, set other personal options or view the list
> archives please visit:
> 
>