[Sunhelp] Search engine

[James Lockwood]

On Fri, 18 Jun 1999, Doug McLaren wrote:

> On Thu, Jun 17, 1999 at 09:14:36PM +0200, Marek G wrote:
> 
> | You may check glimpse & webglimpse at: http://webglimpse.net/
> 
> I second the glimpse reccomendation.  I'm using glimpse (but not
> webglimpse) to make a search engine that searches gigabytes of
> information and it works quite well and is quite speedy in most cases.

I've used glimpse extensively (usually in conjunction with Harvest to
search across web resources) and it definately has limitations with large
amounts of data.  After trying to beat it into shape to perform well on
100k+ document collections I tried using SWISH for a while.  This worked
better for some things (but has somewhat less functionality than glimpse)
but still was not an ideal fit.

I've finally settled on Isearch:

http://www.etymon.com/Isearch/

It's very fast, full featured and has very liberal licensing terms.

If anyone still uses Harvest for gathering web documents I've written a
SOIF input module for it as well, this turns it into quite a nice search
engine for medium-sized (under 10 million) web document collections.  If
anyone has come up with a better free replacement I'd love to hear about
it.

-James