Well another LEP Newsletter rolls into libraries and this is my 10th Newsletter article (my first was back in February 2001!).
Since the last Newsletter, work has continued on trialing a new search engine on the ABS web site. We are looking at implementing Lotus' Discovery Server (LDS). I expect implementation in the next few weeks.
The LDS search engine we are evaluating is a high-level discovery engine. The ABS will be using it for the ABS Intranet as well as the web site.
Currently the ABS uses an "out of the box" search engine (Domain Search) on the ABS web site, which, at the simplest level, uses "string searching" rather than the librarian's favourite - Boolean. The Domain Search "relevance ranking" algorithm also places too much emphasis on the frequency with which words occur and doesn't take account of "location on the page" and "off the page" attributes such as metadata.
In testing, LDS has obtained clearly better result sets than our existing Domain Search engine. The LDS search engine has a number of advantages over Domain Search. Its "simple search" uses Boolean by default and the search algorithm takes account of location, linking and click-through. Discovery server can also index and integrate content from different web sites - spidering content and automatically compiling a taxonomy which reflects the attributes of the content indexed. Consequently, it would be possible for the ABS to index and categorise the content of our own web site and then spider content of other statistical and related web sites, integrating this content into our own web site structure (taxonomy). For more information about Lotus' Discovery Server see: http://www.lotus.com/products/discserver.nsf, including a feature article on the ABS.
I'm not sure if you find my articles of interest so I would like to hear from you - please drop me an email about the sorts of things that you would like to hear about @ firstname.lastname@example.org