Installing WAIS is suggested for large databases such as the regional index servers that may store more than 10000 records and serve requests for an entire region.
The default Dienst search engine keeps all indexes in memory so for large collections on heavily loaded systems this may cause problems with limited memory resources, especially when Dienst forks children to handle search requests. Increasing swap space to at least 400MBytes will help get rid of "server fork failer" and "no swap space" messages for the short term; using the WAIS search engine is another way to resolve this problem.
It is suggested that you get the system working with the built-in search engine before installing the WAIS search engine. You might find the Dienst indexer works fine for your collection.
The steps are:
Dienst relies on freeWAIS-sf returning a header that contains the docid, title, date and all authors. The length of this headline tends to be greater than the default 100 character limit freeWAIS imposes so you will need to download the latest version of freeWAIS-sf and compile it.
You may ftp the release freeWAIS-sf-2.1.2 from Cornell's ftp server (ftp.cs.cornell.edu) from the directory /pub/Dienst.
To unzip and unpack the tar file, use the command:
gunzip -cd freeWAIS-sf-2.1.2.tar.gz | tar xvf -
There is further documentation on freeWAIS-sf, including installation instructions, at:
FreeWAIS-sf Home Page
When you run the freeWAIS-sf configure script to create configuration constants, it will ask a series of questions. You may take the default answers for nearly everything. You should take the following options:
And here's something you'll need to change in freeWAIS-sf before you compile with the make command: At the top level the is a file named defaults.h. Change HEADLINE from 100 to 300 before compiling freeWAIS-sf-2.1.2 -- this will address the character limit mentioned above.
Note that you do not need to create WAIS indexes or start up the wais server -- this will be done for you by the Dienst auto_config.pl script after you change some settings in install.config.
A special non-interactive version of the 'waissearch' program is necessary for Dienst to query the waisserver. You must compile this program and place the binary file in the Utilities/bin directory. A Makefile is provided in the Utilities/src/link_waissearch directory. The makefile will install the binary file if you type 'make install'. You will need to modify the Makefile paths to include files and libraries to reflect where you compiled wais.
You may need to go into the freeWAIS/regexp directory and run the make command in order to generate the libregexp.a file before you can get link_waissearch to compile.
You will want to test the link_waissearch program (with Dienst running) before you reinstall Dienst with its new configuration. The options to link_waissearch are <host> <port> <database> <query>. You might want to take a look at one of the bib records in the <wais_bibs_directory> to help you formulate a query.
Ex. link_waissearch cstr.cs.cornell.edu 3000 cstr 'au=lagoze and ti=protocol and pb=01'
A successful query will return the docid, title, date, and author for each hit.
Edit 'install.config' and change the setting <wais_enabled> to 1. The install.config file contains detailed documentation for each WAIS setting. There are several settings you need to look over, like the location of your WAIS binaries, indexes, bib files, etc.
Stop your Dienst server, if it is running. You can do a ps -e to find the process id, and then use the kill command:
ps -eaf | grep dienst
Now (re-)run auto_config.pl.
Auto_config.pl will automatically index all of your bib records if you have enabled WAIS in the install.config file and you have your bib records in a single file. Auto_config.pl will automatically call the 'Indexer/build-wais-indexes.pl' program.
You may also run the 'Indexer/build-wais-indexes.pl' program manually.
When indexing is complete, the build-wais-indexes.pl will attempt to start the waisserver program if the <wais_auto_start> option is set (this is the default).
Auto_config.pl will automatically attempt to start the 'waisserver' program. You should verify that the server is running properly after auto_config runs:
ps -eaf | grep waisserver
If you are not running auto_config.pl then you will need to start the waisserver manually.
Remember to start the 'waisserver' when your system is rebooted, just as you do the Dienst server.
At this point your Dienst server is configured to use WAIS-sf as its search engine. Once you have completed the installation of your WWW server you may search the system using the simple or fielded searches.
If no hits are returned for a query you expect hits for make sure the link_waissearch program is working properly. You may also use the 'waissearch' program to search the 'waisserver' directly.