Modifications to Internet Gopher & WAIS for biology data files.

23Feb92, from d.g.gilbert (gilbert@bio.indiana.edu)

The complete file tree for the Gopher server, excluding the
large data and index files, is available thru ftp to
ftp.bio.indiana.edu, cd util/gopher/ as iubio-gopher.tar.Z
This includes the modified source for waisindex, gopherd,
and gindexd, the 3 programs needed to set this up.

See also the patch files for converting the original WAIS
and Gopher server sources to this biology version.  At the
above ftp site as util/gopher/gopher-bio.patch and as
util/wais/wais-bio.patch

Overview of Gopher server set-up:
	Compile programs waisindex, gopherd and gindexd.
	Set up public directory tree, along lines outlined by Gopher docs
	Set up data and indices folders as needed for questions
	Run waisindex on each data file to have questions
	  (if needed, add new file formats, primarily to ir/ircfiles.c)
	Install a gopherd server for each public, data and secondary
		directory tree that requires access (see example below).
	  Note that all folders under a gopherd tree are effectively open
		to public reading, even if not linked in to main public folder.
	Install a gindexd server for each index/question set.
	Install cron call to clean out tmplist files generated by gindex.
	Install /etc/rc.local (sunos) or equivalent call to start up gopher 
		and gindex daemons at system boot.
		
Source files:
  Unix gopher source, version 0.7, from ftp to boombox.micro.umn.edu
  WAIS source, version 8 b3, from ftp to think.com or boombox...
	
Files modified:
 gopher-src/server/
 		Makefile
		conf.h 
		globals.h
		gopherd.c
		special.c
		
 gopher-src/index-server/
		Makefile
		Waisindex.c	-- listfile additions, misc
		gindexd.c  	-- listfile additions, misc
		globals.h
		
 wais-src/ir/
 		hutil.h
		irbuild.c		-- new file formats
		ircfiles.c	-- new file formats for waisindex
		ircfiles.h 	-- new file formats
		irfiles.h 
		irsearch.c
		irtfiles.c	-- modified waisindexing 
		sersrch.c		-- NOT boolean patch for gindexd
 
 
Gopher server file tree

/bio/gopher/

2)	data/... 		-- non-ftp data files
			fly/			-- drosophila data
			genbank/ 	-- latest full release of genbank
			gbnew/		-- updates of genbank + embl
		
5)	indices/...	-- index files for gindexd
			fly/
			genbank/	
			gbnew/
			journal/
			materials/
			netnews/
			prosite/	
		
1)	public/...	-- file tree seen thru public gopher
			About.
			Drosophila/
			Molecular Biology/
			Software + Data Archive/
			Network News/
			Other Gophers/

	work/...  	-- miscellany private files

3) /bio/archive/...   -- ftp archive files
	
4) /usr/tmp/gopher/tmplist.*	-- temporary list files generated by gindexd


Server daemons installed as processes to handle the above file
structure:

1) gopherd -l /bio/gopher/public  70 
2) gopherd -l /bio/gopher/data  151
3) gopherd -l /bio/archive		150
4) gopherd -l /usr/tmp/gopher  152
5) gindexd for each index/question:
	gindexd -p 160 -l /bio/gopher/indices/fly
	gindexd -p 161 -l /bio/gopher/indices/genbank	
	gindexd -p 162 -l /bio/gopher/indices/gbnew	
	gindexd -p 163 -l /bio/gopher/indices/materials
	gindexd -p 164 -l /bio/gopher/indices/netnews
	gindexd -p 165 -l /bio/gopher/indices/prosite
	gindexd -p 166 -l /bio/gopher/indices/journals


--- install.gopher script ----
#!/bin/sh
# install.gopher
# restart the gopher servers
# this should be in, or called from, /etc/rc.local
# dgg, feb92

if [ -f /usr/local/etc/gopherd ]; then
        /usr/local/etc/gopherd -l /usr/tmp/gopher.log  /bio/gopher/public 70
        /usr/local/etc/gopherd -l /usr/tmp/go-data.log /bio/gopher/data 151
        /usr/local/etc/gopherd -l /usr/tmp/go-arch.log /bio/archive 150
        /usr/local/etc/gopherd -l /usr/tmp/go-list.log /usr/tmp/gopher 152
        if [ -f /usr/local/etc/gindexd ]; then
           /usr/local/etc/gindexd -p 160 -i /bio/gopher/indices/fly     -l /usr/tmp/gi-fly.log
           /usr/local/etc/gindexd -p 161 -i /bio/gopher/indices/genbank -l /usr/tmp/gi-gb.log
           /usr/local/etc/gindexd -p 162 -i /bio/gopher/indices/gbnew   -l /usr/tmp/gi-gnew.log
           /usr/local/etc/gindexd -p 163 -i /bio/gopher/indices/methods -l /usr/tmp/gi-mm.log
           /usr/local/etc/gindexd -p 164 -i /bio/gopher/indices/netnews -l /usr/tmp/gi-news.log
           /usr/local/etc/gindexd -p 165 -i /bio/gopher/indices/prosite -l /usr/tmp/gi-pro.log
           /usr/local/etc/gindexd -p 166 -i /bio/gopher/indices/journals -l /usr/tmp/gi-jrnl.log
        fi
        echo 'local: gopher Daemon for /bio/gopher started.'
fi


---- public folder tree --
about
fly/
molbio/
netnews/
other/
software/


---- indexing ----
#!/bin/csh
# index.all
#

set Indexer=/usr/local/bin/waisindex
set Indices=/bio/gopher/indices
set Data=/bio/gopher/data
set Archive=/bio/archive

cd $Indices/genbank; $Indexer -t genbank $Data/genbank/*.SEQ
cd $Indices/gbnew; $Indexer -t embl $Data/gbnew/emseq.all
cd $Indices/gbnew; $Indexer -a -t genbank $Data/gbnew/gbseq.all
cd $Indices/methods; $Indexer -t mail $Data/methods/*.text
cd $Indices/fly; $Indexer -t redbook $Data/fly/new.* 

cd $Indices/journals; $Indexer -t biojournal $Archive/molbio/journals/*.dat
cd $Indices/prosite; $Indexer -t prositedoc $Archive/molbio/data/prosite/prosite.doc
cd $Indices/prosite; $Indexer -a -t prositedat $Archive/molbio/data/prosite/prosite.dat
# need to fix usenet archive files 1st...
# cd $Indices/netnews; $Indexer -r -t mail $Archive/usenet/*.news