Installing the Dienst FTP repositories package
This section of the Dienst installation instructions describes how to
install the FTP repositories feature, if you have customized the
Dienst document database by creating a unique custom.pl file. Enabling FTP access is an optional
feature of Dienst; you can run a Dienst server without this feature
In the following instructions, all directory names,
except when noted otherwise, are implicitly relative to the dienst
directory that you created when you downloaded the Dienst server.
Overview of FTP repositories
A basic Dienst server accesses documents from a document database. This is an
organized directory sub-tree on the file system accessible to the
Dienst server by standard Unix file I/O. The FTP repository package
allows you to store non-paged
document formats in an anonymous FTP repository, and provide access to
these through your Dienst server. This access is transparent to the
user of the Dienst server. The FTP repository package is useful for
- you may already provide access to documents from an anonymous FTP
- you may wish to provide access to documents both from a Dienst
server and an anonymous FTP site.
- you may wish to set up a single central Dienst server that acts
as the indexer for a number of distributed FTP repositories and a
gateway for access to documents in those repositories.
Prerequisites for this installation
In order to install the FTP repository feature, you must have previously
installed a basic Dienst system.
In addition, you will need some working knowledge
of perl to
complete this installation. Finally, the
.h C header files to
perl header files) must be available on your system. This command is
part of the standard perl software distribution.
Overview of this installation
This installation includes four steps:
- Declaring storage formats for FTP repositories.
- Enabling FTP repositories in Dienst.
- Creating perl header files for the FTP package.
- Setting variables for special handling of FTP connections
(i.e., for firewalls).
Detailed installation instructions
To install the FTP repository package, please complete the following
Declaring storage formats for FTP repositories
You can declare an FTP-resident storage format in the same manner as a
normal storage format except for the following arguments:
WARNING: Defining storage formats for anonymous FTP repositories may
slow down the response of your Dienst server. Dienst determines
the formats in which an individual document is available by looping
through each of the defined storage formats. This check in the
document database is reasonably fast (since it just involves a local
file system check). It can be slow for anonymous FTP sites. We
highly recommend that you only use anonymous FTP repositories for no more
than one or two storage formats. Also, make sure to add a
naming authority argument to the format definition if you use
anonymous FTP repositories for only specific naming authorities in your
- pattern - a sprintf string, encoding the FTP URL for this
kind of data. For documentation on sprintf, see the Unix man
page. The string may contain arguments, the value for these arguments comes from the variables argument to def_format. The value produced by
sprintf is a URL to a file available via anonymous FTP.
As described under using compressed files, the pattern argument in your
def_format storage declarations should always identify
the file name without its compression suffix.
- naming authority -
an optional string that specifies a specific naming authority to which this
storage format applies. You can omit this argument if your
site indexes and stores documents from one naming authority, or if the
storage formats for all naming authorities are the same. If Dienst does not
find a storage format defined specifically for the naming authority of a document, it
will default to the declaration defined without a specific naming authority
argument. Generally, you will only define storage formats that are
specific for a naming authority when using an anonymous
An example: Assume that your site manages documents from two
naming authorities, PUB1 and PUB2. Assume that PostScript
files for PUB1 are stored in
ftp://ftp.pub1.edu/reports/%s.ps and PostScript files for
PUB2 are stored at ftp://ftp.pub2.edu/reports/%s.ps
(where %s is a sprintf pattern which will be filled
by the name of the document).
In the basic installation you must have already setup a
Dienst server with a document database that contains bibliographic
files for these naming authorities.
To make the FTP files for these documents
available to your Dienst server, make the two following
format definitions in Config/custom.pl.
(See complete variable listing in document to Define your own document database.)
&def_format ("postscript_ftp", "application/postscript", 0,
&def_format ("postscript_ftp", "application/postscript", 0,
NOTE:The suffixes defined by the compression_schemes setting in install.config are
also used for file name matching in the FTP repository.
Next, check that the following two variables are correctly set in Config/config_constants.pl:
- $ftp_enable - set to 1(true)
- $ftp_timeout - set to an integer that specifies the time in seconds that the Dienst server should wait before aborting FTP connnection attempts. At Cornell, we set this at 20.
Creating perl header files for the FTP package
The perl code for handling FTP connections requires to system dependent
These files may already be available in your system;
check with your
system administrator if you are unsure. If they are not available,
you will need to run the
h2ph command to convert the
.h files to these files (try
usr/include/syscall.h and usr/include/sys/socket.h).
To do this,
cd to the
UI_Server directory. Then, run the
cat <path>/socket.h | h2ph > socket.ph
cat <path>/syscall.h | h2ph > syscall.ph
Where <path> is the full path to the respective
At the completion of these commands the two files
syscall.ph should be present
Activating these changes in Dienst
Run the command
to register these new setting with your Dienst server.
Special handling of FTP connections
Most sites open connections to external FTP sites with an
open to the host name of the ftp site, the user
and then use an e-mail address as the password. This is the default in the
Dienst FTP repository package (the setting of the
$maintainer variable is used as the password). If your
site follows this standard procedure you can ignore the rest of this
If you have a firewall, then it will take some extra finagling to get
communication going back and forth. As an example,
at Cornell, we have a firewall that requires a non-standard method for
opening FTP connections. The three following variables must be set in
Config/config_constants and Utilities/bin/reload-code rerun in order to change the manner in which
Dienst opens an FTP connection. Each of these variables is
evaled before use (so you may put variable names in their
values, which will be interpreted before use). The variable
$host is passed as in argument into the
subroutine; this is the host address of the FTP repository. You do
NOT have to set these variables if your site opens FTP
connections in the standard manner.
$ftp_host - a string which will be
evaled before use as the host to which an FTP connection
will be established.
$ftp_userid - a string which will be
evaled before use as the userid for an FTP connection.
$ftp_password> - a string which will be
evaled before use as the password for an FTP connection.
Up to Main Information Menu
Any comments or questions?
Contact us at firstname.lastname@example.org.