AUTOMATIC FTP PROGRAMS FOR UNIX SYSTEMS
OVERVIEW
These automatic FTP programs for UNIX systems provide a nearly
effortless
way to transfer files from the public-domain archives on SIMTEL20 using
the
InterNet File Transfer Protocol, FTP. The principal "worker" in this
collection is the program GET20, a Bourne shell script written by Ferd
Brundick
of the U.S. Army Ballistic Research Laboratory. GET20 accepts inputs
from the
keyboard, or more conveniently from another shell script, and then calls
the
FTP program on the user's system and provides all needed inputs. Three
file
transfer modes are supported, ASCII, binary image, and binary in 8-bit
bytes.
SIMTEL20 is a 36-bit word-size PDP-20 running the TOPS-20 operating
system.
Therefore, only the ASCII and 8-bit-byte transfer modes will be useful
for
obtaining files from the public domain archives, as the data in these
files
must be unpacked from the 36-bit SIMTEL20 words and repacked for storage
in
16 or 32-bit UNIX words. The binary image transfer mode is provided only
for
special applications. GET20 can be (and has been) easily edited to allow
automatic retrieval of files from other machines that honor an anonymous
FTP
login. Once GET20 has been set in action, all aspects of the FTP process
happen automatically.
Archive files on SIMTEL20 are stored in two formats, ASCII (American
Standard Code for Information Interchange) and ITS binary (a binary
format that
originated at MIT). More information on the formats of the archive files
is
contained in "The SIMTEL20 Archives", available by request from the infocpm
list maintainer <info-cpm-request@amsaa>. The key factors are that ASCII
files
must be transferred with FTP in ASCII mode, and ITS binary files must be
transferred with FTP in a binary mode that causes the information in the 36bit
SIMTEL20 words to be properly stored in the 16 or 32-bit words of the
UNIX
machine. Thus, GET20 must be edited to conform with the FTP command
syntax for
the USER'S machine.
More on this later.
ITS binary files have at the beginning an identifier word that
appears as
the first four bytes in the received UNIX file. These bytes have been
added by
the file process, and they must be removed and discarded once such files
have
been transferred to a UNIX machine. Ferd Brundick has provided the
program
BEHEAD for this purpose. BEHEAD is written in the C language, and must
be compiled on the user's system. Once compiled, BEHEAD is called
automatically,
when appropriate, by GET20. BEHEAD checks the first four bytes of the
file it
is called to act upon, and will do nothing if these bytes are not the ITS
binary file header.
There are currently five archives on SIMTEL20:
MICRO:<CPM>
MICRO:<SIGM>
MICRO:<CPMUG>
MICRO:<UNIX>
MICRO:<PC-BLUE>
All files in <UNIX> are in ASCII. Some files in <CPM>, <SIGM>, <CPMUG>
and
<PC-BLUE> are in ASCII, while others are in ITS binary. The general
file-name
format for all archive files is:
MICRO:<ARCHIVE_NAME.DIRECTORY_NAME>PROGRAM_NAME
GET20 has the device-name MICRO: built-in, but the other three parts of
the
path-name must be supplied by the user. Thus, a typical command-line for
GET20
looks like this:
get20 -a sigm.vol007 james.bond
or alternately,
get20 -a sigm.vol007 james.bond new_name
The first form will transfer the file keeping the same name (in this
case,
james.bond), and the second form will give the transferred file a new
name on
the local system. If you give the command "get20" (with no arguments),
GET20
will display a usage statement.
The REAL convenience of GET20 comes from driving it with one-liner
shell
scripts that accept user input in VERY abbreviated form. For example,
the
one-liner "siga" , which obtains ASCII files from the <SIGM> archive,
contains:
get20 -a sigm.vol$*
To obtain the file of the previous example, a user need only type:
siga 007 james.bond
If a user wants to do frequent ASCII transfers from the <CPM.MODEM7>
directory, the one-liner "m7a" (or some such name) having the form:
get20 -a cpm.modem7 $*
can be used.
The user will then type only:
m7a mdm730.asm
The possibilities are endless.
SAMPLE SHELL SCRIPTS (ONE-LINERS)
The following one-liners are currently being used by the author to
call
GET20 for FTP transfers from the various SIMTEL20 archives:
Name
Contents
Files Transferred
-------------------------------------------------------------cpma
cpmb
siga
sigb
muga
mugb
pca
pcb
unix
get20
get20
get20
get20
get20
get20
get20
get20
get20
-a
-8
-a
-8
-a
-8
-a
-8
-a
cpm.$*
ASCII files from
cpm.$*
ITS binary files
sigm.vol$*
ASCII files from
sigm.vol$*
ITS binary files
cpmug.vol$*
ASCII files from
cpmug.vol$*
ITS binary files
pc-blue.vol$* ASCII files from
pc-blue.vol$* ITS binary files
unix.$* ALL files from <UNIX>
<CPM>
from <CPM>
<SIGM>
from <SIGM>
<CPMUG>
from <CPMUG>
<PC-BLUE>
from <PC-BLUE>
Note that the string "vol$*" contains a lower-case "L", NOT the digit
"one".
Example uses of the above one-liners follow:
cpma
cpma
cpmb
siga
modem7 mdm730.asm
modem7 mdm730.asm new.name
modem7 mdm730.com
145 vfiler.doc
mugb 040 compare.com
unix cpm crck.c
Note that, although only one example of a different local file-name is
shown
above, a new local file-name could have been specified for any of the
examples.
Each of the SIMTEL20 archives includes an ASCII file with a name
ending
in ".CRCLST". These files list the names, storage-modes, lengths and
cyclic
redundancy code (CRC) check-values for all files in the respective
archives.
They are updated whenever changes are made. The following shell scripts
can be
used to obtain these ".CRCLST" files:
get20
get20
get20
get20
get20
-a
-a
-a
-a
-a
cpm cpm.crclst
cpmug cpmug.crclst
pc-blue pc-blue.crclst
sigm sigm.crclst
unix unix.crclst
It is recommended that FULL PATH-NAMES be used in ALL shell scripts.
example, use something like:
For
/filesystem/account/.bin/get20 -a cpm.$*
instead of just
get20 -a cpm.$* .
This will allow your one-liners to be run from other accounts without the
caller having to keep copies of all your files. On the machine used by
the
author at work, the automatic FTP files are in a system directory, where
they
can be used by everyone.
INSTALLATION
Installation is largely a matter of personal taste. For example, in
the
author's case, a subdirectory called ".bin" holds a variety of such
utilities.
(It is called dot-bin so that it doesn't clutter directory listings
unless the
a-switch is specified with "ls".) This subdirectory is included in the
account's search path. Other than putting GET20 and the desired oneliner
shell scripts in a convenient directory and adjusting the search path to
find
them, the only things the installer must do are compile BEHEAD (the
header-byte
stripper for ITS binary files), and edit GET20 for use with the specific
target
system. Instructions for compilation are given starting at line 49 of
BEHEAD.C.
Instructions for editing GET20 are at several places in the script, so be
sure
to read it through completely and look for them. Most of the needed
changes
tailor GET20 for the particular FTP commands recognized by the target
system's
FTP server. If you need help with this, try doing "man ftp", or consult
your
friendly system wizard.
IN CASE OF DIFFICULTY
If you need help installing GET20 or BEHEAD, send the following
information
to Ferd Brundick <fsbrn@brl.arpa> with a copy to Dave Towson
<towson@brl.arpa>,
and we will try to assist you:
1. Manufacturer and model number of host machine.
2. Version of UNIX operating system in use.
3. A copy of the manual entry for the host system's FTP server (do
"man ftp > man.ftp", and send us the resulting file, "man.ftp").
4. Examples of command lines that don't work.
5. Any other enlightening symptoms or information.
Please do not ask for help with running your C-compiler; see your
friendly
system wizard for that (after first reading the manual, of course).
Dave Towson
25 June 1984