Advanced Bash-Scripting Guide
telnet
Utility and protocol for connecting to a remote host.
The telnet protocol contains security holes and should therefore probably be avoided.
Its use within a shell script is not recommended.
wget
The wget utility noninteractively retrieves or downloads files from a Web or ftp site. It works well in
a script.
wget -p http://www.xyz23.com/file01.html
# The -p or --page-requisite option causes wget to fetch all files
#+ required to display the specified page.
wget -r ftp://ftp.xyz24.net/~bozo/project_files/ -O $SAVEFILE
# The -r option recursively follows and retrieves all links
#+ on the specified site.
wget -c ftp://ftp.xyz25.net/bozofiles/filename.tar.bz2
# The -c option lets wget resume an interrupted download.
# This works with ftp servers and many HTTP sites.
Example 16-42. Getting a stock quote
#!/bin/bash
# quote-fetch.sh: Download a stock quote.
E_NOPARAMS=86
if [ -z "$1" ] # Must specify a stock (symbol) to fetch.
then echo "Usage: `basename $0` stock-symbol"
exit $E_NOPARAMS
fi
stock_symbol=$1
file_suffix=.html
# Fetches an HTML file, so name it appropriately.
URL='http://finance.yahoo.com/q?s='
# Yahoo finance board, with stock query suffix.
# ----------------------------------------------------------wget -O ${stock_symbol}${file_suffix} "${URL}${stock_symbol}"
# -----------------------------------------------------------
#
#
#
#
#
#
To look up stuff on http://search.yahoo.com:
----------------------------------------------------------URL="http://search.yahoo.com/search?fr=ush-news&p=${query}"
wget -O "$savefilename" "${URL}"
----------------------------------------------------------Saves a list of relevant URLs.
exit $?
# Exercises:
# --------#
# 1) Add a test to ensure the user running the script is on-line.
Chapter 16. External Filters, Programs and Commands
283
Advanced Bash-Scripting Guide
#
(Hint: parse the output of 'ps -ax' for "ppp" or "connect."
#
# 2) Modify this script to fetch the local weather report,
#+
taking the user's zip code as an argument.
See also Example A-30 and Example A-31.
lynx
The lynx Web and file browser can be used inside a script (with the -dump option) to retrieve a file
from a Web or ftp site noninteractively.
lynx -dump http://www.xyz23.com/file01.html >$SAVEFILE
With the -traversal option, lynx starts at the HTTP URL specified as an argument, then "crawls"
through all links located on that particular server. Used together with the -crawl option, outputs
page text to a log file.
rlogin
Remote login, initates a session on a remote host. This command has security issues, so use ssh
instead.
rsh
Remote shell, executes command(s) on a remote host. This has security issues, so use ssh
instead.
rcp
Remote copy, copies files between two different networked machines.
rsync
Remote synchronize, updates (synchronizes) files between two different networked machines.
bash$ rsync -a ~/sourcedir/*txt /node1/subdirectory/
Example 16-43. Updating FC4
#!/bin/bash
# fc4upd.sh
# Script author: Frank Wang.
# Slight stylistic modifications by ABS Guide author.
# Used in ABS Guide with permission.
#
#
#
#+
Download Fedora Core 4 update from mirror site using rsync.
Should also work for newer Fedora Cores -- 5, 6, . . .
Only download latest package if multiple versions exist,
to save space.
URL=rsync://distro.ibiblio.org/fedora-linux-core/updates/
# URL=rsync://ftp.kddilabs.jp/fedora/core/updates/
# URL=rsync://rsync.planetmirror.com/fedora-linux-core/updates/
DEST=${1:-/var/www/html/fedora/updates/}
LOG=/tmp/repo-update-$(/bin/date +%Y-%m-%d).txt
PID_FILE=/var/run/${0##*/}.pid
E_RETURN=85
# Something unexpected happened.
# General rsync options
# -r: recursive download
# -t: reserve time
Chapter 16. External Filters, Programs and Commands
284
Advanced Bash-Scripting Guide
# -v: verbose
OPTS="-rtv --delete-excluded --delete-after --partial"
# rsync include pattern
# Leading slash causes absolute path name match.
INCLUDE=(
"/4/i386/kde-i18n-Chinese*"
#
^
^
# Quoting is necessary to prevent globbing.
)
# rsync exclude pattern
# Temporarily comment out unwanted pkgs using "#" . . .
EXCLUDE=(
/1
/2
/3
/testing
/4/SRPMS
/4/ppc
/4/x86_64
/4/i386/debug
"/4/i386/kde-i18n-*"
"/4/i386/openoffice.org-langpack-*"
"/4/i386/*i586.rpm"
"/4/i386/GFS-*"
"/4/i386/cman-*"
"/4/i386/dlm-*"
"/4/i386/gnbd-*"
"/4/i386/kernel-smp*"
# "/4/i386/kernel-xen*"
# "/4/i386/xen-*"
)
init () {
# Let pipe command return possible rsync error, e.g., stalled network.
set -o pipefail
# Newly introduced in Bash, version 3.
TMP=${TMPDIR:-/tmp}/${0##*/}.$$
trap "{
rm -f $TMP 2>/dev/null
}" EXIT
# Store refined download list.
# Clear temporary file on exit.
}
check_pid () {
# Check if process exists.
if [ -s "$PID_FILE" ]; then
echo "PID file exists. Checking ..."
PID=$(/bin/egrep -o "^[[:digit:]]+" $PID_FILE)
if /bin/ps --pid $PID &>/dev/null; then
echo "Process $PID found. ${0##*/} seems to be running!"
/usr/bin/logger -t ${0##*/} \
"Process $PID found. ${0##*/} seems to be running!"
exit $E_RETURN
fi
echo "Process $PID not found. Start new process . . ."
fi
}
Chapter 16. External Filters, Programs and Commands
285
Advanced Bash-Scripting Guide
# Set overall file update range starting from root or $URL,
#+ according to above patterns.
set_range () {
include=
exclude=
for p in "${INCLUDE[@]}"; do
include="$include --include \"$p\""
done
for p in "${EXCLUDE[@]}"; do
exclude="$exclude --exclude \"$p\""
done
}
# Retrieve and refine rsync update list.
get_list () {
echo $$ > $PID_FILE || {
echo "Can't write to pid file $PID_FILE"
exit $E_RETURN
}
echo -n "Retrieving and refining update list . . ."
# Retrieve list -- 'eval' is needed to run rsync as a single command.
# $3 and $4 is the date and time of file creation.
# $5 is the full package name.
previous=
pre_file=
pre_date=0
eval /bin/nice /usr/bin/rsync \
-r $include $exclude $URL | \
egrep '^dr.x|^-r' | \
awk '{print $3, $4, $5}' | \
sort -k3 | \
{ while read line; do
# Get seconds since epoch, to filter out obsolete pkgs.
cur_date=$(date -d "$(echo $line | awk '{print $1, $2}')" +%s)
# echo $cur_date
# Get file name.
cur_file=$(echo $line | awk '{print $3}')
# echo $cur_file
# Get rpm pkg name from file name, if possible.
if [[ $cur_file == *rpm ]]; then
pkg_name=$(echo $cur_file | sed -r -e \
's/(^([^_-]+[_-])+)[[:digit:]]+\..*[_-].*$/\1/')
else
pkg_name=
fi
# echo $pkg_name
if [ -z "$pkg_name" ]; then
# If not a rpm file,
echo $cur_file >> $TMP
#+ then append to download list.
elif [ "$pkg_name" != "$previous" ]; then
# A new pkg found.
echo $pre_file >> $TMP
# Output latest file.
previous=$pkg_name
# Save current.
pre_date=$cur_date
pre_file=$cur_file
Chapter 16. External Filters, Programs and Commands
286