CSc 352
Shell Scripts
Saumya Debray
Dept. of Computer Science
The University of Arizona, Tucson
debray@cs.arizona.edu
What is a shell script?
• A shell script is a list of commands to be run by a
shell
– basically a program, but using shell commands instead of C
or Java statements
• Why?
– automate repetitious tasks
• e.g.: testing a program on a large set of test inputs
– package up commonly executed command sequences
– create our own commands
2
Creating and executing a shell script
1. Create a file, say “foo”, containing the commands to
be executed by the script
2. Execute the script:
–
invoke the appropriate shell with foo as argument, e.g.:
• bash foo
• csh foo
– chmod a+x foo
./foo
3
A simple example
• a shell script to print “hello world!”
the “echo” command,
as executed from the
command line
contents of the shell
script file
“hello_world.script”
this script behaves the
same for a variety of
different shells
with appropriate
permissions, the script
can be executed
directly
4
Syntax issues across shells
Different shells have slightly different syntax rules
– the same script may not work on all shells:
script contents
different behaviors
5
Which shell runs my script?
my current shell is csh
the script executes as
expected when csh is
invoked explicitly
if not invoked explicitly,
the default shell /bin/sh
is used
6
Solution: specify shell in the script
#!pathToShell on first line
specifies which shell to use
executes as expected when
invoked in csh
executes as expected when
invoked in bash
7
Specifying the shell interpreter
#!/bin/csh
• ‘#!’ at the beginning of a file indicates that it is a
script
• The rest of the first line specifies (absolute) path to
the interpreter
• What if the interpreter is at a different location on a
different system?
– use /usr/bin/env to improve portability:
#!/usr/bin/env csh
8
Variables
• Variables different than in C or Java:
– don’t have to be declared in advance
– they are untyped: the same variable can hold an integer
value or a string
• Syntax for using variables (bash):
– Defining the value of a variable X:
X=value
– Using the variable X:
$X or ${X}
9
Example
variable
definitions
variable uses
undefined variable
evaluates to empty string
10
Guarding against uninitialized variables
-u : “exit if using an
unitialized variable”
-u :
• works if we specify /bin/bash
• but not with /usr/bin/env
11
Guarding against uninitialized variables
12
Bash scripts: whitespace
without whitespace 
the command works as
expected
with whitespace 
• weird error message
• variable value not set
as expected
13
Different kinds of quotes
back quotes:
evaluates to the
output of the
enclosed
commands
single quotes: variables
not expanded
double quotes: variables
inside the quotes are
expanded
14
A simple scripting problem
• Problem:
– Iterate over the files in a given directory:
– for each file, express its size in human-readable form (i.e.,
using KB and MB where appropriate):
• give 2 decimal places for MB sizes; 1 decimal place for KB sizes
• functionality similar to using “ls –lh”
• Requires:
–
–
–
–
list files in a directory [valid directory?  if-then-else]
iterate over these files
obtain the size of each file
perform arithmetic
[KB/MB decision  if-then-else]
15
The test command
• Used to guide control flow in conditionals and loops
• Syntax:
test operand1 operator operand2
test operand
– often abbreviated to:
[ operand1 operator operand2 ]
[ operand ]
– see “man test” for a list of operators
• Returns a nonzero status if the test is true; zero o/w
16
Example: test
17
Example: if-then-else
if-then-else command
18
Example: if-then-else
need a ‘;’ between
the test expression
and ‘then’
19
Example: if-then-else
20
Alternative formulation of test
21
For Loops
• Syntax 1:
for X in word ; do
cmds
done
Convenient for iterating
over lists
• Syntax 2:
for (( e1; e2; e3 )) ; do
cmds
done
Convenient for iteration
involving arithmetic
expressions
22
For Loops: Example 1
23
For Loops: Example 2
24
Iterating over lists: pitfalls
print out the first couple of
lines of a file listing
!
25
Iterating over lists
use quotes to prevent
splitting at whitespace
Note: the loop iterates
only once – not once
per file!
26
Arithmetic
arithmetic evaluation doesn’t
happen automatically; must
be specified explicitly
get file size (see “man stat”
for other file properties)
27
Arithmetic
integer arithmetic only!
arithmetic evaluation:
$(( expr ))
28
Arithmetic
get bc to do the floating point math for us
29
Command-line arguments
• Arguments at known positions, e.g., 1st, 2nd, ..., can
be accessed using $1, $2, …
• $0 = the name of the script file
• Total no. of arguments: $#
• However, we can’t get to the ith argument this way
– $i  value of variable i
30
Example
31
Iterating over the argument list
32
Getting at the entire list of arguments
33
Iterating over the argument list
Placement of quotes matters:
all of the arguments are seen
as a single word
34
Iterating over the argument list
Placement of quotes matters:
35
Arguments with whitespace
• $@ is similar to $*, but each argument is a quoted
string
– no interpretation or expansion
– allows us to properly access arguments that may contain
whitespace
– needs to be quoted “…”
36
Arguments with whitespace
37
Debugging
38
Debugging: -v
prints shell input lines
as they are read
39
Debugging: -v
these are the lines from
the script that are being
executed
Note that variable
values etc. are not
being shown
40
Debugging: -x
prints out command
traces before they are
executed
41
Debugging: -x
variable values,
command arguments
are printed out
42
Debugging: summary
• Debugging can be done by observing what
commends get executed
– “set –v”
• prints out shell commands as they are read in
• does not print out runtime variable values
– “set –x”
• prints out a trace of the command before it is actually executed
• prints out variable and argument values
• Debugging can be turned off using “set +v”, “set +x”
– useful for localizing which parts of the script are observed
43
String manipulation
• shell scripts often involve string manipulation
– e.g., to pattern match on file names, create new files
based on existing files, etc.
• bash gives extensive facilities for string matching and
manipulation
– see “man expr”
44
A sample problem
Write a bash script to convert Unix make files to
“mymake” make files
unix make
a : b.o c.o d.o
gcc *.o –o a
b.o : b.c b.h
gcc –Wall –c b.c
mymake

@target a : b.o c.o d.o
@cmd gcc *.o –o a
@target b.o : b.c b.h
@cmd gcc –Wall –c b.c
45
Attempt 1
• Ignore definitions
– assume make file does not have stuff like “CC = gcc”
• Assume
– user has added whitespace where necessary
• e.g.: on either side of ‘ : ’ in rules
– all dependencies have been made explicit
• Issues:
– need to process a line at a time
• lines may contain whitespace
– need to check whether a line starts with a ‘\t’
46
Attempt 1
47
Attempt 1
IFS = “Internal Field Separator”
used by bash to split the input
into words
the new definition of IFS means
that the input is now split at
newlines
identify lines that begin with a
tab
48
Attempt 1
49
Attempt 2
• relax assumption about whitespace around :
– want:
• input: “a.o: a.c b.h c.h”
• output: “@target a.o : a.c b.h c.h”
• Issue:
– replace ‘:’ by ‘ : ’
• bash offers two replacement operators:
– ${str/substr/replacement}
• replaces the first occurrence of substr in str with replacement
– ${str//substr/replacement}
• replaces all occurrences of substr in str with replacement
50
Attempt 2
51
Attempt 2
52
Attempt 2
Only the first ‘:’ had
whitespace inserted
53
Attempt 2
“replace all occurrences”
54
Attempt 2
55
Attempt 3
• Relax assumption about definitions
– allow “CC=gcc”, “CFLAGS= –Wall –g –c”
– assume no whitespace around ‘=‘
• Issues:
– identify definitions, i.e., lines that contain a ‘=‘
– extract LHS, RHS of the definition
– replace every occurrence of LHS by RHS
56
Attempt 3
if the line contains a ‘=‘ …
figure out where it occurs
LHS = substring from
position 0 to position of ‘=‘
RHS = substring after
position of ‘=‘ to end
substitute RHS for every
occurrence of LHS
57
Attempt 3
58
Attempt 3
Oops!
59
Attempt 3
60
Attempt 3
61