CPS 393
Introduction to Unix and C
START OF WEEK 5 (UNIX)
4/13/2015
Course material created by D.
Woit
1
Testing & Debugging Shell Pgms
• use shell parameters -x or -v
•
bash -v pgm
to debug interactively
– displays every line of pgm just before executing it
•
bash -x pgm
– like -v but substitutes values of variables in each
• Before we proceed let us recall:
• `command(s)` means that command(s) will be executed in a
sub-shell and resulting value will be put in place of
command(s)
4/13/2015
Course material created by D.
Woit
2
Testing cont.
• Example:
•
•
•
•
•
•
•
•
•
•
•
•
•
#!/bin/bash
#Source: findtrunc
truncates input to 8 characters &
#looks it up in file gfile
# a msg is printed to indicate if found/not found
# use first method on linux, second on unix (why? bug in linux bash-# the read variable only valid within a subshell
# that runs each part of a pipe in a different process, so vars do not
# get into the process running the "read“
# third method always works, but no help if want to use read to separate
#by whitespace
# fourth method is valid on both linux/unix is to put a ( before the read
# and a ) at the end of the program to keep everything IN the subshell
4/13/2015
Course material created by D.
Woit
3
Testing cont. code for
findtrunc
• #first method:
•
echo "$1" | cut -c1-8 >tmp
•
read trunc <tmp
•
#second method:
•
echo "$1" | cut -c1-8 | read trunc
•
#third method
•
trunc=`echo "$1" | cut -c1-8` #no spaces between trunc and =
•
if [ "`grep $trunc gfile`" ]
•
then echo Found
•
else echo not found
•
fi
4/13/2015
Course material created by D.
Woit
4
Testing continued, code for
•
try to run:
findtrunc
findtrunc abcdefghijkl
– prints Found #if abcdefgh in gfile
•
try to run:
findtrunc
– no output, just hangs there , then press ^C , ^D
to stop it
• prints not found
•
What's the problem with findtrunc? Why does it hang?
•
Use -x to find out why:
4/13/2015
Course material created by D.
Woit
5
Testing continued, code for
•
•
•
bash -x findtrunc
+ echo ''
+ cut -c1-8
•
•
•
•
•
•
•
•
•
findtrunc
+ read trunc
++ grep gfile
<---- hangs, wanting to grep string gfile
from stdin. If we type ^D signals
end of stdin, giving...
+'[' '' ']'
<---- since grep printed nothing on stdout
+ echo not found <---- since [ ] (null string) is considered false
not found
+ test -e tmp
+ rm tmp
<---- terminates
4/13/2015
Course material created by D.
Woit
6
Testing continued, code for
findtrunc
•
with bash –e can tell where hanging
•
•
In this case, we forgot the argument to findtrunc, so it was null
we can fix this by adding an :
– enclosing if [ $1 ] (if $1 not null)
– or something similar, such as if [ $# -ge 1 ]
• Note: Could have used ( ) to solve problem of reading from pipe in above
program, as in:
• echo "$1" | cut -c1-8 | ( read trunc
•
if [ "`grep $trunc gfile`" ]
•
then echo Found
•
else echo not found
•
fi
•
)
• Recall ( commands ) means that they will be executed in sub-shell
4/13/2015
Course material created by D.
Woit
7
Shift
• Shift command is used in while loops to process arguments
• #!/bin/bash
• #Source: shiftex
• #pgm to print its args.
•
while [ "$1" ]
#<--- stops when $1= "" (null)
•
do
•
echo $1
•
shift
#<--- moves $2 to $1, $3 to $2 etc.
•
done
•
exit 0
4/13/2015
Course material created by D.
Woit
8
Xargs
•
used to perform cmd repetitively on a group of things from
stdin but can be used after redirection
• find . -name "*" | xargs grep bash
– #prints LINES in files * that contain the string bash
•
find . -type d -empty | xargs rmdir
– #remove empty directories from tree rooted in current dir
•
ls | xargs -I{} cp {} {}.bak
– makes backup copy of all files in current dir. (no spaces between {} please)
– if xx is a dir get cp xx xx.bak -- error, but will just print msg for each dir
and continue on with files
• e.g., what does this print on stdout?
• (echo a b c ; echo d e ; echo f) | xargs -I{} echo :{}:
4/13/2015
Course material created by D.
Woit
9
HMWK
• 1. write a shell program called clean that removes all files
called "core" from your directory structure, and also removes
all files that have size 0 (0 bytes) from your dir structure.
That's your WHOLE dir structure, not just the current dir.
• (note: "find" can find files that are zero bytes long)
4/13/2015
Course material created by D.
Woit
10
HMWK
• 1.write a shell program called avgs which will read lines from
•
•
•
•
•
•
•
•
•
•
stdin such as the following (where the dots indicate more lines
of the same kind:
SID
LNAME I T1/20 T2/30
92876035 WANG S 15 26
95908659 CHIANG R 10 29
.
.
. .
..
.
. .
.
91234987 MYRTH R 15 16
Your shell program will print out a line such as:
Average of T1/20 is 17 and of T2/30 is 24
– where 17 and 24 are the averages of the last 2 columns respectively. Note
that your program must keep a total and count for each of the last 2
columns, and must not include data from the first line in the totals and
counts.
4/13/2015
Course material created by D.
Woit
11
HMWK
• 2. Modify script avgs so that the "title" line, e.g.,
•
SID
LNAME I T1/20 T2/30
above, could be located at ANY LINE of stdin (ie., not necessarily the FIRST
line, as above.)
• 3. Modify avgs so that the highest and lowest marks are printed for each of
T1 and T2, as well as averages.
•
Your program should produce output such as the following:
•
T1/20: Average: 17 Hightest: 19 Lowest: 1
•
T2/30: Average: 24 Hightest: 29 Lowest: 9
4/13/2015
Course material created by D.
Woit
12
HMWK
• 4. Modify avgs so that the output above is in PERCENT, e.g.,
•
•
T1/20: Average: 85% Hightest: 95% Lowest: 5%
T2/30: Average: 80% Hightest: 96% Lowest: 3%
•
If variable x contains "T1/20" you could use an echo piped to
•
•
•
•
•
•
a cut, and assign the result to another variable, or you
could use ${x#*/} which will match pattern "*/" against the
beginning of the contents of x, delete the shortest part that
matches, and return the rest. Thus typeset -i y=${x#*/} would
assign 20 to y. Note that pattern */ means anything followed by
a "/"
4/13/2015
Course material created by D.
Woit
13
HMWK
• 5. Write a shell program called findext which will list directories under a
given path that contain files/dirs having the given extensions.
•
Program findext takes at least 2 command line arguments. Its usage is:
•
findext path ext1 ext2 ... where "path" is the path to the start of the
directory structure you wish to search, and ext1, ext2, etc are extensions.
•
Program findext will print out the directories under path which contain
files/dirs that end in .ext1 .ext2 etc
•
Your program does not need to work properly if the user passes glob
constructs or special chars as arguments.
•
For example:
•
findext /home/dwoit/courses c scm
•
will list all directories under /home/dwoit/courses that contain files/dirs
named *.c and/or *.scm
•
4/13/2015
Course material created by D.
Woit
14
HMWK 5 cont.
•
•
•
•
•
•
•
•
•
•
•
If findext is sent less than 2 command line args, it should print on stderr
Usage: findext path arg1 arg2
and exit with exitcode 1
(the word findext above should be printed using $0)
If it is passed 2 or more arguments, it should print a list of dirs in which
files/dirs of the form *.ext1 *.ext2 etc reside. Then it should exit with
exitcode 0.
Note that only the DIRECTORY names are printed. Therefore you will
have to chop off the trailing file/dir name (the part after the last "/")
You can do this with ${var%/*} which says to match pattern "/*" (slash
followed by anything) at the END of variable var; and delete the shortest
part that matches and return the rest. You will also find utilities
sort and uniq useful.
4/13/2015
Course material created by D.
Woit
15
Built in commands
• We saw already: echo, exit, pwd, shopt, break, cd, set, test,
read, typeset, etc.
• Some others : eval
•
v1="cmd1 | cmd2"
#make a string "cmd1 | cmd2"
•
v2="cmd3 | cmd4" #make a string
•
eval "$v1 | cmd5 | $v2" #is really:
•
cmd1 | cmd2 | cmd5 | cmd3 | cmd 4 #pipes
• However if we typed:
• $v1 | cmd5 | $v2
– # error: the "|" in v1 not interpreted as pipe
4/13/2015
Course material created by D.
Woit
16
Eval cont.
• Example for eval:
• v1="cat table | grep 2"
•
v2="grep 0 | more "
•
$v1 | $v2
#some errors like:
–
grep: |: No such file or directory
–
grep: more: No such file or directory
• Correct invocation is: eval “$v1 | $v2”
– Output is: 9
•
10
11
12
eval has 2 passes:
– 1) makes substitutions
– 2) result evaluated i.e, meta chars etc. interpreted properly
4/13/2015
Course material created by D.
Woit
17
Eval example
•
•
•
•
•
•
•
•
•
•
#!/bin/bash
#Source: printXth
#Expects a bunch of command line arguments.
#Prompts user and reads an integer (call it X)
#then it prints the Xth command line argument
echo -n "enter an integer: "
read X
the_arg='$'${X}
eval echo "command line arg number ${X} is: ${the_arg}"
echo "command line arg number ${X} is: ${the_arg}"
– #note the diff
4/13/2015
Course material created by D.
Woit
18
Eval example
•
•
•
•
•
•
•
•
•
•
#!/bin/bash
#Source: for11
#one way to make command line arg "abc de" work:
typeset -i i=1
while [ $i -le $# ]
do
a='$'$i
eval "echo $a" # or b=`eval "echo $a"` ; echo $b #see for12
i=i+1
done
4/13/2015
Course material created by D.
Woit
19
HMWK
• Use eval to write a shell program called revargs which will
print out its command line args in reverse order.
• Write a shell program that doesn't use eval to print its clas in
• reverse order.
4/13/2015
Course material created by D.
Woit
20
Shell commands
•
most shell cmds (i.e. not built in ones) in "bins"
•
•
•
•
/bin, /usr/bin , /usr/local/bin , even ~/bin
shell searches directories to find cmds.
searches those in "PATH"
to see path
echo $PATH
•
•
•
•
•
•
•
•
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/courses/bin/i686::
to add (or change) default path, reset PATH in .profile
PATH=${PATH}:${HOME}/bin
export PATH
- Adds /home/dwoit/bin to end of current path (for me)
(I put all shell my personal pgms in /home/dwoit/bin)
-export makes PATH global
(so all subsequent processes can use it.)
4/13/2015
Course material created by D.
Woit
21
Problem with path
•
Problem occurs if we have two different commands with
same name,
– Shell will execute first one it finds in path
• i.e. if you wrote a "rm" cmd to use instead of /bin/rm
– you must put your bin /dwoit/bin before /bin in PATH, or /bin/rm
will be always executed.
•
(or can make it a function or alias, which are always done
first)
4/13/2015
Course material created by D.
Woit
22
Functions
•
•
•
•
•
Can use functions to modularize shell programs
Can also use them in shell as you use any shell program
Often placed in a .profile
Syntax: function_name ( ) { command_list; }
For example: if we are tired of doing "ls -ld" so make a
function called ll
•
ll () {
•
ls -ld
•
}
• Can also enter interactively in shell:
•
/home/dwoit> ll () {
•
Linux> ls -ld
•
Linux> }
4/13/2015
Course material created by D.
Woit
23
Arguments for functions
• Just like for shell programs: $1 $2, ... $* etc
• Functions in shell programs
• In a shell program called xy
•
•
•
•
•
•
•
•
•
#!/bin/bash
#Source: xy
abc () {
echo "in function abc. My \$2 is: $2"
}
echo in xy...calling abc x "y z" w
abc x "y z" w
echo in xy...calling abc a b c d
abc a b c d
4/13/2015
Course material created by D.
Woit
24
Like complex "alias"
• Functions are done before shell commands
• Thus, to make "vi" different make a function (in .profile)
• vi () {
• for i
# each parameter to vi which can file or options +• do
•
x=`echo $i | cut -c1`
•
if [ "$x" != "-" -a "$x" != "+" ]
•
then
•
cp $i $i.bak
•
fi
• done
• /usr/bin/vi "$@“
# “$@” is equivalent to “$1” “$2” …
• }
4/13/2015
Course material created by D.
Woit
25
Functions cont.
• To define "ls" to always be "ls -l" could do alias, or
•
ls () {
•
/usr/bin/ls -l
•
}
– But this ls always shows whole contents of the directory.
– There is no option to show only selected files and/or directories
• Also NOTE1: do not do:
•
ls () {
•
ls -l
•
}
• Why? It is a *recursive* call to the newly-defined ls *function*
•
(Infinite loop)
4/13/2015
Course material created by D.
Woit
26
Functions cont.
• NOTE2: in ls() function above we should really do:
• /usr/bin/ls -l ${1+"$@"}
• instead of: /usr/bin/ls -l
• Why? the latter ALWAYS does "ls -l" even if you supply
arguments
• e.g., ls x y z
• will NOT do ls -l ONLY for x y z, it does it for *all* files
• The ${1+"$@"} means: if $1 is unset or null replace
${1+"$@"} with null;
• otherwise replace ${1+"$@"} with the expansion of $@
• (which are all the function's args)
4/13/2015
Course material created by D.
Woit
27
Determining if something is a function or not
•
•
•
•
•
•
•
•
•
•
•
type function_name
e.g. once we defined ll()
/home/dwoit> type ll
ll is a function
ll ()
{
ls -ld
}
/home/dwoit> unset ll
/home/dwoit> type ll
-bash: type: ll: not found
4/13/2015
• for previous example ls
• /home/dwoit> type ls
• ls is a function
• ls ()
• {
•
/bin/ls ${1+"$@"}
• }
•
• /home/dwoit> unset ls
• /home/dwoit> type ls
• ls is hashed (/bin/ls)
Course material created by D.
Woit
28
Determining what functions are defined
•
typeset -f
• #lists all functions defined this session
• << Deleting a function >>
• unset -f function_name #unset f unsets *variable* f, not
function f
• #however, if no variable defined then unset f will unset
function f
• << Function scope>>
• Functions executed in current shell, not subshell (as normal
shell pgm)
• Thus: if function changes variables, current dir, etc – change
of variables is PERMANENT
4/13/2015
Course material created by D.
Woit
29
Function Scope
•
•
•
•
•
•
•
•
•
e.g.
/home/dwoit> ff () {
>
x="def"
>
}
/home/dwoit> x="abc";
echo $x
abc
/home/dwoit> ff
/home/dwoit> echo $x
def
4/13/2015
•
•
•
•
•
•
•
•
•
From a shell program:
e.g.,
#!/bin/bash
ff () {
x="def"
}
x="abc"
ff
echo $x #will print def
Course material created by D.
Woit
30
Variables in a function
• We can make LOCAL variables in a function by using
typeset:
• typeset x=30
– makes x local to the function.
• Otherwise x is set permanently in current shell
4/13/2015
Course material created by D.
Woit
31
Getting at a shell function's positional
parameters from within a function it calls:
• A shell program has parameters $1 $2 ...
• If it calls a function, then WITHIN the function, params. $1
$2 ... are those of the function
– (the shell program's original $1 $2 ... are not available within the
function, but they are still available to the shell program after the
function returns)
–
•
Even if the shell program calls the function with no arguments, the
shell program's params. $1 $2 ... are still unavailable in the function
(because they're all null within the function.)
If we need to access shell pgm's args from within a function:
– Then store $1, $2, ... in arguments before calling function
4/13/2015
Course material created by D.
Woit
32
Passing the shell program’s args to function:
•
•
•
•
•
•
•
•
•
•
•
#!/bin/bash
#source: floc
#try running this as: floc a b c
ff () {
echo "in ff: my \$2 is: $2"
echo "in ff: my caller's \$2 is ${A[2]}"
}
echo in floc before ff: \$2 is: $2
A=($0 $*) #initialize an array A to be all the positional params
ff xxx yyy zzz
echo in floc after ff: \$2 is: $2
4/13/2015
Course material created by D.
Woit
33