UNIX Shell-Scripting Basics Agenda What is a shell? A shell script? Introduction to bash Running Commands Applied Shell Programming What is a shell? What is a shell? /bin/bash What is a shell? #!/bin/bash What is a shell? INPUT shell OUTPUT ERROR What is a shell? Any Program But there are a few popular shells… Bourne Shells /bin/sh /bin/bash “Bourne-Again Shell” Steve Bourne Other Common Shells C Shell (/bin/csh) Turbo C Shell (/bin/tcsh) Korn Shell (/bin/ksh) An aside: What do I mean by /bin ? C Shell (/bin/csh) Turbo C Shell (/bin/tcsh) Korn Shell (/bin/ksh) An aside: What do I mean by /bin ? /bin, /usr/bin, /usr/local/bin /sbin, /usr/sbin, /usr/local/sbin /tmp /dev /home/borwicjh What is a Shell Script? A Text File With Instructions Executable What is a Shell Script? % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world What is a Shell Script? A Text File % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world An aside: Redirection cat > /tmp/myfile cat >> /tmp/myfile cat 2> /tmp/myerr cat < /tmp/myinput cat <<INPUT Some input INPUT cat > /tmp/x 2>&1 INPUT 0 env OUTPUT 1 ERROR 2 What is a Shell Script? How To Run % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world What is a Shell Script? What To Do % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world What is a Shell Script? Executable % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world What is a Shell Script? Running it % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world Finding the program: PATH % ./hello.sh echo vs. /usr/bin/echo % echo $PATH /bin:/usr/bin:/usr/local/bin: /home/borwicjh/bin % which echo /usr/bin/echo Variables and the Environment % hello.sh bash: hello.sh: Command not found % PATH=“$PATH:.” % hello.sh Hello, world An aside: Quoting % echo ‘$USER’ $USER % echo “$USER” borwicjh % echo “\”” ” % echo “deacnet\\sct” deacnet\sct % echo ‘\”’ \” Variables and the Environment % env […variables passed to sub-programs…] % NEW_VAR=“Yes” % echo $NEW_VAR Yes % env […PATH but not NEW_VAR…] % export NEW_VAR % env […PATH and NEW_VAR…] Welcome to Shell Scripting! How to Learn man man bash man cat man man man –k man –k manual Learning the Bash Shell, 2nd Ed. “Bash Reference” Cards http://www.tldp.org/LDP/abs/html/ Introduction to bash Continuing Lines: \ % echo This \ Is \ A \ Very \ Long \ Command Line This Is A Very Long Command Line % Exit Status $? 0 is True % ls /does/not/exist % echo $? 1 % echo $? 0 Exit Status: exit % cat > test.sh <<_TEST_ exit 3 _TEST_ % chmod +x test.sh % ./test.sh % echo $? 3 Logic: test % % 0 % % 1 test 1 -lt 10 echo $? test 1 == 10 echo $? Logic: test test [ ] [ 1 –lt 10 ] [[ ]] [[ “this string” =~ “this” ]] (( )) (( 1 < 10 )) Logic: test [ [ [ [ -f /etc/passwd ] ! –f /etc/passwd ] -f /etc/passwd –a –f /etc/shadow ] -f /etc/passwd –o –f /etc/shadow ] An aside: $(( )) for Math % echo $(( 1 + 2 )) 3 % echo $(( 2 * 3 )) 6 % echo $(( 1 / 3 )) 0 Logic: if if something then : # “elif” a contraction of “else if”: elif something-else then : else then : fi Logic: if if [ $USER –eq “borwicjh” ] then : # “elif” a contraction of “else if”: elif ls /etc/oratab then : else then : fi Logic: if # see if a file exists if [ -e /etc/passwd ] then echo “/etc/passwd exists” else echo “/etc/passwd not found!” fi Logic: for for i in 1 2 3 do echo $i done Logic: for for i in /* do echo “Listing $i:” ls -l $i read done Logic: for for i in /* do echo “Listing $i:” ls -l $i read done Logic: for for i in /* do echo “Listing $i:” ls -l $i read done Logic: C-style for for (( expr1 expr2 expr3 do list done ; ; )) Logic: C-style for LIMIT=10 for (( a=1 ; a<=LIMIT ; a++ )) do echo –n “$a ” done Logic: while while something do : done Logic: while a=0; LIMIT=10 while [ "$a" -lt "$LIMIT" ] do echo -n "$a ” a=$(( a + 1 )) done Counters COUNTER=0 while [ -e “$FILE.COUNTER” ] do COUNTER=$(( COUNTER + 1)) done Note: race condition Reusing Code: “Sourcing” % cat > /path/to/my/passwords <<_PW_ FTP_USER=“sct” _PW_ % echo $FTP_USER % . /path/to/my/passwords % echo $FTP_USER sct % Variable Manipulation % FILEPATH=/path/to/my/output.lis % echo $FILEPATH /path/to/my/output.lis % echo ${FILEPATH%.lis} /path/to/my/output % echo ${FILEPATH#*/} path/to/my/output.lis % echo ${FILEPATH##*/} output.lis It takes a long time to become a bash guru… Running Programs Reasons for Running Programs Check Return Code $? Get Job Output OUTPUT=`echo “Hello”` OUTPUT=$(echo “Hello”) Send Output Somewhere Redirection: <, > Pipes Pipes Lots of Little Tools INPUT 0 echo echo “Hello” | \ wc -c OUTPUT 1 ERROR 2 A Pipe! INPUT 0 wc OUTPUT 1 ERROR 2 Email Notification % echo “Message” | \ mail –s “Here’s your message” \ borwicjh@wfu.edu Dates % DATESTRING=`date +%Y%m%d` % echo $DATESTRING 20060125 % man date FTP the Hard Way ftp –n –u server.wfu.edu <<_FTP_ user username password put FILE _FTP_ FTP with wget wget \ ftp://user:pass@server.wfu.edu/file wget –r \ ftp://user:pass@server.wfu.edu/dir/ FTP with curl curl –T upload-file \ -u username:password \ ftp://server.wfu.edu/dir/file Searching: grep % % % % % grep grep grep grep grep rayra /etc/passwd –r rayra /etc –r RAYRA /etc –ri RAYRA /etc –rli rayra /etc Searching: find % find /home/borwicjh \ -name ‘*.lis’ [all files matching *.lis] % find /home/borwicjh \ -mtime -1 –name ‘*.lis’ [*.lis, if modified within 24h] % man find Searching: locate % locate .lis [files with .lis in path] % locate log [also finds “/var/log/messages”] Applied Shell Programming Make Your Life Easier TAB completion Control+R history cd Study a UNIX Editor pushd/popd % cd /tmp % pushd /var/log /var/log /tmp % cd .. % pwd /var % popd /tmp Monitoring processes ps ps –ef ps –u oracle ps –C sshd man ps “DOS” Mode Files #!/usr/bin/bash^M FTP transfer in ASCII, or dos2unix infile > outfile sqlplus JOB=“ZZZTEST” PARAMS=“ZZZTEST_PARAMS” PARAMS_USER=“BORWICJH” sqlplus $BANNER_USER/$BANNER_PW << _EOF_ set serveroutput on set sqlprompt "" EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER'); _EOF_ sqlplus sqlplus $ARG1 if [ $? then exit 1 fi if [ -e then $USER/$PASS @$FILE_SQL \ $ARG2 $ARG3 –ne 0 ] /file/sql/should/create ] […use SQL-created file…] fi Ask Amy Lamy! Passing Arguments % cat > test.sh <<_TEST_ echo “Your name is \$1 \$2” _TEST_ % chmod +x test.sh % ./test.sh John Borwick ignorethis Your name is John Borwick INB Job Submission Template $1: user ID $2: password $3: one-up number $4: process name $5: printer name % /path/to/your/script $UI $PW \ $ONE_UP $JOB $PRNT Scheduling Jobs % 0 0 * 0 % % crontab -l 0 * * * daily-midnight-job.sh * * * * hourly-job.sh * * * * every-minute.sh 1 * * 0 1AM-on-sunday.sh EDITOR=vi crontab –e man 5 crontab Other Questions? Shells and Shell Scripts bash Running Commands bash and Banner in Practice