CPS 393
Introduction to Unix and C
START OF WEEK 2 (UNIX)
4/7/2015
Course material created by D.
Woit
1
Glob Constructs (Metacharacters) and other
Special Characters
• ? * [ ]
#constructs
– saves typing, time
– encourages good naming conventions
– when shell sees a glob construct, it first *expands* word containing
construct, then does command.
• ?
any single character
• *
any string of characters (incl. null string)
• [...] any single char of series of chars within the brackets
• e.g., assume /home/dwoit has 7 files:
•
lab1.jav lb2.jav new.c pie.c prog1.c prog2.c top.c
•
/home/dwoit> ls *.c
•
new.c pie.c prog1.c prog2.c top.c
4/7/2015
Course material created by D.
Woit
2
(Metacharacters) and other Special Characters
cont.
• /home/dwoit> ls *
lab1.jav lb2.jav new.c pie.c prog1.c prog2.c top.c
• /home/dwoit> ls prog?.c
prog1.c prog2.c
• /home/dwoit> ls p*.c
prog1.c prog2.c pie.c
• /home/dwoit> ls ???.*
lb2.jav new.c pie.c top.c
• /home/dwoit> ls prog[123].c
prog1.c prog2.c
4/7/2015
Course material created by D.
Woit
3
Metacharacters and other Special Chars
cont.
• /home/dwoit>
ls [m-s]*
# all files starting with m ... s
new.c pie.c prog1.c prog2.c
• [1-5A-Z]
• [!ABC]
• [!a-z0-47]
matches any char 1,2,3,4,5,A,B,C, ... Z
matches any char *except* A, B or C
matches any char *except* a,b,c, ... z, 0,1,2,3,4,7
– NOTE: ranges, such as a-z depend on locale having LC_COLLATE set
normally.
– It can be set to something weird, such as thinking order is aAbBcC...zZ
instead of the normal ASCII sequence abc..zABC...Z
– You can do a simple test to check it out: e.g., if ls [a-z] matches A it is not
normal. (in latest bash, either ! or ^ is not)
4/7/2015
Course material created by D.
Woit
4
Other special characters, expanded to
• ~
your home directory name (e.g. /home/dwoit )
– from any dir, ls ~
lists your home dir (/home/jmisic for me)
– cd ~/cps393 changes into dir /home/jmisic/cps393 for me
• ~usr the home dir of user with userid "usr "
• ~your previous working dir (note in bash cd - == cd ~-)
• ~+
your current working dir
• Example: toggle between 2 dirs with "cd -" in bash
– /home/dwoit>
cd bin
– /home/dwoit/bin> cd – /home/dwoit> cd – /home/dwoit/bin>
4/7/2015
Course material created by D.
Woit
5
Directories and files
• /home/jmisic/cps393>
echo my working dir is ~+
– my working dir is /home/jmisic/cps393
• /home/jmisic/cps393/labs> echo all the files in this dir are *
– all the files in this dir are lab1.txt
• However, if we put the quotes we get:
• /home/jmisic/cps393/labs> echo “all the files in this dir are *”
– echo all the files in this dir are *
• /home/jmisic/cps393/labs> echo all the files in this dir are “*”
– echo all the files in this dir are *
4/7/2015
Course material created by D.
Woit
6
Directories and files cont.
• Special chars are not expanded if within single or double
quotes
• escape character \ before any special char works like quotes
• /home/jmisic/cps393/labs>
echo all the files in this dir are \*
– echo all the files in this dir are *
4/7/2015
Course material created by D.
Woit
7
Patterns and Regular Expressions in Shell
• in ksh they work normally
• -in bash if you execute first: shopt -s extglob (extended
pattern matching)
– Without this option only *, ? And […] work.
• For example:
• *(exp)
0 or more occurrences of exp
• +(exp)
1 or more occurrences of exp
• ?(exp)
0 or 1 occurrences of exp
• @(exp1|exp2|...) exp1 or exp2 or ...
• !(exp)
anything that doesn't match exp
4/7/2015
Course material created by D.
Woit
8
Patterns and Regular Expressions in Shell
• Example: if you had files in the current dir such as below:
–
A Ax Axxx Axxxx X X.bak x xx xxxx
•
Then (but carefully since no extra spaces are allowed in braces):
•
•
•
•
•
•
•
shopt -s extglob
ls A*(x)
prints A Ax Axxx Axxxx
ls A*(xx)
prints A Axxxx
ls A+(x)
prints Ax Axxx Axxxx
ls A?(x)
prints A Ax
ls X?(.bak)
prints X X.bak
ls @(*xx|*ak) prints Axxx Axxxx X.bak xx xxxx
•
ls !(@(*xx|*ak)) prints A Ax X x
4/7/2015
Course material created by D.
Woit
9
Homework
•
•
•
•
•
•
How would you list all file/dir names that:
1. contained the letter "e";
2. had a total of 7 letters and no extension;
2a. are 7 characters long;
3. had 3 letter extensions;
3a. had 3 letter alpha-numeric extensions;
• 4. contained the word "tst" *somewhere* in the name;
• 5. started with the letter "A" and contained a dash (-);
• 6. are at least 2 characters long and don't have the following letters
anywhere in the first 2 characters: a,b,c,f,h,x,y,z
• 7. end in .ex with an optional c at the end
• 8. are not java or text files (don't end in .jav or .txt)
• 9. end in a : followed by a digit from 2-8 followed zero or more digits.
4/7/2015
Course material created by D.
Woit
10
Interactive Shell Use
• Is usually used for
– typing on command line
– one-time task
– Prototyping
• For repetitive use we build Shell scripts (Shell programs)
– task complex
– repeat it later
4/7/2015
Course material created by D.
Woit
11
Example shell script
• #!/bin/bash
• #source: showSource
• #shell program to list all source java and c files in current directory
•
•
•
•
•
•
•
echo -n "Current directory is: "
pwd
echo "C files are:"
ls *.c
echo "Java files are:"
ls *.jav
To run: make sure it is executable (chmod ugo+x
showSource or chmod +x showSource ), then
• jmisic@metis:~/cps393/labs$ ./showSource
4/7/2015
Course material created by D.
Woit
12
Example output of the script is:
•
•
•
•
•
•
jmisic@metis:~/cps393/labs$ ./showSource
current directory is/home/jmisic/cps393/labs
c files are:
smitz5.c smitz_scanf.c
java files are:
proba.jav
4/7/2015
Course material created by D.
Woit
13
Arguments to shell script: $1 $2 $3 etc.
•
•
•
•
•
•
•
•
•
•
•
$0 name of shell pgm
$# count of # of args passed
$@ lists args as one string: "arg1 arg2 ...“
$* lists args as separate : "arg1" "arg2" ...
Example:
#!/bin/bash
#source: prnargs
#shell pgm to print out its first 2 args
echo The first argument is: $1
echo The second argument is: $2
exit 0
4/7/2015
Course material created by D.
Woit
14
Arguments: $1 $2 $3 etc.
• To run set execute permissions:
• jmisic@metis:~/cps393/labs$ ./prnargs "Hello there " world
– The first argument is: Hello there
– The second argument is: world
• Note:
• could have used quotes in echo:
–
–
echo "The first argument is: " $1
echo "The second argument is:" $2
• or could have used echo backslash-escape TAB char, \t
(see man echo)
– echo -e "The first argument is:\t" $1
– echo -e "The second argument is:\t" $2
4/7/2015
Course material created by D.
Woit
15
How to leave out the ./
• current dir has to be in your path
• If you wish to invoke prnargs (instead of ./prnargs )
• you need to edit your .profile file in your home directory
(but save it first)
• Since current directory is represented as null name
followed by a colon you have to insert at the end :
– PATH=$PATH::
– export PATH
• Export command makes PATH variable available to all subsequent
processes initiated by the user.
4/7/2015
Course material created by D.
Woit
16
Filters
• Filter refines or transforms the input to produce (usually
different) output
•
stdin -------> filter --------> stdout
• E.g. more < file1
–
transforms sequence of characters from file1 (its input)
– into the same sequence but with pagination
4/7/2015
Course material created by D.
Woit
17
Grep (globally look for a regular
expression and print)
• grep string filename(s)
–
–
–
–
string:
the string to search for
filename(s): one or more files to search in
result:
displays lines of file(s) filename(s) that contain the given string
(precisely: sends to stdout lines of stdin that contain string)
• e.g. grep 'applet' lab1.jav lab2.jav
•
sends to stdout (screen) those lines of files lab1.jav and
•
lab2.jav that contain the string applet
•
/home/jchan> grep 'applet' lab1.jav lab2.jav
•
lab1.jav:
applet.init();
•
lab1.jav:
applet.start();
•
lab2.jav:
applet.num=6;
4/7/2015
Course material created by D.
Woit
18
Grep cont.
• e.g. You forgot the exact syntax for 'for' construct in java.
•
You know it is used in one of your .jav files. Do:
• grep 'for' *.jav
– to get a list of all files/lines containing for
– Then edit or more one file to see exact syntax
• Note:
– single quotes are good for now. Once we study variables, you
– we will have reason to use double quotes around the search string.
• Options for grep
• -i option: ignore case of search string
• -v option: print lines *not* matching search string
4/7/2015
Course material created by D.
Woit
19
Grep with regular expressions and
metacharacters
•
•
•
•
•
•
•
•
•
.
any character, except newline (like ? in glob)
*
0 or more repetitions of previous character (not like glob!)
^
beginning of line
$
end of line
[ ... ] any character inside the brackets (like glob)
[^ ... ]
any character *not* inside the brackets (like ! or ^ in glob)
[! ... ]
same as above on some implementations
\{m\} exactly m repetitions of previous character
\{m,n\} any number of repetitions of prev char between m and n
inclusive
• \<
beginning of word
• \>
end of word
• remember to quote these as in: grep '\<a_word\>' a_file
4/7/2015
Course material created by D.
Woit
20
Examples: Grep with special chars.
• grep '^Assignment' fname # lines starting with Assignment
• grep -v '^Assignment' fname # lines not starting with
Assignment
• grep 'Assignment$' fname # lines ending with Assignment
• grep 'd.g' fname
# lines containing dag, dbg, dcg, d0g,
d1g, etc
• grep 'su*m' fname
# lines containing sm, sum, suum,
suuum, etc.
• grep '\<so\>' fname
# *word* so (vs. social, absolute)
• grep '[A-Za-z][A-Za-z]*' fname # lines containing ANY nonempty alpha string (no blank lines, lines with digits, etc)
4/7/2015
Course material created by D.
Woit
21
Examples: Grep with special chars
• grep 'xyz\.[^ ]* '
# lines containing string xyz followed
by a dot followed by 0 or more non-spaces, then one space
e.g., "xyz. " "xyz.w " "xyz.ta " etc (see slide 20)
• grep 'xyz\.w\{2,3\}X'
# lines containing xyz.wwX or
xyz.wwwX
4/7/2015
Course material created by D.
Woit
22
Grep with Extended Regular Expressions:
• use grep -E or egrep (system dependent)
OR
• + 1 or more repetitions of previous character
• e.g.,
• grep -E 'dog|cat|bird' fname # lines with at least 1 of dog,
cat, or bird
• grep -E '[A-Za-z]+' fname
# lines with at least one alpha
char
• Note this also matches lines with no alpha chars since *
matches null:
• grep
4/7/2015
'[A-Za-z]*' fname
Course material created by D.
Woit
23
Homework
• In the shell, is grep '[A-Za-z][A-Za-z]*' fname the same as
grep -i '[A-Z]*' fname ? Why or why not?
• Why do you think "\<" and \>" are used instead of simply "<" and ">"
for beginning/end of line?
• For questions 1-6 below, write a different shell program.
1. list all lines in file fname containing the string "dog";
2. list all lines in fname containing the word "dog";
3. list all lines in fname containing the string "dog" at the beginning of a
line;
4. list all lines in fname containing the word "dog" at the beginning of a
line.
5. list all lines in fname containing exactly 6 "a"s in a row.
6. list all lines in fname containing one or more words that start with 93
and end in a sequence of any number of W (not 0)
7. Find 3 different grep commands that will list lines of fname that end in
a vowel.
8. Use the man page to find the option of grep that shows the matched
string in a different color. Try it.
4/7/2015
Course material created by D.
Woit
24
find (list all files and dirs meeting certain
criteria )
• e.g., this lists all java files in portion of the file system rooted
at /home/dwoit:
• find /home/dwoit -name "*.jav"
– match any file with name *.jav (glob)
– dir to search (this dir and all its subdirectories )
•
find . -type f
# lists all *files* in filesystem
rooted at the current dir (.)
(i.e., no directories listed;
for dirs -type d)
4/7/2015
Course material created by D.
Woit
25
find cont.
•
find /usr -name banner 2>/dev/null
• find . -type d -perm -g+r,u+r # all dirs rooted in current dir
that are readable by both owner and group LOTS of ways to
specify perms--see man find
4/7/2015
Course material created by D.
Woit
26
Notes for find
•
If you want it to go down symbolic links, you might have
to use the -follow option.
• Some versions of linux have default find commands that
follow automatically, and others don't...
• Some years scs students' home dirs are symbolic links.
• Note: if glob constructs not in quotes, shell expands *first*
and then sends all to find. Again, single or double quotes
OK for now.
4/7/2015
Course material created by D.
Woit
27
HMWK
1. Use the "find" command to list all files in directory /bin that start with
2.
3.
4.
5.
the letter "m".
What would happen if we did not put a name such as "*.jav" in quotes
in the find command? Make sure you have some .jav files in your
directory and try it out.
Look at the man pages for "find" to determine what the option "mtime" does in the find command. Then..Figure out a sequence of
commands (involving a find, temp file and a grep) to display those
files in the current directory whose contents werechanged within the
last 24 hours and whose names contain the string "tst".
Make a shell program to do question 3 above. Call your program
tstRecent. Your program should delete any temporary files it
creates.Test it to make sure your program works.
Use the man pages to find out how find's -maxdepth option works.
Use -maxdepth 1 to list all files in the given directory only (not the
whole filetree)
4/7/2015
Course material created by D.
Woit
28
Head, Tail
•
•
•
•
•
•
•
head sends to stdout the first 10 lines of stdin
tail sends to stdout the last 10 lines of stdin
e.g.,
head <myfile
# displays first 10 lines of file
myfile on screen
tail <myfile >end.of.myfile # puts copy of last 10 lines of
myfile into file called end.of.myfile
• option -number displays first/last "number" lines
• e.g., head -20 f1
# first 20 lines of file f1
• (options are in man pages. Lots for tail. )
4/7/2015
Course material created by D.
Woit
29