UNIX COMP-145
LECTURE 4: THE SHELL COMMAND STRUCTURE
SOURCE:
S. DAS, “YOUR UNIX: THE ULTIMATE GUIDE”, 2ND EDITION, MCGRAW HILL, 2006
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
1
OBJECTIVES
•
•
•
•
•
REVIEW OF FILE ACCESS PERMISSIONS /chmod
UNDERSTAND THE SHELL AS A COMMAND PROCESSOR
LEARN ABOUT WILD CARDS ( *, ?, [] ).
UNDERSTAND HOW THE UNIX PIPE FACILITY WORKS.
LEARN ABOUT I/O REDIRECTION
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
2
REVIEW OF FILE PERMISSIONS
WHAT FILE PERMISSIONS RESULT FROM THE FOLLOWING COMMANDS?
$ chmod a=rwx <FILE>
• SET READ, WRITE & EXECUTE ACCESS FOR ALL
$ chmod a=rw <FILE>
• SET READ, & WRITE ACCESS FOR ALL
$ chmod o+x <FILE>
•
ADD EXECUTABLE ACCESS FOR OTHERS
$ chmod u-rwx <FILE>
•
REMOVE ALL ACCESS FOR OWNER
$ chmod 777 <FILE>
•
SET READ WRITE & EXECUTE ACCESS FOR ALL
$ chmod 750 <FILE>
•
SET READ, WRITE & EXECUTE FOR USER, READ EXECUTE FOR GROUP, NO ACCESS OTHERS
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
3
SHELL AS A COMMAND PROCESSOR
• STEPS:
1. ISSUES PROMPT AND WAITS FOR USER INPUT
2. SCANS FOR META-CHARACTERS IN COMMAND LINE AND
EXPANDS ABBREVIATIONS
3. PASSES COMMAND LINE TO KERNEL FOR PROCESSING
4. SHELL WAITS FOR COMMAND TO COMPLETE, NORMALLY LOCKS
TERMINAL
5. RETURNS TO STEP 1
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
4
DETERMINING THE COMMAND TO RUN
• WHEN A COMMAND IS INVOKED WITH A FULLY QUALIFIED
PATHNAME (E.G., /bin/echo), THE SHELL RUNS PROGRAM
(E.G., ECHO) FOUND IN THE SPECIFIED LOCATION.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
5
PATTERN MATCHING: WILD CARDS
• SUBSET OF META-CHARACTERS USED IN AN EXPRESSION
TO MATCH MULTIPLE BUT SIMILAR FILENAMES
(SEE TABLE 7.1 OF TEXT)
• SHELL CREATES LIST OF FILENAMES BEFORE ALLOWING
COMMAND TO RUN.
• EXPANSION CAN BE PREVENTED BY QUOTING AND
ESCAPING.
• FEATURE ALSO FOUND IN FIND AND SHELL’S FOR AND CASE
CONSTRUCTS.
• FILENAMES MUST NOT CONTAIN WILD-CARD CHARACTERS.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
6
WILD-CARDS SET FOR ls/mv/cp/rm, ETC.
*
?
[ ]
!
—
ANY CHARACTER OR SEQUENCE OF CHARACTERS
ANY SINGLE CHARACTER
ANY OF THE CHARACTERS SPECIFIED IN THE BRACKETS
NOT
IDENTIFIES A RANGE OF CHARACTERS OF NUMBERS
E.G.,
ls *[ab]
ls *[5-7]
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
7
WILD-CARDS SET (CONT’D)
*
ANY NUMBER OF CHARACTERS INCLUDING NONE
ls *.lst
LISTS ALL FILES WITH EXTENSION .LST.
?
A SINGLE CHARACTER
rm ??*
REMOVES ALL FILES COMPRISING AT LEAST 2 CHARACTERS.
[ch]
A SINGLE CHARACTER THAT IS EITHER A c OR h
cp *.[ch] cprogs
COPIES ALL FILES WITh .c OR .h EXTENSION.
[!ch]
A SINGLE CHARACTER THAT IS NOT A c OR h
rm *[!a-zA-Z]*
REMOVES FILES NOT CONTAINING AT LEAST ONE LETTER.
ls .??*
LISTS ALL FILENAMES BEGINNING WITH A DOT AND
COMPRISING AT LEAST TWO MORE CHARACTERS.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
8
THE SHELL META-CHARACTERS
• WILD-CARD CHARACTERS LIKE THE *, ?, ETC.
• REDIRECTION CHARACTERS LIKE >, >>, <, ETC.
• THE PIPE CHARACTER, |
• COMMAND SUBSTITUTION CHARACTERS (` `)
$ echo The date today is `date`
$ Echo “There are `ls | wc –l` files in the current directory”
• THE $ AS A VARIABLE PREFIX, E.G., $PATH
9/30/2009 rwj
BROOKDALE COMMUNITY COLLEGE
9
THE SHELL METACHARACTERS (CONT’D)
cat chap*
SHELL EXPANDS * TO MATCH ALL FILENAMES IN THE CURRENT
DIRECTORY THAT BEGIN WITH CHAP.
date > foo
SHELL SEES THE > FIRST, OPENS THE FILE foo AS A NEW FILE
AND CONNECTS THE DATE OUTPUT TO IT.
date >> foo
SHELL SEES THE >> FIRST, OPENS THE FILE FOO IN THE
CURRENT DIRECTORY AND APPENDS THE DATE OUTPUT TO IT.
who | sort
SHELL UNDERSTANDS THE STRINGS ON EITHER SIDE OF THE |
AS TWO SEPARATE PROGRAMS AND CONNECTS THEM.
ls `cat foo` SHELL FIRST RUNS cat AND SUPPLIES THE OUTPUT AS
ARGUMENTS TO ls.
echo $HOME
9/30/2009 rwj
EVALUATES $HOME AS A VARIABLE BEFORE RUNNING echo.
BROOKDALE COMMUNITY COLLEGE
10
Redirection Characters
• THE PIPE CHARACTER = SPECIAL OPERATOR REDIRECTS
THE OUTPUT OF ONE COMMAND INTO INPUT FOR
ANOTHER COMMAND.
$ ls | wc –l
• THE “>” CHARACTER = REDIRECTS COMMAND OUTPUT
FROM STANDARD E.G., THE SCREEN, TO A FILE
$ ls | wc –l > wordCountOfFiles
9/30/2009 rwj
BROOKDALE COMMUNITY COLLEGE
11
I/O REDIRECTION SUMMARY
• THE SHELL INPUT REDIRECTION:
–
–
–
–
STANDARD INPUT (THE KEYBOARD) - DEFAULT
USE < TO CHANGE THE DEFAULT INPUT TO BE A FILE
USE | TO CHANGE THE DEFAULT INPUT TO BE A COMMAND
A VARIABLE, E.G., $PRINTER
• SHELL REDIRECTION OF OUTPUT:
– USE > TO CREATE IT OR >> TO APPEND TO EXISTING FILE
– USE | TO SEND OUTPUT TO ANOTHER COMMAND
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
12
ESCAPING (USING A \ BEFORE A CHARACTER)
• REVERSES USUAL MEANING OF METACHARACTER FOLLOWING IT.
(rm \* removes a file named *.)
• CAN ALSO PROTECT ITSELF.
(echo \\ prints a \.)
• PROTECTS SPACE AND [ENTER].
(cd My\ Documents will work.)
• INCONVENIENT TO USE WHEN COMMAND LINE CONTAINS TOO MANY
METACHARACTERS THAT NEED TO BE ESCAPED.
• PRINCIPLE ALSO USED BY COMMANDS IN THEIR EXPRESSIONS.
(grep “\.” foo looks for a dot in foo.)
9/30/2009 rwj
BROOKDALE COMMUNITY COLLEGE
13
QUOTING
• Protects most metacharacters from interpretation by
the shell, i.e., indicates to be treated literally.
$ echo “*”
prints an asterisk (*)
• More convenient than escaping when protecting a
group of metacharacters.
• Quoted string understood as a single argument by
shell and C programs. (a.out foo “My Documents”
has 2 arguments and not 3.)
• Double quotes and single quotes are not equivalent.
(echo “$SHELL” not the same as echo ‘$SHELL’)
• Quoting doesn’t protect the backward slash (\),
escaping is also required.
9/30/2009 rwj
BROOKDALE COMMUNITY COLLEGE
14
SINGLE QUOTES OR DOUBLE QUOTES?
• SINGLE QUOTES PROTECT ALL CHARACTERS EXCEPT \
$ echo ‘\’ won’t work
• DOUBLE QUOTES PROTECT ALL CHARACTERS EXCEPT THE \, $ AND `
$ echo “$” doesn’t print a $
• SINGLE QUOTES PROTECT THE “
• DOUBLE QUOTES PROTECT THE ‘
• DOUBLE QUOTES PERMIT VARIABLE EVALUATION AND COMMAND
SUBSTITUTION.
9/30/2009 rwj
BROOKDALE COMMUNITY COLLEGE
15
THE USER MASK
REASSIGNS DEFAULT FILE AND DIRECTORY PERMISSIONS.
• DEFAULT PERMISSIONS BEFORE APPLYING MASK ARE
COMPLETELY INSECURE:
•
• 666 FOR FILES
• 777 FOR DIRECTORIES
• SYSTEM-WIDE DEFAULT CHANGED BY umask
(A SHELL BUILT-IN).
• umask STATEMENT PLACED IN A STARTUP SCRIPT
(TYPICALLY, /etc/profile).
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
16
FILE SYSTEMS
• SYSTEM OF ORGANIZING FILES INTO MULTIPLE MANAGEABLE UNITS.
• EACH FILE SYSTEM HAS A SEPARATE DIRECTORY STRUCTURE WITH
AT TOP.
• FOR A FILE TO BE VISIBLE, ITS FILE SYSTEM MUST BE ATTACHED TO
THE MAIN FILE SYSTEM.
• TWO FILES IN TWO FILE SYSTEMS MAY HAVE THE SAME INODE
NUMBER.
• NOT EASY TO UNDERSTAND WHETHER A DIRECTORY STRUCTURE
COMPRISES MULTIPLE FILE SYSTEMS.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
17
DETERMINING THE COMMAND TO RUN (CONT’D)
• WHEN A COMMAND IS INVOKED WITHOUT A PATHNAME,
THE SHELL FIRST CHECKS WHETHER IT IS AN ALIAS OR BUILTIN:
• IF ALIAS OR BUILT-IN, THE SHELL RUNS IT WITHOUT
LOOKING IN DISK.
 TYPE, WHATIS AND WHICH = EXAMPLES OF BUILT-IN COMMANDS
• IF NOT, THE SHELL LOOKS AT THE PATH VARIABLE FOR
DIRECTORIES WHERE THE COMMAND MAY RESIDE.
 $ ECHO $PATH
7/17/2009 rwj
WHERE ECHO=SHELL, $PATH
BROOKDALE COMMUNITY COLLEGE
= INPUT VARIABLE
18
FILE ATTRIBUTES STORED IN INODE
• TYPE: WHETHER ORDINARY, DIRECTORY, DEVICE, ETC.
• PERMISSIONS: DETERMINES WHO CAN READ, WRITE OR EXECUTE A FILE.
• LINKS: NUMBER OF NAMES A FILE CAN HAVE. A PROGRAM CAN BE DESIGNED
TO
BEHAVE DIFFERENTLY DEPENDING ON THE NAME BY WHICH IT IS INVOKED.
• OWNER: A FILE IS OWNED BY A USER, BY DEFAULT ITS CREATOR. THE OWNER CAN
CHANGE MANY FILE ATTRIBUTES AND SET THE PERMISSIONS.
• GROUP OWNER: THE GROUP WHICH OWNS THE FILE. THE OWNER BY DEFAULT
BELONGS TO THIS GROUP.
• FILE SIZE: NUMBER OF BYTES OF DATA CONTAINED.
• FILE TIME STAMPS:
• DATE AND TIME OF LAST MODIFICATION
• DATE AND TIME OF LAST ACCESS
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
19
DIRECTORY PERMISSIONS
• READ PERMISSION: WHETHER FILENAMES IN DIRECTORY
CAN BE LISTED BY A PROGRAM (LIKE ls).
• WRITE PERMISSION: WHETHER FILES AND DIRECTORIES
CAN BE CREATED IN THE DIRECTORY.
• EXECUTE OR SEARCH PERMISSION: WHETHER ONE CAN
PASS THROUGH DIRECTORY TO SEARCH FOR FILENAMES.
• DESIRABLE PERMISSION SETTING: 755
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
20
AN OWNERSHIP-PERMISSIONS PROBLEM
ASSUMPTION: ROMEO AND JULIET BELONG TO THE USERS GROUP.
OWNERSHIP AND PERMISSIONS OF FILE FOO AND ITS DIRECTORY
$ who am i
romeo
$ ls -l foo
-r-x-w-r-x 1 juliet
users
7017 2004-11-14 13:53 foo
$ ls -ld .
drwxr-xr-x 21 romeo
users
8192 2004-11-28 11:40 .
NOTE: foo IS OWNED BY JULIET BUT DIRECTORY IS OWNED BY ROMEO.
JULIET:
• CAN’T EDIT foo WITHOUT CHANGING THE PERMISSIONS.
• CAN CHANGE PERMISSIONS (AS OWNER) AND THEN EDIT foo.
• CAN’T DELETE foo (DIRECTORY WRITE-PROTECTED FOR GROUP).
ROMEO:
• CAN EDIT OR DELETE foo.
• CAN’T CHANGE PERMISSIONS OF foo.
• CAN’T DISPLAY OR COPY foo.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
21
THE INODE REVISITED
• SYSTEM OF ORGANIZING FILE ATTRIBUTES SEPARATELY
FROM CONTENT.
• IDENTIFIED BY INODE NUMBER BUT INODE DOESN’T
CONTAIN THIS NUMBER.
• INODE NUMBER DISPLAYED BY ls -i
• BOTH INODE AND DIRECTORY ENTRIES ARE LOOKED UP BY
INODE NUMBER.
• POSSIBLE TO CONSUME ALL INODES EVEN WHEN THERE IS
ADEQUATE DISK SPACE.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
22
MAKING A PROGRAM BEHAVE DIFFERENTLY
• USING ARGUMENTS AND OPTIONS.
• MANIPULATING A CONFIGURATION FILE READ BY A
PROGRAM ON STARTUP.
1. MODIFIABLE TEXT FILES BY HAND OR VIA SHELL
SCRIPTS
• USING DIFFERENT NAMES FOR THE SAME FILE.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
23
LINKS OR HARD LINKS
• MECHANISM BY WHICH A FILE IS ALLOWED TO HAVE
MULTIPLE NAMES.
• LINKED FILENAMES SHARE INODE BUT HAVE SEPARATE
DIRECTORY ENTRIES.
• EACH LINK INCREMENTS LINK COUNT IN INODE BY 1 AND
ADDS AN ENTRY TO THE DIRECTORY.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
24
LINKS OR HARD LINKS (CONT’D)
• FILE CONSIDERED TO BE DELETED AND INODE FREED ONLY
WHEN LINK COUNT DROPS TO 0.
• LINKED FILENAMES EQUIVALENT IN ALL RESPECTS.
• THE ln COMMAND CREATES LINKS AND rm REMOVES THEM
1. INODE # REVEALS WHETHER FILENAMES ARE LINKS OR COPIES.
2. TWO FILENAMES IN THE SAME DIRECTORY AND HAVING THE
SAME INODE NUMBER MUST BE LINKED TO EACH OTHER.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
25
LINKS OR HARD LINKS (CONT’D)
QUESTION:
IF TWO FILES IN TWO DIFFERENT DIRECTORIES HAVE THE SAME INODE
NUMBER, HAVE A LINK COUNT OF 2, AND ALSO HAVE IDENTICAL
CONTENT, ARE THEY LINKED FILENAMES?
ANSWER:
NOT NECESSARILY; THEY CAN BE TWO SEPARATE FILES ON TWO FILE
SYSTEMS HAVING THE SAME INODE NUMBER BY COINCIDENCE. THEY
MAY ALSO HAVE THEIR OWN SEPARATE HARD LINKS. IT’S DIFFICULT TO
CREATE THIS SITUATION, BUT IT’S NOT IMPOSSIBLE.
BUT LINKS CAN BE IN DIFFERENT DIRECTORIES, SO HOW DOES ONE
LOCATE THEM? USE THE -INUM OPTION OF THE FIND COMMAND THAT
CAN TRACK DOWN A FILE BY ITS INODE NUMBER.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
26
ADVANTAGES OF HARD LINKS
• BACKUP: PREVENTION FROM ACCIDENTAL DELETION.
• ALLOWS THE SAME FILE TO BE EXECUTED AS TWO SIMILAR
BUT SEPARATE PROGRAMS.
• TAKES CARE OF OLD PROGRAMS THAT ACCESSES A FILE
WHOSE NAME OR LOCATION HAS CHANGED.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
27
LIMITATIONS OF HARD LINKS
• CAN’T LINK DIRECTORIES.
• CAN’T LINK ACROSS FILE SYSTEMS.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
28
SYMBOLIC LINKS OR SYMLINKS
• SEPARATE FILE TYPE AND HAVING ITS OWN INODE.
• CONTAINS THE PATHNAME OF ANOTHER FILE OR DIRECTORY.
• CAN LINK ACROSS FILE SYSTEMS.
• LINK AND FILE POINTED TO ARE NOT EQUIVALENT.
• PATHNAME MAY BE STORED EITHER IN INODE OR IN A SEPARATE
FILE.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
29
UNDERSTANDING OWNERSHIP AND GROUP OWNERSHIP
ENTRY FOR romeo in /etc/passwd:
romeo:x:1003:101::/export/home/romeo:/usr/bin/ksh
ENTRY FOR romeo’s GROUP (101) in /etc/group:
users::101:
• A FILE HAS A UID (NAME AND NUMBER).
• BOTH UID PARAMETERS MAINTAINED IN /ETC/PASSWD (1ST AND 3RD FIELD).
• A FILE HAS A GID (NAME AND NUMBER).
• BOTH GID PARAMETERS MAINTAINED IN /ETC/GROUP (1ST AND 3RD FIELD).
• THE NUMERIC GID ALSO MAINTAINED IN /ETC/PASSWD.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
30
OWNERSHIP IN SVR4 AND BSD
ATTRIBUTE
SVR4
BSD
OWNERSHIP
CAN BE TRANSFERRED
ONLY BY SUPERUSER BUT CAN BE REMOVED.
GROUP OWNERSHIP
CAN BE TRANSFERRED
ONLY TO ANOTHER GROUP TO REMOVED
BUT CAN BE MODIFIED.
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
31
Backup Slides
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
32
FILE ACCESS PERMISSIONS
-rwxrwxrwx
-
rwx
rwx
File Type
Owner
Group
rwx
World
- = file
d = directory
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
33
OWNER, GROUP, THE WORLD
• OWNER
– THE CREATOR OF THE FILE
• GROUP
– A SET OF USERS GROUPED TOGETHER
• THE WORLD
– EVERY OTHER ACCOUNT NOT IN THE GROUP
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
34
READ, WRITE, EXECUTE
• READ
– FILE CAN BE READ, BUT NOT MODIFIED
• WRITE
– PERMISSION IS GRANTED TO MODIFY THE FILE
• EXECUTE
– RUN DIRECTLY AS IF THE FILE IS A PROGRAM
– ALL PROGRAMS SHOULD BE EXECUTABLE (/BIN)
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
35
WHAT PERMISSIONS MEAN ON A DIRECTORY
• READ
– USERS CAN GET A LISTING OF THAT DIRECTORY
• WRITE
– USERS CAN CREATE AND REMOVE FILES IN THAT DIRECTORY
• EXECUTE
– USERS CAN EXAMINE FILES IN THAT DIRECTORY
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
36
CHANGING PERMISSIONS WITH CHMOD
chmod <SETTINGS> <FILE>
u = user
g = group
o = other
a = all
+ (add)
r = read
- (remove)
w = write
= (set)
x = execute
chmod a-x testFile
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
37
MORE EXAMPLES
• SET READ AND WRITE ACCESS FOR ALL?
– chmod a=rw <FILE>
• ADD EXECUTABLE ACCESS FOR OTHERS?
– chmod o+x <FILE>
• REMOVE ALL ACCESS FOR OWNER?
– chmod u-rwx <FILE>
• SET READ, WRITE & EXECUTE ACCESS FOR ALL?
– chmod a=rwx <FILE>
• 777 PERMISSIONS
o OCTAL REPRESENTATION OF -rwxrwxrwx
o THREE BINARY DIGITS OR BITS CORRESPONDS TO ONE OCTAL DIGIT:
 Read = 4, Write = 2 Execute =1 i.e., rwx = 4+2+1 = 7
 chmod 750 <file> same as chmod u=rwx,g=rx,o= <file>
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
38
ADVANCED CHMOD USAGE
• MOST UNIX HACKERS DON’T USE THIS FORM
• THEY PREFER THE MORE DIRECT APPROACH
– SET PERMISSIONS FOR OWNER, GROUP, AND OTHERS ALL WITH ONE NUMBER
• UNFORTUNATELY, THIS APPROACH REQUIRES A LITTLE BIT OF
INFORMATION
7/17/2009 rwj
BROOKDALE COMMUNITY COLLEGE
39