Command Line Variations
Selected Readings in Chapters 3, 5, 6
 Project
1 kickoff
 Streams & Redirection: Sections 5.2, 5.3
 Pipes and Tees: Sections 5.4, 5.5
 Filenames & Wildcards: Section 3.1
 Job Control: Section 5.10
 Filter Commands: details in Chapter 6
cat, tr, head, tail, cut, paste, sort,
wc (word count)
Figure 5-4
Standard Streams
Standard I/O
 Standard
Output (stdout) labelled (1) above
– default place to which programs write
 Standard
Input (stdin) labelled (0) above
– default place from which programs read
 Standard
Error (stderr) labelled (2) above
– default place where errors are reported
 To
demonstrate -- cat
– Echoes everything you typed in with an <enter>
– Quits when you press Ctrl-d at a new line -- (EOF)
Figure 5-6
Redirecting Standard Output
Redirecting Standard Output
 cat
file1 file2 > file3
– concatenates file1 and file2 into file3
– file3 is created if not there
– Any pre-existing file3 is clobbered if there
 cat
file1 file2 >> file3
– file3 is created if not there
– file3 is appended to if it is there
 cat
> file3
– file3 is created from whatever user provides from
standard input
Figure 5-7
Print who Output
Redirecting Standard Error

In bash (for shell scripts), standard error is redirected
using 2>
– cat myfile > yourfile 2> yourerrorfile




A better example that generates errors:
ls –R /
recursively list all files from root
you will get many screens of output
Redirect output into a file:
ls –R / > allfiles
save output from above
you will get many “permission denied errors”
more allfiles
but you can look at listing in file now
Redirect errors to a second file:
ls –R / > allfiles 2> errors
errors go into file errors
If you don’t care about errors
ls –R / > allfiles 2> /dev/null /dev/null is a ‘bit bucket’
where bits go to die!
Figure 5-5
Redirecting Standard Input
Redirecting Standard Input
 terra[1]
$ cat < oldfile > newfile
 A more useful example:
– terra[2] $ tr string1 string2
Read from standard input.
Character n of string1 translated to
character n of string2.
Results written to standard output.
– Example of use:
terra[3] $ tr aeoiu eoiua
terra[4] $ tr a-z A-Z < file1 > file2
/dev/null
 /dev/null
– A virtual file that is always empty.
– Copy things to here and they disappear.
cp myfile /dev/null
mv myfile /dev/null
– Copy from here and get an empty file.
cp /dev/null myfile
– Redirect error messages to this file
cat myfile > yourfile 2> /dev/null
Basically, all error messages are discarded.
Figure 6-1
Concept Of A Filter
Basic Filters
 Filters
are programs that:
– Read stdin.
– Modify it.
– Write the results to stdout.
 Filters
typically do not need user input.
 Example:
– tr (translate):
 Read stdin
 Echo to stdout, translating some specified
characters
 Many
filters can also take file names as
operands for input, instead of using stdin.
Filtering using tr (translate command)

Using tr to translate what you type into upper case
tr [a-z] [A-Z]
(enter, type, enter etc. ctrl-D to quit)

Using tr to translate a file using input redirection
tr [a-z] [A-Z] < foods
(now foods will appear UC)
tr [a-z] [A-Z] foods (also OK, tr accepts filename args)

Using tr to translate a file using input redirection and
store results using output redirection
tr [a-z] [A-Z] < foods > foodsCAP
tr [a-z] [A-Z] foods > foodsCAP (also OK, see above)
(now foodsCAP will hold UC output from foods)
More filters: grep, wc, sort (see Ch6)
 grep
patternstr
– Read stdin and write lines with patternstr to stdout
terra[1] $ grep "unix is easy" < myfile1 > myfile2
terra[1] $ grep "unix is easy" myfile1 > myfile2 (also OK)
– Write all lines of myfile1 containing phrase unix is easy
to myfile2
 wc
– Count the number of chars/words/lines on stdin
– Write the resulting statistics to stdout
– Example: (wc < mars.txt
OR wc mars.txt )
 sort
– Sort all the input lines in alphabetical order and write to
the standard output.
Figure 5-8
Piping Output to Next Command
Pipes
 The
pipe:
– Connects stdout of one program with stdin of another
– General form:
command1 | command2
– stdout of command1 used as stdin for command2
– Example:
terra[1] $ cat foods | grep apples | wc -l
 A pipe
works the same as (but more efficient than)
terra[2] $ cat foods > tmp1
terra[3] $ grep apples < tmp1 > tmp2
terra[4] $ wc -l < tmp2
 Actually
cat is redundant: grep apples foods | wc -l
Redirecting and Pipes (2)
 Note:
The name of a command always comes
first on the line.
 There may be a tendency to say:
terra[1] $ foods > grep apples | wc -l
– This is WRONG!!!
– Your shell will go looking for a program named foods
 To
do it correctly, many alternatives!
terra[1] $ cat foods | grep apples | wc -l
terra[2] $ grep apples < foods | wc -l
terra[3] $ grep apples foods | wc -l
terra[4] $ grep -c apples foods
The tee Command (don’t worry about this)
 tee
- replicate the standard output
– cat readme.txt | tee myfile
stdin
tee
stdout
myfile
Unix Filename Rules
 Almost
any character is valid in a file name
– all the punctuation and digits
– the one exception is the / (slash) character
– the following are not encouraged
 ? * [ ] “ ” ’ ( ) &:;!
– the following are not encouraged as the first
character
 - ~
– control characters are also allowed, but are not
encouraged
 UPPER
and lower case letters are different
– A.txt and a.txt are different files
Unix Filename Extensions
 No
enforced extensions
– The following are all legal Unix file names
a
 a.
 .a
…
 a.b.c
 Remember
files beginning with dot are hidden
– ls cannot see them, use ls -a
.
and .. are reserved for current and parent
directories
Unix Filename Extensions

Even though Unix doesn't enforce extensions,
– “.” and an extension are still used for clarity
 .jpg for JPEG images
 .tex for LaTeX files
 .sh for shell scripts
 .txt for text files
 .mp3 for MP3’s
– some applications may enforce their own extensions
 Compilers look for these extensions by default
– .c means a C program file
– .C or .cpp or .cc for C++ program files
– .h for C or C++ header files
– .o means an object file
Unix Executable Files
 Executable
files usually have no extensions
– cannot execute file a.exe by just typing a
– telling executable files from data files can be difficult
 “file”
command
Use: file filename
Result: print the type of the file
Example: terra[1] $ file ~/.bashrc
.bashrc:
executable bash-shell script
 Filenames
and pathnames have limits on lengths
– 1024 characters typically
– these are pretty long (much better than MS-DOS
days and the 8.3 filenames)
Fixing Filename Mistakes

It is very easy to get the wrong stuff into filenames
– Say you accidentally typed
terra[3] $ cp myfile -i
– What if you type
terra[4] $ rm -i
 The shell thinks -i is an option, not a file
 Getting rid of these files can be painful

Creates a file
with name -i
There is an easy way to fix this...
– You simply type
terra[5] $ rm -- -i
– Many commands use “--” to say there are no more options
Filename Wildcarding (See section 3.1)

Wildcarding is the use of “special” characters to represent
or match a sequence of other characters
– a short sequence of characters can match a long one
– a sequence may also match a large number of sequences

Often use wildcard characters to match filenames
– filename substitution – generally known as “globbing”

Wildcard characters
* matches a sequence of zero or more characters
– Example: a*.c*
matches abc.c, abra.cpp,
? matches any single character
– Example: a?.c
matches ab.c, ax.c, but not abc.c
[...] matches any one character between the braces
– Example: b[aei]t
matches bat, bet, or bit, not baet
Filename Wildcarding (2)

Wildcard sequences can be combined
terra[6] $ mv a*.[ch] cfiles/
 mv all files beginning with a and ending with .c or .h into
the directory cfiles
terra[7] $ ls [abc]*.?
 list files whose name begins with a, b, or c and ends with .
(dot) followed by a single character

Wildcards do not cross "/" boundaries
– Example: csnow*c does not match csnow/codec

Wildcards are expanded by the shell, and not by the
program
– Programmers of commands do not worry about searching
the directory tree for matching file names
– The program just sees the list of files matched
Filename Wildcarding (3)


Matching the dot
– A dot ( . ) at
 the beginning of a filename, or
 immediately following a /
must be matched explicitly.
– Similar to the character /
– Example:
terra[8] $ cat .c*
cat all files whose names
begin with .c
As mentioned earlier, [....] matches any one of the
characters enclosed
– Within “[...]”, a pair of characters separated by “-” matches
any character lexically between the two
 Example:
lists all files beginning
terra[9] $ ls [a-z]*
with a character between
ASCII ‘a’ and ASCII ‘z’
Processes and Job Control
Figure 5-14
Job States
Foreground and Background (1)
 Unix
is a multi-tasking operating system
– some of these tasks are being done by other
users logged in
– some are being done by you in the
background
e.g. watching for incoming mail
 When
you run a task (a Unix command,
like ls or vi) it executes in the foreground
of your shell
– it has the “control" of your screen and
keyboard
Foreground and Background (2)
 If
you still want to use your shell command line
 terra[1]
$ a_heavy_task &
 [1] 13607
 terra[2] $
 When
you put a task in background
– task keeps running, but you continue to work at the
shell in the foreground
– if any output is done, it appears on your screen
immediately (can be confusing)
– if input is required, process prints a message and
stops
– when it is done, a message will be printed
Example ls –R /





(again)
Kick off a long command
ls –R / > listall
Suspend, Ctrl-Z
Send to background
bg
[1] ls –R / >listall&
Oops, still get errors! Now you have to kill a background
job:
kill %1
(or kill -9 %1)
[1] terminated
Restart it the right way, in background, with err redirect:
ls –R / > listall 2> /dev/null &
Foreground and Background (3)
 Explicit
background processes are needed
less often with windowing systems
– Just go to another window and run the
command
 But
explicit background processes are
used often ...
– A command needs a long time, you do not
want to close that window by accident
– Run a job at the background and logout
– netscape& will open a new window, but leave
the current shell window still available to use
A Simple Script



We use the following shell script to illustrate job control
Edit a file make_noise
terra[1] $ cat > make_noise
#!/bin/sh
while true
do
date
sleep 1
done
^D
terra[2] $ chmod u+x make_noise
make_noise then is a shell script repeats to print the
time for every second, until you terminate it using
Ctrl-c.
Job Control – Suspending Jobs
 csh,
tcsh, and bash allow you to manage
the running of different processes
 Suspending jobs
– the Ctrl-z special character stops the job
terra[1] $ make_noise
Fri May 16 14:14:43 EDT 2003
……
^Z
Suspended
terra[2] $ vi readme
^Z
Job Control - Monitoring Jobs
 The
"jobs" command shows which of your
jobs are running and/or stopped.
terra[3] $ jobs
[1] + Suspended
[2] + Suspended
 Here
make_noise
vi readme
there are two suspended processes,
the make_noise and a vi process.
Job Control – Resuming Jobs
 Putting
jobs back into the foreground:
– Use the "fg" command to move a job into the
foreground.
terra[4] $ fg %2
– Puts job number 2 into the foreground.
– Works with either a background or stopped
job.
 Putting
jobs into the background:
terra[5] $ bg %1
Job Control – Killing Jobs
 Jobs
can also be killed
– Use the Unix "kill" command
terra[6] $ kill %1
or if it won't die ...
terra[7] $ kill –9 %1
 Jobs
can be stopped and continued
terra[8] $ a_heavy_task &
terra[9] $ stop %1
terra[10] $ bg %1
Using ps (1)
 Jobs
are really just a special case of Unix
processes
 ps can list your current processes
terra[11] $ ps
PID TT
S
2312 pts/0 T
2296 pts/0 R
2313 pts/0 R
TIME
0:00
0:00
0:00
COMMAND
vi
tcsh
ps
Using ps (2)
 The
ps command takes a number of
options
-l
gives you a long listing of what is going
on
-u loginid tells you about loginid's
processes
use man ps to see more options
 kill
pid kills the process pid
– TERM signal will be sent to the process pid
– kill -9 or kill -KILL will send the KILL signal
– Use man –s 5 kill to find out more signals