Lesson 6-Using Utilities to
Accomplish Complex Tasks
Overview
 Creating and executing a script to list user information.
 Listing directories and files separately.
 Identifying changes made to files in a directory.
 Creating a complex script.
Creating and Executing a Script to List
User Information
 Creating a script.
 Executing a script.
Creating a Script
 A series of commands can be executed
repeatedly/automatically by placing them in a script file.
 Placing commands in scripts helps avoid errors and save
time.
 The vi editor can be used to create a file.
 The ":wq" commands are used to write to a file and return
to the shell.
Executing a Script
The commands in a file can be executed in two ways:
 Instructions can be given to the current shell to read the file
(source) and execute all the commands.
 The script file can be made executable, and a child shell can
be started to read the script file and execute the commands.
Executing a Script
 The command used to make the shell executable is “chmod
+x filename”.
 By default, the output of the script is redirected to the
screen.
 The output of the script can be redirected to a file.
Listing Directories and Files
Separately
Selecting only directories / files:
 The “ls –F | grep /” command is used to select only lines that
contain directory names.
 The “ls –F | grep / | column” command can be used to put the
output into columns.
 The “ls –F | grep –v / | column” command is used to select
only files.
Listing Directories and Files
Separately
Selecting Directories Using Multiple Utilities
Listing Directories and Files
Separately
Interpretation by grep Using –v option
Listing Directories and Files
Separately
Adding comments to scripts:
 The # (pound) sign is used to add comments to scripts.
 Ensure that the # sign is placed at the beginning of each line.
 The pound (#) sign cannot be used inside a long command
line.
Identifying Changes Made to Files in a
Directory
Collecting data about files in a directory:
 The “ls –l directory name” command can be used to provide
information about the files in a particular directory.
 The output of the code can also be redirected to another file
by the command "ls -l directoryname > filename”.
 The information includes the current status of each file, such
as permissions, date of modification, owner, etc.
Identifying Changes Made to Files in a
Directory
Comparing file information:
 The “comm” command is used to identify the common aspects
between two files.
 It requires two arguments – the names of the two the files
that are to be compared.
 The command gives three columns of output – lines unique in
the first file, lines unique in the second file, and lines in
common.
Identifying Changes Made to Files in a
Directory
Comparing file information:
 The “diff” command is used to identify the differences between
two files.
 It requires two arguments – the names of the two files to be
compared.
 The command gives three columns of output – lines unique in
the first file, lines unique in the second file, and lines common
to the two files.
Creating a Complex Script
 Determining the number of unique words in a file.
 Removing punctuation.
 Converting characters to lowercase.
 Putting each word on a line.
 Removing blank lines.
 Sorting the lines.
Determining the Number of Unique
Words in a File
The “uniq” utility is used to:
 Provide a list of unique words in a file.
 The number of times that each word is used.
 The number of unique words.
Determining the Number of Unique
Words in a File
 The uniq utility outputs both, unique lines as well as single
copies of any lines that are duplicate and adjacent.
 The utility deletes identical lines.
 The uniq utility deletes only those lines that are identical
and adjacent.
Determining the Number of Unique
Words in a File
The following things must be ensured for the uniq utility to
work properly:
 Punctuation and blank lines must be removed.
 Differences in case for the same word must be reconciled.
 The words must be one word to a line in a sorted order.
Removing Punctuation
Shell and Utility Interpretation
Converting Characters to Lowercase
Shell and tr Interpretation
Putting Each Word on a Line
 To remove duplicate words using the uniq utility, the data
must be modified to ensure that each word is on a line by
itself.
 The new line character “\n” is used to separate lines in a
file.
 The ASCII character code 012 can also be used to separate
lines in a file.
Putting Each Word on a Line
 The “man ascii” command is used to examine the ASCII
characters and their associated codes.
 TAB characters can also be replaced with new line
characters.
Putting Each Word on a Line
Using tr Utility to Replace Characters
Removing Blank Lines
A View from the Shell
Sorting the Lines
 The “sort” command can be used to sort the output such
that all lines containing the same word are on adjacent
lines.
 The “uniq –c” command can be used to remove all duplicate
lines.
 The “sort –rn” command can be used to list the most
frequently used words first.
Summary
 A series of commands placed in a script file can be
repeatedly executed, avoiding errors and saving time.
 The ls utility provides a list of all the files and subdirectories
in the current directory.
 The # sign is used to add comments to script files.
 The pipeline feature of UNIX is very useful and is central to
manipulating data effectively with UNIX utilities.