ITEC400 Week Three
ITEC400 Week Three
ITEC400 Week Three
Robert D’Andrea
Agenda
Administrative Issues
Link of the Week
Review Week Two Information
This Week’s Expected Outcomes
Next Lab Assignment
Break-Out Problems
Upcoming Deadlines
Lab Assistance, Questions, and Answers
Administrative issues
Call and/or email me if you don’t understand an
assignment, procedure, or just need a pointer
(614.519.5853).
Link of the week
APA Writing Style
APA Style writing guidelines will be utilized for
all lab assignment reports .
APA Style Sixth Edition
http://www.apastyle.org/learn/
APA Style Writing Workshop
http://www.franklin.edu/student-services/studentlearning-center/academic-support/workshops.html
Grading will focus mainly on capitalization,
punctuation, grammar, and citation.
Link of the week
Linux Forums Web Site
This web site allows individuals to post
questions about Fedora Linux and Redhat Linux. The
people that maintain this site are knowledgeable users
of these operating systems. If your experiencing
problems with commands or just want to learn more
about these systems, you can find this type of
information at this site.
http://www.linuxforums.org/forum/redhat-fedora-linuxhelp/73994-system-commands-not-working-unlesssbin.html
Course expected outcome
Week 3
Upon successful completion of this module, the
student will be able to:
Authors of UNIX
Create scripts using shell/Perl variables and
program control flow.
Authors of AWK
Use redirection and pipes to combine scripts and
executables.
Use man page system and find script tools.
Discuss Perl Language
UNIX Operating System
Authors of UNIX
Dennis Ritchie
Computer Scientist
Dennis Ritchie was an American computer scientist. He
created the C programming language and, with longtime colleague Ken Thompson, the Unix operating
system.
Born: September 9, 1941, Bronxville, NY
Died: October 12, 2011, Berkeley Heights, NJ
Education: Harvard University
Fields: Computer Science
Books: The C Programming Language, More
Awards: Turing Award, National Medal of Technology
and Innovation, IEEE Emanuel R. Piore Award
UNIX Operating System
Authors of UNIX
Ken Thompson
Computer Designer
Kenneth Lane "Ken" Thompson, commonly referred
to as ken in hacker circles, is an American pioneer
of computer science. Having worked at Bell Labs
for most of his career, Thompson designed and
implemented the original Unix operating system.
Born: February 4, 1943 (age 72), New Orleans, LA
Fields: Computer Science
Awards: Turing Award, National Medal of
Technology and Innovation,Tsutomu Kanai
Award, IEEE Emanuel R. Piore Award
Education: University of California,
Berkeley (1966), University of California,
Berkeley (1965)
UNIX Operating System
What is the next user interface going to be?
The textual (command line) and the visual
(graphical user) interfaces are the two most common
modalities used to support engineers in network and
system administration positions. The command line
interface is recognized as the first generation and the
graphical user interface is considered the second
generation. Currently, research is trying to determine
the next best interface. The command line interface is
known as, “under the hood” method of interacting
with the operating system.
UNIX Operating System
CLI Benefits
Manipulate textual data
Quick customization of data allows engineers the
ability to change data to another form
Excellent for filtering data on systems
Commands are rich, expressive, flexible, and
powerful
GUI Benefits
Reduces data overload
Simple filtering and manipulation of the data
Excellent for displaying trends in data
UNIX Operating System
Customizing the UNIX Shell
The UNIX operating system shell is a user
program that is executed by the kernel when you log in
to your terminal. As previously displayed, UNIX
operating systems usually provide several shells for
users to select from. The most common shells are the
Bourne Shell (sh), the C Shell (csh) and the Korn Shell
(ksh). Each of the shells listed vary in some way for
specific reasons.
Demonstrate: cat /etc/shells
Feature
Function
sh
csh
ksh
Job control
Allows processes to be
executed in the
background
No
Yes
Yes
History
substitution
Allows previous commands
to be saved, edited, and
reused
No
Yes
Yes
File name
completion
Allows automatic
completion of partially
typed file name
No
Yes
Yes
Command
line editing
Allows the use of an editor
to modify the command
line text
No
No
Yes
Command
aliasing
Allows users to rename
commands
No
Yes
Yes
UNIX Operating System
UNIX Operating System
Changing shells
It is possible to invoke a different shell, while
within another shell, you just type the name of the shell
you want to execute after the terminal prompt, ksh, csh,
or sh.
Command renaming
Both ksh and csh provide command name
aliasing, which allows you to change a command name.
Command aliasing can save you time and keystrokes
by renaming frequently issued long commands. Two
pieces of information are needed. The command you
want to alias, and the alias you want to use to refer to it.
alias g=‘grep’
UNIX Operating System
File name completion
If you dislike lengthy typing, csh and ksh
provide a feature the allows you to type partial file
names. By typing a partial file name and pressing the
ESCAPE key once for csh and twice for ksh, you can
retrieve the file.
If the beginning characters for a file are entered
and the file does not exist, the shell will beep at you.
Command history substitution
Both ksh and csh maintain an ordered list of
commands that you issued, allowing them to be
retrieved from the list. Typing the history command
will displayed the list of previous commands issued.
UNIX Operating System
Set ksh history command size
HISTSIZE=60; export HISTSIZE
Set csh history command size
set history=70
Retrieve commands using C Shell
Recall the last command
!!
Recall the command number nine
!9
UNIX Operating System
UNIX File System
A disk drive is a device that stores data by making
electrical imprints on a magnetic surface, optical, or
mechanical changes to a surface layer of one or more
rotating disks.
A hard disk consists of one or more circular
aluminum platters of which either or both surfaces are
coated with a magnetic substance used for recording the
data. For each surface, there is a read-write head that
examines or alters the recorded data. The platters rotate on a
common axis; typical rotation speed is 5400 or 7200
rotations per minute, although high-performance hard disks
have higher speeds and older disks may have lower speeds.
The heads move along the radius of the platters; this
movement combined with the rotation of the platters allows
the head to access all parts of the surfaces.
UNIX Operating System
UNIX Operating System Hard Drive
UNIX Operating System
The Power of Memory
Keeping data in memory is the current
direction technology is headed instead of
pulling data in from a disk during processing.
Database companies are interested in getting
their share of the in-memory database market.
Industries actively involved are Aerospike,
Microsoft Research, IBM, Oracle, VoltDB, and
MemSQL.
UNIX Operating System
The Power of Memory
In-memory databases is all so know as
main memory databases. The in-memory
databases speed up processing in two basic
areas.
• Data available in memory
• The time lag caused by fetching the data
from a disk
UNIX Operating System
The Power of Memory
A more basic delay retrieving data from
disk is the way data is stored on disk in blocks.
To get to a particular piece of data, the computer
must import the whole block from disk, decode
it, run the process on the piece it wants, reencode the blocks, and send the block back
where it came from.
UNIX Operating System
The Power of Memory
When you store data in memory, you
eliminate the overhead associated with block
data. When data resides in-memory, you can be
much more efficient.
One database vendor sees in-memory
databases as the wave of the future for online
transactions. They envision the in-memory
database handling a third of the database world.
UNIX Operating System
How a UNIX file system functions
Every item in a UNIX file system can be
identified to belong to one of the four types:
• Ordinary files contain text, data, or program
information. They cannot contain another file or
directory. They can be viewed as a one-dimensional
array.
• Directory contain files, and other directories. A
directory is a file that has one line for each item
contained within the directory. Each line in a
directory file contains the name of the item, and a
numeric reference to the location of the item. The
numerical reference is called, i-node, which will be
discussed at a later time.
UNIX Operating System
How a UNIX file system functions
Every item in a UNIX file system can be
identified to belong to one of the four types:
• Link is a pointer to another file. As a review, a
directory is a list of the names and i-numbers of
files. A directory entry can be a Hard Link or a
Symbolic link. Both of these topics will be
addressed at a later time. Special files are files
representing input/output (I/O) devices. A special
file can be a printer, a disk drive, or a terminal.
Because UNIX treats all such devices as a file, it
achieves a greater degree of compatibility between
I/O devices and I/O of ordinary files, providing
more efficient use of software.
UNIX Operating System
Types of File and Directory Access
Access
File Meaning
Directory
Meaning
r
View file contents Search directory contents
w
x
Alter file contents
Alter directory contents
Run executable file Make your current
directory
-rwx------ Owner (columns 2-4) 700 (111000000)
----rwx--- Group (columns 5-7) 070 (000111000)
-------rwx Other (columns 8-10) 007 (000000111)
UNIX Operating System
File Descriptors
A file descriptor is generally an index for an
entry in a kernel-resident data structure that contains
information on all open files. Each process on the
system has its own file descriptor table. A user
application passes the abstract key to the kernel through
a system call, and the kernel accesses the file for the
application.
UNIX Operating System
File Descriptor
Keyboard
#0 stdin
Program
#1 stdout
Display
Screen
#2
stderr
UNIX Operating System
wc `ls` commands
The word count (wc) command reads either
standard input or a list of files and generates one or
more of the following statistics: newline count, word
count, and byte count.
A snippet of output from the wc –l `ls`
command:
19 xyz.sh
2 zombie
24 zombie.c
9890 total
UNIX Operating System
Shell and Programs Access
To run a shell script, you will need read (r)
and execute (x) access (r-x). The read access mode is a
binary 4. The execute access mode is a binary 1.
To run a binary executable program, you
will need execute (x) access (--x). The execute access
mode is a binary 1.
UNIX Operating System
Character
Action
>
Redirect standard output
>&
Redirect standard output and standard error
>>&
Append standard output and standard error
<
Redirect standard input
UNIX Operating System
Shell redirect
ls –a > /tmp/output 2>&1 (stderr redirected to stdout)
> is equivalent to 1>
(stdout (write) redirect
symbol))
< is equivalent to <0
(stdin (read) redirect
symbol)
UNIX Operating System
Pipe command (command1 | command2)
Command2 does not start executing
until command1 has finished, and a sufficiently large
scratch file is required to hold the intermediate results
as well as whatever work space each task required.
command1 > tempfile
command2 < tempfile
rm tempfile
UNIX Operating System
Pipe command
UNIX Operating System
What is a data structure?
A data structure is a specific way of storing and
organizing data in a computer so that it can be
accessed with high efficiently. Data structures can be
used as a single place for storing unrelated
information.
A data structure is a particular way of storing and
organizing data in a computer so that it can be used
efficiently.
Different kinds of data structures are suited to
different kinds of applications, and some are highly
specialized to specific tasks.
Some common data structures: array, hash table, linked
list, queue, and stack.
UNIX Operating System
What is a shared memory?
Shared memory is memory that may be
simultaneously accessed by multiple programs with an
intent to provide communication among them or avoid
redundant copies. Shared memory is an efficient means
of passing data between programs. Depending on
context, programs may run on a single processor or on
multiple separate processors.
UNIX Operating System
What shared memory looks like?
UNIX Operating System
What shared memory looks like?
UNIX Operating System
What shared memory looks like?
A shared memory arena file is a regular or shared
memory device file /dev/shm is mapped and
shared by one or more processes.
UNIX Operating System
What is a regular expression?
A regular expression (abbreviated regex or
regexp) is a sequence of characters that forms a search
pattern, mainly for use in pattern matching with strings,
or string matching, i.e. "find and replace"-like
operations. The concept arose in the 1950s, when the
American mathematician Stephen Kleene formalized
the description of a regular language, and came into
common use with the Unix text processing utilities ed,
an editor, and grep (global regular expression print), a
filter.
UNIX Operating System
Introduction to Perl
Perl - Practical Extraction and Report Language
Originally developed by Larry Wall, a linguist.
Perl is 21 years old and Perl 5 is 14 years old.
Perl is a simple language
- Compiles and executes like a shell
script or a batch file
- Perl doesn’t impose special growth
limitations on arrays and data strings
- Perl is a composite of C, AWK, and
Basic
- Originally developed to process text and
automating tasks
UNIX Operating System
Perl’s range of flexibility
 System administration
 Web development
 Network programming
 GUI development
Major features
 Procedural Programming
Sequence or unstructured statements
Includes routines, subroutines,
methods, or functions
 Object Oriented Programming
Module uses “objects” and their
interactions to design applications and
computer programs.
UNIX Operating System
Major features (continued)
Powerful built-in support for text
processing
Large collection of third-party
modules.
Variable
A variable is a storage location that can hold any
of various kinds of value, as a program sees fit. This
storage location is an abstract area known as
namespaces.
UNIX Operating System
Perl language
Perl is a free-form language, but is not free of
form. Perl is a free-form language in which you can put
spaces, tabs, and new lines anywhere you like, except
where they aren’t allowed.
Whitespace cannot be placed within a token. A
token is a sequence of characters with a unit of
meaning, like a word in the English language. But
unlike the typical word, a token can contain other
letters and characters as long as they are kept together.
UNIX Operating System
Scope of a Perl variable
The scope of a variable means how far away you
can see a variable, looking through one. Perl has two
visible mechanisms.
 Dynamic scoping of local variables, meaning that the
rest of the block, and any subroutines that are called
by the rest of the block, can see the variables that are
local to the block.
 Lexical scoping of “my” variables, meaning that the
rest of the block can see the variable, but other
subroutines called by the block cannot see the
variable.
UNIX Operating System
Variable and constant
A variable is a named object that resides in RAM
memory and is capable of being examined and
modified. A variable is used to hold information critical
to the operation in an embedded system.
A constant is a named object that resides in
memory (usually in ROM) and is only capable of being
examined.
UNIX Operating System
AWK and Perl
The AWK utility is an interpreted programming
language typically used as a data extraction and
reporting tool. It is a standard feature of most Unix-like
operating systems.
AWK was created at Bell Labs in the 1970s, and
its name is derived from the family names of authors –
Alfred Aho, Peter Weinberger, and Brian Kernighan.
The power, terseness, and limits of early AWK
programs inspired Larry Wall to write Perl just as a
new, more powerful POSIX AWK and gawk (GNU
AWK) were being defined.
UNIX Operating System
AWK
Alfred Aho
Computer Scientist
Alfred Vaino Aho is a Canadian computer scientist best
known for his work on programming languages,
compilers, and related algorithms, and his textbooks on
the art and science of computer programming.
Born: August 9, 1941 (age 74), Timmins, Canada
Doctoral advisor: John Hopcroft
Residence: United States of America
Education: University of Toronto, Princeton
University
Awards: IEEE John von Neumann Medal
UNIX Operating System
AWK
Brian Kernighan
Computer Scientist
Brian Wilson Kernighan is a Canadian computer
scientist who worked at Bell Labs alongside Unix
creators Ken Thompson and Dennis Ritchie and
contributed to the development of Unix. He is also
coauthor of the AWK and AMPL programming
languages.
Born: January 1, 1942 (age 73), Toronto, Canada
Fields: Computer Science
Education: Princeton University, University of Toronto
UNIX Operating System
AWK
Peter J. Weinberger
Computer Scientist
Peter Jay Weinberger is a computer scientist best
known for his early work at Bell Labs. He now works
at Google. Weinberger was an undergraduate at
Swarthmore College, graduating in 1964.
Born: August 6, 1942 (age 73), New York City, NY
Known for: AWK
Doctoral advisor: Derrick Henry Lehmer
Education: Swarthmore College, University of
California, Berkeley
Fields: Number theory, Computer Science
UNIX Operating System
Why is awk language so important?
Awk language is an excellent filter and report
writer. Many UNIX utilities generate rows and columns
of information. Awk is an excellent tool for processing
rows and columns, and it is easier to use awk than other
conventional programming languages. Perl recognized
the importance of awk, so it was included and enhanced
in Perl.
Example: awk '{print $1}'
UNIX Operating System
Points of interest
who | sort > /tmp/test_file.txt
The output of the who command is piped to the
sort function and written in ascending order in the
/tmp/test_file.txt file. The “who” and “sort” commands
both write and read an intermediate file.
Test one liners:
who
who | sort
who | sort | /tmp/test_file.txt
UNIX Operating System
Points of interest
Knoppix software was designed to be used as a
Live CD because of specific features that make it’s
performance and stability very suitable. It has been
noted in several articles that Knoppix works best from
a Live CD. Knoppix enthusiasts have attempted to
install this software on a hard disk and encountered
problems in the process. These problems are most
pronounced when installing updates and new software.
UNIX Operating System
Demonstrate
The Advanced Scripting lab assignment requires
two shell scripts to be written.
srch.sh
srchfile.sh
Execution of srch.sh and srchfile.sh
Case #1: ./srch.sh <pattern> <file name / directory
name>
Case #2: ./srch.sh <pattern> <.>
The srch.sh script will call the srchfile.sh script
to perform a specific task. The srchfile.sh searches for
a file with a pattern and outputs the matching
information to standard output. After all directory
entries have been read, control is returned to the main
script, srch.sh.
UNIX Operating System
Moving Around in UNIX
history
who –r
uname –n
cat
chgrp
chown
hostname
kill
Break-out problems
1. scalar@ARGV
2. $ARGV[0]
3. filter
4. unless
5. $NUMBER
6. exit 1
7. $ARGV[1]
8. %
(%directory)
9. $
($quote)
10. @ (@names)
11. Regular expressions
12. tr [a-z] [A-Z] < foobar > /tmp/foo
Hands-On-Information





Lab Assignment 2-1, Simple Shell Scripting, due
January 24, 2016.
Lab Assignment 3-1, Advanced Scripting, due
January 30, 2016.
Read Chapters 3 and 4 in Essential System
Administration text.
Read Module Two listed under the course Web site.
Everyone should have received a Shell Quick
Reference document and script logic for Lab
Assignment 2-1.
After Class Assistance
Questions?
Comments?
Concerns?
After each Franklin Live session, I will remain on the
session to provide assistance unless otherwise
indicated.
Lab assistance, questions, and
chat time