The UNIX Operating System 1 TCS Confidential 1 1 UNIX Operating System Objectives : Learn about the Unix Operating System and develop the skills required to build software applications in the Unix Environment. 2 TCS Confidential 2 2 Contents : 1. Introduction & History of UNIX 2. File System 3. Basic Utilities 3 4. Shell Features 5. Advanced Utilities 6. Communication Features 7. System Calls TCS Confidential 3 3 Chapter 1 Introduction and History of 4 UNIX TCS Confidential 4 4 Introduction and History of UNIX 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. Evolution Development Standard release- AT&T versions BSD UNIX Other implementations of UNIX 5 Features of UNIX UNIX Philosophy UNIX operating system UNIX Kernel Programming Environment Design Principles Limitations TCS Confidential 5 5 Evolution Primarily influenced by MULTICS MULTICS - Developed for GENERAL ELECTRIC 645 mainframe computer 6 - Interactive operating system - Batch - mode characteristics UNIX Operating System - First Version developed in 1969 by Ken Thompson on DEC PDP Series ( PDP - 7 ) - Business, Scientific and Industrial Users TCS Confidential 6 6 Development * Originally written in assembly language for PDP-7 * Transportability facilitated by Thompson who 7 developed the B language * B language modified by Ritchie and renamed as C language * Thompson (1980) and others wrote UNIX in C which could be ported to any computer TCS Confidential 7 7 Standard release - AT&T Versions Version 3 • • Version 6 • • • • Version 7 Version 8 • • • Written in C Moved to PDP - 11/45 and PDP - 11/70 Released in 1975 8 use by universities only For Released in 1978 First commercial version but primarly used in universities 32V - VAX version UNIX Support Group(USG) takes responsibility Research Group still busy TCS Confidential 8 8 Standard release - AT&T Versions …Contd... System III • • • • Released in 1981 Commercial orientation Over 100 manufacturers of UNIX-based micro , mini and mainframes UNIX users > 100,000 • • Released in 1983 9 Enhancement of System III • • Released in 1984 Enhancements of System V , Release 1 System V Release 3 • Released in 1988 System V Release 4.0 • System V Release 1 System V Release 2 TCS Confidential 9 9 BSD UNIX (from Berkeley Software Distributions) * UNIX's entry into University of California, Berkeley Campus during 1976-77 * Inputs of UCB Faculty/researchers incorporated in System V of AT&T UNIX UNIX 3BSD • • • Released in 1978 Developed by Bill Joy and Ozalp Baboglu Added 10 Virtual memory, Demand paging to the VAX version 32V UNIX 4BSD • • • UNIX 4.1 BSD • DARPA - funded Development Provided support for DARPA TCP/IP C-Shell , ex/vi , new compilers for PASCAL and LISP were added Released in 1981 UNIX 4.2 BSD • Released in 1983 UNIX 4.3 BSD • Latest Release TCS Confidential 10 1 Other Implementations of UNIX (based on Version 7, System III, System V of AT&T UNIX or UNIX 4.2 BSD) Xenix IBM PC/IX Zeus DEC Ultrix HP-UX UNIX Elxsi UNIX Pyramid UNIX Data General UNIX Perkin Elmer UNIX MassComp UNIX NBI UNIX Amdahl UNIX Microsoft’s UNIX operating system to run on microcomputers IBM PC Zilog Computers DEC 11 HP SUN ELXSI Pyramid Data General Perkin Elmer Mass Computers NBI Amdahl TCS Confidential 11 1 UNIX Features Multitasking Multi-user Performing tasks simultaneously rather than sequentially e.g., While printing a document , start editing another document Each task is considered to be a process Several users can use the same computer 12 simultaneously i.e , more than one keyboard and terminal can be connected to one computer Highly cost-effective Portability Easier to modify the UNIX system code for installation on a new computer rather than write from scratch a new operating system TCS Confidential 12 1 UNIX Features UNIX System supplied tools Integral utilities Basic Operation of Operating 13 like Command system interpreter or Shell Tools Text print formatting like nroff and typesetting troff TCS Confidential 13 1 * * * * * * Available on micros, minis and mainframes UNIX Communication and electronic mail • Communication between different terminals hooked to same computer • Communication between computers of different sizes and types in different locations as far away 14 as other countries Incorporation of third party application programs Hierarchical File System On-Line Help facility Example : man <Command> Library of application software TCS Confidential 14 1 The UNIX Philosophy Make each program do one thing well Expect the output of one program to become the input to another Suited for prototype development i.e., design and build easily 15 Modular approach i.e., be prepared to throw and rebuild Usage of tools Building essential tools TCS Confidential 15 1 The UNIX Operating System Kernel( Heart of Unix Operating System) Shell Interpreter Tools and Applications Kernel Schedules tasks 16 and manages data storage Shell A program that • interprets the user commands/requests • calls programs from memory and • executes them one at a time or in a series (called a pipe) Tools & Applications Specific capabilities added on to the OS TCS Confidential 16 1 The Users Shells and Commands Compilers and Interpreters System Libraries System Call Interface to Kernel Signal File System CPU Scheduling Terminal Handling Swapping Page Replacement Character I/O System 17 Block I/O System Demand Paging Terminal Drivers Disk & Tape Drivers Virtual Memory Kernel Interface to the Hardware Terminal Controller Device Controller Memory Controller Terminals Disks & Tapes Physical Memory TCS Confidential 17 1 The UNIX Kernel Networking Services File Management And Security Input / Output Services Date and Time Services 18 UNIX SYSTEM Signal Handling KERNEL Process Scheduling System Memory Administration Management and Accounting TCS Confidential 18 1 The Programming Environment * A GOOD operating system !! • • • * Simple yet powerful Allows building of tools, writing of small programs and putting them together 19 A rich & productive environment A PORTABLE operating system !! • Written in C • Runs on a range of computers • Source code is available TCS Confidential 19 1 The Design Principles * * * Designed for Programmers Interactive Multiple Processes can be initiated Foreground Process Background Process * * * * * * * 20 Methods to Co-ordinate Process Signal to communicate with processes Files, Directories, Devices treated as files Tree structured directories to hold files Emphasis on program development facilities Sources available on-line Simplicity before efficiency TCS Confidential 20 2 Drawbacks of UNIX * Designed for computer professionals * Not a real-time operating system 21 * File Security depends on owner * Defective programs may keep running TCS Confidential 21 2 Chapter 2 22 File System TCS Confidential 22 2 The UNIX File System 1. 2. 3. 4. 5. 6. 7. 8. 9. File System Layout Concepts of file Features of File system Starting and Ending a session 23 File Management utilities Directory Management File operation Mountable File System File security TCS Confidential 23 2 File Systems in UNIX Disks are partitioned into File Systems. Logical blocks of 512 bytes or multiples Arranged to form a hierarchical structure Convenient to manage data File System Layout Boot block Super block Inode list Data block 24 Boot block - generally occupies the first sector may contain bootstrap code - Information about the file system How large, where free and other details - Contains a list of inodes One inode is the root inode - Contains file and administrative data Allocated data can belong to that block only Super block Inode list Data block TCS Confidential 24 2 Classification of files in UNIX - Ordinary Files - Directory Files - Special Files25 - Standard Files TCS Confidential 25 2 Concept of Ordinary Files * A File in UNIX is a Stream of bytes ( 8-bits) Kernel does not impose a structure on files * File may contain text, data and even machine language code Examples Text Files : Lines of ASCII characters separated by a new-line 26 * Commands : Sequence of commands interpreted by UNIX text Data : File containing data as stream of bytes Executable : File containing machine language instructions The files format remains unchanged but only the way the program interprets it is different TCS Confidential 26 2 Concept of Directory Files * Gathering together related files in a common place * Every file is assigned to a Directory * Directories have names (maximum of 14 characters) * A file within a Directory can itself be another Directory (A Subdirectory) Internally a Directory is just a file that contains a list of file names 27 their Inode Numbers and a pointer to the actual file on the disk * * An upper limit of around 35 sub -directories Examples / ( Forward slash ) Prime or Root Directory Note - in DOS it is \ ( Backward slash ) /usr Directory for the user /usr/trg1 Sub-Directory for trg1 under Directory usr TCS Confidential 27 2 Concept of Special Files * Used to reference Physical devices such as terminals, disks, floppy drives and tape drives * Read and written like ordinary files * Requests cause activation of the associated Physical Device * Device Drivers associated with each file 28 * Types : Character file Terminal (tty0, tty1 ) Block file Disk Drives (hd0,fd0) Transmits data Character by Character Transmits data in 512 or 1024 byte chunks * Major and Minor numbers identify the hardware link Major No.s : Indicates type of device Minor No.s : Different instances of the device TCS Confidential 28 2 Concept of Standard Files * * * * • * Helps display information on the screen Special Names for Communication channels Keyboard input channel is called Standard input (stdin) file id is 0 Terminal Screen output is29called Standard output (stdout) file id is 1 Diagnostic error messages (generated by a program) are sent to Standard error (stderr) file id is 2 ( shown on terminal screen ) All three files are open by default at the time of login TCS Confidential 29 2 File Names * * * * * * A means for the system to identify a file Note : Unix is case sensitive User accesses a file by a user-assigned file name Can be a sequence of characters (a maximum of 1430) May be identified by two or more names (Multiple links to a file) Internally assigned a unique inode number (for a File System) Different files can be grouped under a directory TCS Confidential 30 3 Path Names Absolute Path name * e.g., A file is identified by the path name from the root /usr/trg/c/test.c where • • • test.c is an ordinary file usr, trg, c are directories trg is a sub-directory under usr 31 Relative path name * UNIX keeps track of the user's current directory * If a "/" does not precede a file name then the name interpretation begins with the current directory e.g., If current directory is /usr/trg then the file could be just referenced as c/test.c TCS Confidential 31 3 Features * Hierarchical * Security on each file - Owner - Group - All others * Separate security for - read 32 - write and - execute * Removable * File Independence - * Time stamp on each file - Modification time - Access time TCS Confidential 32 3 Unix dev bin lib / bin console lp0 tty0 src sh csh --Libc.a --- uch 33 tmp man --- bin local lib --usr include spool etc spell troff --- *.h tmac passwd group init --- lib troff --tmp --- TCS Confidential 33 3 Standard File Hierarchy /bin Basic Unix utilities cp, mv, ln /dev Special I/O device files fd0, lp0 /etc Administrative Programs passwd, shutdown /lib 34Unix Libraries used by libc.a /usr/bin Unix utilities cal, bc,spell /usr/adm Administrative commands and files adduser /tmp Temporary files created on error conditions TCS Confidential 34 3 /usr/games - Game Programs /usr/include - /usr/lib - Include files for language procedure Examples : C-header files stdio.h, math.h Archive libraries Example : troff /usr/mail - /usr/news - News files /usr/spool - Spool files /usr/tmp - Temporary files /usr/src - Program Source Files 35 Mail files Example : mailbox TCS Confidential 35 3 Security and Access Methods * * Three levels of access User/Owner, group, others Three types of access on Files and Directories Read, Write, Execute Access Mode Ordinary Directory Read Examination of File Contents 36 Listing of files within Directory Write Allows changing of file contents Creating new files within Directory Execute Executing file as a command allowed Searching the Directory allowed TCS Confidential 36 3 Starting and Ending a Session login : User can type his name and password to identify himself login command can be used as $ exec login to log-on onto another user account after identifying yourself in response to prompts for user name and password su setuser 37 This is used to become another user or super-user provided the password is known. e.g., $su Prompt the user for the superuser password $su - trg2 Prompt the user for the password of user trg2 $su - trg2 -c "ls -l" Temporarily changes to trg2 and executes the command ls -l and comes back to the original user TCS Confidential 37 3 Starting And Ending A Session …Contd…. passwd Change the password for the user e.g., $ passwd Prompt you for old password and new password logout 38 This command exits or logs-out from the current user and executes the file .logout before coming out e.g., $ logout or $ exit or $ <ctrl-d> exits from the current login TCS Confidential 38 3 File Management Utilities Directory Management Operation cd pwd mkdir rmdir mvdir File Comp. Security cmp comm passwd crypt chown chgrp umask chmod 39 File contents File compression Mountable file cat ls wc file pack unpack mount umount Copy, Move Remove & Time cp ln mv rm touch TCS Confidential 39 3 mkdir creates a new directory rm removes a file rmdir removes a directory du displays40 disk usage df displays number of free block touch updates the time of last modification find locates files that match certain area file displays the type of file pwd displays full pathname of current directory TCS Confidential 40 4 Command Structure General Structure: Command Name [Options] [Arguments] E.g., Command Name Tells the shell what to do (in this case list files) Options Control how the 41command will work (in this case ask for a long listing) Arguments on which the command works (in this case the directory usr/trg ) ls -l /usr/trg TCS Confidential 41 4 Directory Management cd cd.. cd. e.g., $ cd /usr/trg/c $ cd .. $ cd ./c or $ cd c $ cd mkdir $ mkdir pathname Change working Directory Parent Directory Current Directory ( current Directory is c ) ( current Directory is trg ) ( current Directory is again c ) 42 ( home directory - in this case /usr/trg) Make a Directory Makes Directory in 777 mode Write permission should at least be permitted for owner in parent Directory e.g., $ mkdir /usr/trg2 ( makes directory trg2 ) TCS Confidential 42 4 rmdir * rm -r Remove a Directory $ rmdir pathname Directory should be empty, or else (recursively remove) 43 e.g., $ rmdir /usr/trg2 (removes directory trg2) pwd Print Working Directory TCS Confidential 43 4 File Contents cat Concatenate & Print on screen or printer $cat [Options] [Arguments] Options - take input from stdin -n no. of output lines -s squeeze adj. blank lines -v enable display of non-printing characters -b used with -n to avoid numbering blank lines 44 e.g., $ cat try.c Display the contents of try.c on the screen $ cat Takes input from stdin i.e. keyboard and displays on screen TCS Confidential 44 4 $ cat f1 > f2 $ cat f2 > f3 $ cat f4 >> f3 $ cat try[0-3] > final $ cat test* > report Takes input from file f1 & puts it on file f2 f3 contains the contents of f1 Appends the contents of f4 to 45 file f3 The file final contains contents of try0, try1, try2 try3 The file report contains all files beginning with test TCS Confidential 45 4 Is[Options] List the Directory Contents Options -1 number one single column output -l long format (II also used) -a -s all entries including dot files 46 gives no. of disk blocks -i inode no. -t ordered by modification time recent first recursively display all directories, starting specified or current directory TCS Confidential 46 4 $ Is -I List the files along with the protection bits and the user $ Is -a List the files starting with .and..also $ Is -1 symtab.c symtab.o treegen test $ Is -I -rw-r—r— 1 smj proj1 30766 Mar 3 15:08 symtab.c -rw-r—r— 1 smj proj1 8759 Mar 3 15:12 symtab.o 47 Mar 3 15:23 treegen -rwxr-xr-x 4 smj proj1 40743 drwxrwxr-x 1 smj proj1 53 Mar 1 09:15 test $ Is -a . .. .profile .cshrc symtab.c ... $ Is -iI 10936-rw-r—r—I smj proj1 3076 Mar 3 15:08 test.c 10936 - inode number of file test.c TCS Confidential 47 4 wc Word Count $wc [Options] filename Options e.g., nl Display no. of lines, words, characters -I Display no. of lines -w Display no. of words -c Display no. of characters 48 $ wc test.c 20 200 5678 20 - lines 200 - words 5678- characters no. of lines in the file and temp’ly lists out the file Similar to wc -l < filename > TCS Confidential 48 4 file Determine file types $file [Options] [Arguments] Options -f filelist Normal File Types C49program text assembler program text commands text ASCII text English text e.g., $ file test.c C Program test TCS Confidential 49 4 cp copy a file -i - user interactive mode e.g., $ cp test.c test.c.bak test.c and test.c.bak contain the same contents Extra disk storage 50 In Create link e.g., $ ln first.c second.c The file is referenced by two different names No Extra disk storage TCS Confidential 50 5 mv Moves or renames files and directories -i interactive mode e.g., rm $ mv old.c new.c Renames the file old.c as new.c Deletes the indicated file(s) files rm -i -f -r touch e.g., removes files and directories remove interactively 51 forcible remove remove recursively • Dangerous • used in conjunction with -i Updates access, modification or change times of a file -a update access time -m update modification time -c prevents creating the file $ touch f1 * The current system date & time stamp is put on the file f1 * If f1 does not exist then it is created with 0 bytes TCS Confidential 51 5 File Comparison cmp number and Compare two files If files are same no output is sent to the terminal, or else The line the byte at which the first difference occurs is reported -s Outputs nothing Registers return code Return code 0 1 2 52 if files are identical if files are different on error e.g., $ cmp test1 test2 test1 and test2 differ in char 36 line 3 $ cmp -s test1 test2 $ echo $status outputs 1 indicating that the files are different TCS Confidential 52 5 diff - Reports more than one differences -b -e $diff [Options] file1 file2 Ignores trailing blanks Gives a list of ed commands so as to convert file1 into file2. 53 e.g., $ diff test1 test2 Outputs: n1 a n3,n4 n1,n2 d n3 n1,n1 c n3,n4 where * * n1 ,n2, n3 ,n4 are line numbers a ,d, c means append, delete ,change respectively TCS Confidential 53 5 comm Display common lines $comm -[123] f1 f2 Prints a three column output: - lines that occur only in f1 - lines that occur only in f2 - lines that occur in both comm -12 - prints 54 lines common to the two files comm -23 - prints only lines in the first file but not in the second comm -123 - prints nothing e.g., $ comm test1 test2 Reports the common lines between files test1, test2 and reports the lines differing $ comm -12 test1 test2 Prints line common to both TCS Confidential 54 5 File Compression pack Compress the file $ pack <filename> e.g., $ pack try - Creates a file try.z which is packed - Normally the executables are packed - The size is reduced by 25 - 40 % 55 unpack or pcat e.g., Uncompress packed file $ unpack try.z or $ pcat try.z unpacks the file try.z TCS Confidential 55 5 Mountable File System mount e.g., umount Associates a directory with a device Mounting a floppy on the root file system Dissociates directory from the device 56 e.g., $ mount /dev/fd096 /mnt $ umount /mnt Mounts the floppy on the directory /mnt Dissociates /mnt from the floppy TCS Confidential 56 5 File Security passwd To change the password chown To change the ownership of the file 57 $ chown owner filename e.g., $ chown trg2 test.c * Initially the owner is trg1 * Only the owner or the superuser can change the ownership of the file TCS Confidential 57 5 chmod change the permissions of the file $ chmod who op permission <filelist> who op permission a, u, g, o +, -, = r,w,x all, user, group, others + add, - remove, = set r read, w write, x execute e.g., 58 $ chmod a=rw test.c * users, group, others have read and write permissions $ chmod u+r, g+w, o+x test.c * read for users write for groups execute for others $ chmod 777 test.c * Sets read, write, execute Permissions TCS Confidential 58 5 umask Set file creation mode mask $ umask nnn (nnn set file creation mode) umask can also be set as a shell variable 59 e.g., umask 022 - Files normally created with 777 mode is assigned 755 permission The value of each digit is subtracted from the corresponding "digit" specified by the system for the creation of a file. TCS Confidential 59 5 File Operation tail Displays the last lines of file options : -n (n= no. of lines) e.g., $ tail -30 test.c Displays the last 30 lines of file test. c head Displays the top lines of file e.g., $ head -10 test.c Displays the first 10 lines of test.c split Splits the file into different files as specified by the number of lines e.g., $ split -20 test.c 60 Splits the file test.c in blocks of 20 lines and creates files xaa, xab, xac and so on, such that xaa has first 20 lines of test.c xab has the next 20 lines of test.c ... The file test.c is unaffected $ split-20 test.c try Generates files as tryaa , tryab , tryac paste Joins the two or more files horizontally e.g., $ paste xaa xab File xaa and xab are joined horizontally and output to the terminal TCS Confidential 60 6 Chapter 3 61 BASIC UTILITIES TCS Confidential 61 6 Basic Utilities 1. Line Editor Ed 2. Visual Editor Vi 3. 62 Debuggers 4. Profiling Tools 5. C-Beautification TCS Confidential 62 6 The Line Editor - ed Invoking ed: $ ed filename Prompt : Ed has no formal prompt P - Prompts with a “*” This is a toggle function Append: a - Appends given text after current line w - Writing to the file q - Quit 63from ed Write : Exiting ed : Example : $ ed test.c ? test.c P *a <————> <————> <————> *w *q The edit buffer can be written to some other file also as *w filename TCS Confidential 63 6 Pattern Matching abc abc a*bc abc, aabc, aaa...bc a.c abc, aac, acc, a.*c abc, axyc, a$+pmc, abcdefgc a[b2m]c abc, a2c, amc a[0-2b-d]c adc a0c, 64 a1c, a2c, a[^0-2]c a3c, a5c, axc, .... ayyyc axc, a[xy]*c ac, axc, axxc, a\{3,\} aaa, aaaa - 3 or more a’s a\{3,5\} aaa, aaaa, aaaaa between 3 & 5 a’s a$c abc, acc, TCS Confidential 64 6 Adding and Replacing Text.. Command Explanation Usage p print current line (s) 1,3p l list current line(s) (display invisible characters like tabs etc.) 2,4l n print line(s) with 65 number before it 1,5n a append text after current line 4a insert text before current line 3i i TCS Confidential 65 6 Traversing within a file Command Explanation . Pointer to current line .= Gives line number n Current line is ‘n’ +n Move up by n lines -n Move down 66 by n lines $ points to the last line , short for 1,$ ; short for .,$ /.../ forward search for string of characters enclosed between slashes Examples *- Previous line * - Current line *$= Prints total no. lines but does not changecurrent line TCS Confidential 66 6 Modifying Text Command d c m t j u Explanation delete line(s) in text change line(s) in text move line(s) in text copy line(s) in text join lines in text undo last command Usage 2,5d 2,4c 2,3m5 2,3t5 2,3j Note : The above commands can be easily remembered by associating them with the first characters of their action. 67 Substitute Commands (n,n)s/oldstr/newstr s/oldstr/newstr/g s/oldstr/newstr/p s/oldstr/newstr&/ s/oldstr/%/ - replaces oldstr with newstr (n,n) range of lines - optional all oldstrs' on the current line are replaced with newstr the replacement is only effected in print but not executed newstr is inserted at every match matched strings are replaced by the replacement string in the most recent substitute command TCS Confidential 67 6 Finding Text /^ pattern/ searches for patterns at beginning of line /pattern $/ searches for pattern at end of line 68 /pattern/ the pattern is searched forward // forward search ?? backward search TCS Confidential 68 6 Command Execution Command Explanation (n,n)g/findstr/commandlist Executes given commandlist for every occurrence of findstr (n,n)G/findstr/ Inputs one command to execute for every occurrence of findstr 69 (n,n)v/findstr/commandlist Executes given commandlist for every non-occurrence of findstr (n,n)V/findstr/ Inputs one command to execute for every non-occurrence of findstr TCS Confidential 69 6 Special ed Commands Command Explanation f print current filename w write contents of buffer into file w file2 write contents of buffer to file 2 r read current 70 file r file2 read contents of file2 after current line e file2 edit file2 —> invoke ed on file2 !shellcmd Execute shell command by preceding with ‘!’ ! ls –l gives a listing of current directory after current line TCS Confidential 70 7 The ex editor * * * * * an improved version of ed less terse provides display options like numbered 71 line allows shorthand versions of commands clear response for error messages TCS Confidential 71 7 The vi - Visual Editor The editor * * * * * * Invoking Powerful full screen editor vi v/s ed, ex Mostly single key stroke commands Interface with ‘ex’ Macro facility Ability to process text 72 $ vi filename e.g., $ vi pgm.c Modes : * * * Command mode Insert mode From Insert mode Pressing <ESC> remitts Command mode TCS Confidential 72 7 Types of commands * vi-commands (invisible) Command mode Commands can be categorised as : * Cursor movement * Text manipulations 73 - insert, delete, copy, change * Marking/Selecting, Positioning * Search Objects of interest recognized by ‘vi’: * characters * words * lines * block TCS Confidential 73 7 Format of commands [count] command [operand] Use “.” to repeat last command Use “u” to undo last command Cursor Movement Line Oriented : 74 ^ or (zero) $ Character oriented h l j k beginning of line end of line move left move right move down move up TCS Confidential 74 7 Format of commands (contd.) Word Oriented : e move to end of next word w move to beginning of word b 75to beginning of previous word move E move to end of next word ignoring punctuation W move to beginning of word ignoring punctuation B move to previous word ignoring punctuation TCS Confidential 75 7 Block Oriented : ) move forward one sentence ( move backward one sentence } move forward one para { move backward one para % move to find matching parenthesis Rightly used in C- Programming for matching ‘(‘ & ‘)’ 76 m mark a particular line with a label. e.g., ma (marks the line with label a) “ return cursor to position (m) e.g., "a moves the cursor to the line which was marked with label a ^g file information, line number TCS Confidential 76 7 Text Manipulation Insert mode : a A i I o O esc Delete mode : u nx ndw ndd D append after cursor at end of line insert at cursor at beginning of line enter in new line after current line enter 77in before current line to exit insert mode undo last command delete n- next character deleting n next words deleting n lines current line and n-1 below current line are deleted. delete till end of line TCS Confidential 77 7 Changing Text : nr s S cw Cut & Paste : replace char delete current char, enter insert mode delete line, enter insert mode change word 78 nyy yank n line Named Buffers : a to z ( 26 ) Unnamed Buffers : 1 p print buffer J join next line to current EOL Join : TCS Confidential 78 7 Search Searching for a character : fc search forward for character ‘c’ tc similar to79 f, cursor placed 1 char left of ‘c’ Fc search backward for character ‘c’ Tc similar to F, cursor place 1 char right of ‘c’ ; continue search in same direction ‘ continue search in reverse direction TCS Confidential 79 7 Search File Related : G go to the end line nG go to the nth line M go to the middle line H go to the top line L go to the last line 80 TCS Confidential 80 8 Text Manipulation Screen Adjustments : ^d Scroll down half a screen ^u ^f Scroll up half a screen 81 Page forward ^b Page backward ^e exposes one more line at bottom of screen ^y exposes one more line at the top of screen TCS Confidential 81 8 Text Manipulation Searching for a string : /string 82 for searching forward ?string for searching backward Use n to continue search in the same direction Use N to continue search in the opposite direction s/oldstr/newstr commands can be used in vi TCS Confidential 82 8 Common Set commands set ai Causes automatic indentation set noai set nu Nullifies the effect of auto-indent 83 Causes line numbers to be displayed set wrapmargin = n Sets n column right margin set bf The beautify option Removes all unimportant control characters set Causes a displays of current set options TCS Confidential 83 8 Text Manipulation :w! file force write :q :q! quit normally 84 quit, no write :n next file, for $ vi f1 f2 f3 f4 :n! edit next file, discarding changes to current file :n args specify new argument list TCS Confidential 84 8 Debuggers : Why ? * * for running a Program under the control of the programmer for examining the values of various variables and stack contents Types fsdb File System Debugger Fixes damaged file system 85 kdb Kernel Debugger Resides in kernel, Allows memory examination Disassembles instructions Executes programs sdb - Debugging source code written in C, Fortran 77 - Available on System V Role The functionality of any debugger is essentially the same TCS Confidential 85 8 Debugger - sdb * Invoked as sdb <executable file> e.g., $ cc -g test.c $ sdb a.out 86 * sdb checks for the status of files * sdb gives warning messages if source files are not present or source files were modified after the object files * sdb gives a prompt * after invocation * some commands can be used to avail the important features of sdb TCS Confidential 86 8 SDB Commands Command Explanation *r <arguments> run the program (arguments optional ) stepping through the instructions 87 setting a break point at line number 20 *s *20b (Normally break points are set at function calls) */pattern *<function-name> b \*B *d <line-no> *D searches for the pattern setting the break point at the function name gives a list of break points\ deletes the break point at the specified line no. deletes all the break points TCS Confidential 87 8 sdb Commands (contd.) Command Explanation *S stepping over the function call *c continue upto the break point *variable name prints the value of variable active at that time *variable name/x prints the value of variable in hex *variable name/c prints the value of variable in character *variable name/o prints the value of variable in octal *variable name/s prints the value of variable in null terminated string *variable name/a *function:variable character strings at the specified address prints the value of the variable in the specified function 88 TCS Confidential 88 8 sdb Commands (contd.) Command Explanation *p print the current line of the source *<line-no>p print the corresponding line-no of the source *<ctr-d> prints next 10 lines of source or data or instructions 89 depending on the previous command *w prints a window of 10 lines of source or data or instructions around the current line *<Enter-key> executes the previous command *!<command> executes the shell command *k kills the debugger *q quits the debugger TCS Confidential 89 8 CTRACE - Debugger Different from the conventional debuggers like sdb Works in the absence of a debugger Ctrace simulates the insertion of printf statements – Programmer’s way of debugging is to insert printf statements at the required places – Results in more printf statements than the program itself Ctrace also traces infinite loops 90 TCS Confidential 90 9 CTRACE - Debugger (Contd .) * Invoked as ctrace [options] < source-file > traced-file options : -o -x print the variable in octal 91 in hex print the variable -u print the variable in unsigned format -e print the variable in floating point format -f <function-names> trace only the selected function names -v <function-names> complement of option -f trace all functions except those specified in the list TCS Confidential 91 9 e.g., $ ctrace < test.c > ctest.c test.c ctest.c - source file traced file Compile the traced file as $ cc ctest.c 92 creates in a default file a.out Run the executable $ a.out <arguments> arguments are optional Outputs the values of variables and statements as desired by the user TCS Confidential 92 9 PROFILING TOOLS : PROF and GPROF * Improves efficiency and debugging * gprof is available in BSD versions * The program should have 93 been compiled with -p option for prof and -g option for gprof * prof - refers default input file mon.out * gprof - refers default input file gmon.out * Gives an idea as to which function used up the maximum time TCS Confidential 93 9 Example of prof $ cc -p test.c $ a.out $ prof a.out Generates an output like.... %time name 91.2 7.2 1.0 0.0 cumsecs %time Percentage of time consumed by the function cumsecs Running sum or cumulative sum of the functions called seconds Seconds consumed by the function calls Number of times the function was invoked name Function name 9.34 10.08 10.18 10.24 seconds #calls 9.34 0.74 0.10 0.06 20100 1 94 100 1 msec/call _func2 acnt _func1 _main TCS Confidential 94 9 CB - C Beautification * Beautifies the C-program code * Indents all the statements * Improves readability * Invoked as 95 $ cb < source-file > target-file e.g., $ cb < test.c > test1.c test.c - source file test1.c - Beautified version of test.c TCS Confidential 95 9 Chapter 4 96 Shell Features TCS Confidential 96 9 Shell Features Shell Features 1. 2. 3. 4. 5. 6. 7. 8. Various kind of shells Shell commands Shell Startup Dot files 97 Shell variables C Shell Test conditions Shell programming - Bourne Shell Shell programming - C Shell TCS Confidential 97 9 Various types of Shells : 1. Bourne Shell sh 2. C Shell csh 98 3. Korn Shell ksh 4. Restricted Shell rsh 5. Visual Shell vsh TCS Confidential 98 9 Shell Commands Simple commands using shell Metacharacters Exit status - return value 0 - Successful return value > 0 - Failure Redirection Default input from Keyboard Default output on the Screen < - Read from a file > - 99 Redirect it to a file >> - Append to a file n>&m - Merge output from file descriptor n with file descriptor m n<&m - Merge input from file descriptor n with file descriptor m where n and m are file descriptors for the stdin, stdout and stderr files Pipeline sequence of commands separated by | TCS Confidential 99 9 ; sequential execution & asynchronous execution && following list executed if preceding pipeline returns zero value || 100 following list executed if preceding pipeline returns nonzero value * matches 0 or more characters ? matches any single character in filenames $var value of shell variable var var=variable assign variable to var TCS Confidential 1 100 # rest of the line is a comment ‘...‘ run command within backward quote ‘...’ treats ... literally “...” treats ... literally except for $ and ‘...‘ and \ 101 e.g., $ (sleep 5 ; echo “The date is `date` “ ) & date The Output of the above command will be 6345 Wed Sep 23 10:20:45 EDT 1991 $The date is Wed Sep 23 10:20:50 EDT 1991 TCS Confidential 1 101 PATH * If full path is given while executing the command, the command is executed from the path specified * Else it is searched in the Shell variable $PATH 102 Search order is defined by value of Shell variable Path * e.g., PATH = :.:/usr/bin:/bin: TCS Confidential 1 102 Shell Startup Dot files * Helps in customizing UNIX System * Executes .profile for Bourne Shell when user logs on * Executes .login and .cshrc for C Shell when user logs on * Executes .logout for C Shell when user logs out from C-Shell 103 Keeps track of the history mechanism In C Shell through the .history * TCS Confidential 1 103 Shell Variables User defined assignment : name = value e.g., (on Bourne shell) $ $ x=”Hello how are you” echo $x 104 Hello how are you $ $ $ PATH=$PATH:/usr/games export PATH echo $PATH :.:/usr/bin:/bin:/usr/games: TCS Confidential 1 104 Positional Parameters e.g., 105 shellscript arg1 $0 $1 arg2 $2 TCS Confidential 1 105 Evaluation of Shell Variables * $var value of var ; nothing if undefined * ${var} same as $var ; useful if alphanumeric characters follow variable name - derefrencing variable e.g., $ var=hello $ 106 varx=goodbye $ echo $var hello $ echo $varx goodbye $ echo ${var}x hellox TCS Confidential 1 106 * $ {var?message} If defined , $var Otherwise print message if message not defined print default message e.g., * $ echo ${var?} hello $ 107 echo ${junk?} junk : parameter not set $ echo ${junk?my message error} junk : my message error ${var-thing} Value of var if defined , else use thing $var unaffected TCS Confidential 1 107 * ${var=thing} Value of var if defined ; otherwise thing $var is set to thing e.g., * $ {var+thing} $ echo $ {junk-’Hi there’} Hi there $ echo ${junk?} 108 junk : parameter not set $ echo $ {junk=’Hi there’} Hi there $ echo ${junk?} Hi there If var defined then thing Otherwise nothing TCS Confidential 1 108 Command Environment * Variables & associated values 109 Shell 1 Shell 2 Export A e.g., export A * Export from parent to child shell * Vice-versa not true TCS Confidential 1 109 Predefined Special Parameters $# number of positional parameters $? exit value110 of last command $$ process number of a process $* all the parameters $! process id of last command started with asynchronous execution i.e., & TCS Confidential 1 110 Variables Used by Bourne Shell in .profile CDPATH search path for cd HOME login dir PATH search path for commands PS1 primary system prompt PS2 111 secondary system prompt IFS internal field separator MAIL files containing mail messages TERM EXINIT terminal type list of set commands for vi TCS Confidential 1 111 The C Shell * Developed by Bill Joy ( Berkeley University) * More helpful 112 interaction * Permits shorthand repetition * Permits aliasing TCS Confidential 1 112 The variables can be defined using a set for CShell variables e.g., (on - C shell) 113 % set x=‘date‘ % echo $x Wed Sep 23 12:15:20 EDT 1991 TCS Confidential 1 113 History previous Records previous commands ! is used to execute command Command Format : ! <event> [ <word> <modifier> ] <event> - absolute - relative - name expr !1 <word> - 0, 1, ...n <modifier> absolute range last All arguments !?su? 114 !-3 !c a-e $ * - Substitute :s/trg1/trg - Print only :s/trg1/trg/p TCS Confidential 1 114 History Mechanism Keeps track of commands % history Lists previous commands % ^old^new Modify last command e.g., 115 % cd /usr/trg/SRC % ^SRC^src % cd /usr/trg/src % !15 % !c Run command 15 Run previous command beginning with c TCS Confidential 1 115 Changing Commands :S/pattern/replacement e.g., % !cd:s/trg/trg1 116 cd /usr/trg1/src % !ls : p Prints the previous command beginning with ls but does not execute it TCS Confidential 1 116 Using Arguments !$ last argument !^ Begin argument !* All117 arguments !10:5-9 Arguments 5 to 9 for command 10 e.g., %ls !15:2-4 TCS Confidential 1 117 Alias mechanism Short names for frequently-used long commands e.g., % % % alias cdms ‘cd /usr/man/man\!$’ 118 cdms 2 pwd /usr/man/man2 % alias cd ‘cd \!*;set prompt=”`pwd`>”’ sets the prompt to the present working directory as and when cd is used to change the working directory. TCS Confidential 1 118 Filename Grouping { } %cp Attach filenames to common root /usr/trg4/{ac,docs,test} . %cp/usr/trg4/ac . %cp /usr/trg4/docs . %cp /usr/trg4/test . 119 %ls {/usr/bin,/bin}/{ls,more} /usr/bin/ls : not found /bin/ls /usr/bin/more /bin/more : not found TCS Confidential 1 119 Using C - Shell e.g., % cc test.c % vi test.c % !c % a.out % !v 120 % !c - o test % test % car /usr/trg2/document/report % !! : s/car/cat or % ^car ^cat TCS Confidential 1 120 Variables in .cshrc of C SHELL % set variable=value histchars : % set histchars=";," will thereafter use ; as execution character 121 character , as substitution history : % set history=40 path: % set path=:.:/bin:/usr/bin: prompt : % set prompt=”\!>” TCS Confidential 1 121 Variables in .cshrc of C SHELL setenv used to make a variable as an environment variable so that whenever changes are made it is also reflected in the environment like TERM , DISPLAY is used % setenv TERM=AT386-M filec % set filec Its usage : : file name completion : Predefined 122 % set variable % unset variable ignoreeof % set ignoreeof : To ignore all eof character ^D noclobber % set noclobber : Abandons all command using output redirection which destroys an existing file and appends to non-existing file TCS Confidential 1 122 Test Conditions TESTing Strings test - z $string String length equal to zero test - n $string String length not equal to zero 123 test $str1 = $str2 str1 is equal to str2 test $str1 != $str2 str1 is not equal to str2 test $string string is not a null string TCS Confidential 1 123 To algebraically compare variable values test $num1 -eq $num2 Other algebraic operators -ne, -gt -ge -lt, -le File related $ test -f file 124 True if file exists and is an ordinary file $ test -r file True if file exists and is readable file $ test -w file True if file exists and is writable file $ test -s file True if file exists and has size greater than zero TCS Confidential 1 124 Shell Programming - The Bourne Shell Control flows if command then commands [ elif commands ] ... [ else commands ] 125 fi case word in, pattern) commands ;; pattern) commands ;; ... esac TCS Confidential 1 125 for name [ in word ...] do commands done while command do 126 commands done until command do commands done TCS Confidential 1 126 break n exit from for / while / until from level n continue n next iteration of for / while / until from level n exit n exit with value export [name..] export test expr 127 evaluate conditions read [arg] read variables from stdin or Keyboard shift n shift positioned parameters w.r.t $1 TCS Confidential 1 127 Examples $ > > > > > > > for i in *.dat do size=‘wc -c $i‘ if test $size -eq 0 then rm $i fi done - The complete for loop can also be edited in a file and, by granting the execute permission on the file , the Shell program can be invoked By default any shell program is parsed to Bourne shell - 128 However, by mentioning #! /bin/sh or #! /bin/csh in the first line of the shell program, it can be parsed to either Bourne shell or C-shell resp’ly TCS Confidential 1 128 Example echo If you have a TTY vt100 enter vt100 echo If you have a TTY AT386-M enter AT386-M echo If you have a TTY sma84 enter sma84 read term case $term in vt1001) TERM=vt100 ;; AT386-M) TERM=AT386-M ;; 129 sma84) TERM=sma84 ;; *) echo ‘Invalid TERM’ ;; esac # Making it is an environment variable export TERM echo end of program This can be a part of your .profile TCS Confidential 1 129 IF Control Flows if (expr) then command; [ else if ] command; [ else ] command; endif SWITCHswitch (string) 130 case string1: command; breaksw case string2: command; breaksw default : command; breaksw TCS Confidential 1 130 FOREACH foreach name (word) commands end WHILE while ( expr ) 131 commands end UNTIL until ( expr ) commands end TCS Confidential 1 131 Chapter 5 132 Advanced Utilities TCS Confidential 1 132 Advanced Utilities 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. Redirection Pipes Pipe fitting with tee utility Filters Find Sort Utilities that process tabular data- cut, join, csplit Ps and Kill133 Timely Execution- nohup, at, time User backup utilities- tar, cpio, dd , doscp Output related commands- pr, lpr,lp Other utilities- sleep, sync, wait, clear, tr Introduction to Make Lint Lex Yacc TCS Confidential 1 133 Redirection Metacharacter Performs “< “ input “> ” output write “>>” output append e.g., $ who trg1 trg2 tty00 tty02 $ who > who_out $ cat who_out trg1 tty00 trg2 tty02 Apr 8 Apr 8 09:33 11:10 134 Apr 8 Apr 8 09:33 11:10 $ cat date_out Fri Apr 8 14 : 30 : 10 est $ who >> date_out $ cat date_out Fri Apr 8 trg1 tty00 trg2 tty02 14 : 30 : 10 est Apr 8 09:33 Apr 8 11:10 $ date > date_out 1983 1983 TCS Confidential 1 134 Pipes : Metacharacter Performs “|” e.g., piping 135 Without Pipe $ cat /etc/passwd > temp $ sort < temp $ rm temp With Pipe $ cat /etc/passwd | sort * No need for creation/deletion of a file * Sorts the file passwd as per the first entry in passwd TCS Confidential 1 135 A UNIX Pipeline Standard Input >Prgm #1 Prgm #2 Prgm #3 Prgm #4 136 >Prgm #5 Prgm #6 Standard Output< TCS Confidential 1 136 The TEE Utility : Helps in creating the intermediate file during the pipe operation e.g., 137 $ ls *.c | tee Cflst | sort $ cat cflst * * Creates the intermediate file Cflst Cflst contains the list of c files TCS Confidential 1 137 FILTERS : grep get regular expressions only 138 fgrep fast, several simple strings at one time egrep extended grep, can handle more powerful expressions like | - or operators TCS Confidential 1 138 $ grep pattern filenames Options : -c number of lines matched -i ignore case -n print line with line-number -v print lines which do not match 139 TCS Confidential 1 139 Searching Files with grep e.g., $ grep -ni func1 *.c Prints all the lines and line numbers in files *.c that match 140 func1 ignoring the case pattern $ grep ‘*’ * Search for the pattern * in all the files $ ls -l | grep ‘^d’ Searches for all subdirectories TCS Confidential 1 140 e.g., $ egrep ‘int|long’ test.c Searches for all those lines containing either int or long in test.c 141 $ egrep ‘ (^[A-Z])’ testfile Searches for all the lines which start with a capital letter $ cat exprfile Searches for lines having at least 3 commas ,.*,.*, in file testfile using the exprfile $ egrep -n -f exprfile testfile TCS Confidential 1 141 Find To locate one or more files find path-list cond-list action-list path-list cond-list -type x one or more directory names -user name for a specificed user name -group name for a specified group name -size n File size n. blocks If n follows c then so many characters -links n x can be d, f, or c d directory f ordinary file c character special142 file locate for file with n links -atime n files accessed during n days ago -mtime n modified n days ago -ctime n created n days ago -perm nnn permission flags match nnn -name fname with specified filenames TCS Confidential 1 142 actionlist - print display path names - exec command execute the unix command - cpio device copy the files on the tape in specified format 143 - ok command like exec , but executed interactively e.g., find / -atime 7 -print - find / -atime +7 -print - find / -atime -7 -print - will print files accessed in exactly 7 days will print files accessed earlier than 7 days will print files accessed within 7 days TCS Confidential 1 143 Examples $ $ $ $ $ find / -atime +30 -print find / -atime +30 -size +10 -print find / -atime +30 144 -size +10 -ok rm -f {} \; find . -perm 0777 -print find . -perm 0777 -print -exec rm {} \; $ find / -name passwd -print Will find the passwd file TCS Confidential 1 144 Sort * * Sort keys can be fields or lines A field is a string of characters separated by a field separator or new line. $ sort {-options} {+post1} {-post2} {files} The sort key begins at post1 ands ends just before post2. There can be several keys. Options b ignore leading blanks and tabs c only145 check if input file is already sorted d dictionary order letters, digits, blanks f ignore case i ignore non-printable characters n r tp u o filenameout numeric sorting reverse order use p as separator output only lines with unique keys save output in filenameout TCS Confidential 1 145 e.g., $ sort -t: +2n -3 /etc/passwd Print the passwd file sorted by numeric user id 146 $ who | sort +4n Sorts according to the login time stamp of the user TCS Confidential 1 146 uniq Finds and eliminates duplicate lines in a file and is often used with sort $ sort <file> | uniq -c Sorts and ouputs the147 number of count of lines containing unique fields $ sort <file> | uniq -d Gives only the duplicated lines tsort Accepts as input a partial ordering and produces a fully ordered list of the items. $ tsort psortfile TCS Confidential 1 147 Processing Tabular Data Cut * Deletes columns from a file producing a new file with shorter lines * Cuts out selected fields of each line in a file. * Cuts columns from a table or148 fields from a file which can be of type - Fixed length fields or Delimited by some character TCS Confidential 1 148 Cut (contd.) cut -c list { file1 file2 ...} e.g., cut -c 5-70 file1 cut would pass 5-70 characters from file1 cut -flist {-d char } {file1 file2....} 149 e.g., who | cut -d” “ -f1 gives a list of user login names cut -d: -f 1,5 /etc/passwd gives a list of user ID and names TCS Confidential 1 149 JOIN * Combines corresponding lines in two files by relating the contents of one or more columns. * Implements a relational data base ‘join’ on two tabular files -jn m join on the mth field of file ‘n’ -on.m output mth field of nth file n - file no. m - field no. -tc use char c’ as separator 150 TCS Confidential 1 150 e.g., $ join -j1 4 -j2 3 -o 1.1 2.1 1.6 -t: etc/passwd etc/group - joins field group id - 151 parameters outputs the following login name group login name dir TCS Confidential 1 151 csplit context split $ csplit [-k] [-f prefix] [-s] file name arg1 [..argn] Reads file and separates it into n+1 section defined by arg1...argn Options : -s Normally csplit prints152 the character counts for each file , -s is to suppress this -k csplit removes previously created files in case of error , -k is to avoid this csplit normally creates file as xx00 , xx01 ... , xx99 -f prefix creates file with that prefix instead of default xx TCS Confidential 1 152 csplit(contd.) e.g., $ csplit -f cobol inpfile /Procedure division/’ / Para5./ /Para6./ /Para 7./ * * * Creates file as cobol 00 .. cobol 03 Edit these Cobol files Can be recombined as 153 $ cat cobol 0[0-3] > file csplit -k prog.c ‘%main(%’ ‘/^}/+1’ {20} * Breaks file prog.c containing C routine upto a maximum of 21 * ‘% expr %’ —> no. file is created for this section * No csplit for the main routine ‘%main%’ TCS Confidential 1 153 PS, kill $ somecommand & 5511 - pid $ps pid tty 3432 5765 5511 2 2 2 $ kill 5511 $ ps pid 3432 5985 time 0 : 24 154 0 : 03 0 : 51 tty time 2 2 0 : 24 0 : 03 command -sh ps somecommand command -sh ps TCS Confidential 1 154 $ stubborn-cmd & pid 3432 6004 5995 $ kill 5995 tty 2 2 2 time 0 : 24 0 : 03 0 : 44 command -sh ps stubborn-cmd tty 2 2 2 time 155 0 : 24 0 : 03 0 : 44 command -sh ps stubborn-cmd tty 2 2 time 0 : 24 0 : 03 command -sh ps $ ps pid 3432 6004 5995 $ kill -9 5995 $ ps pid 3432 6103 TCS Confidential 1 155 Timely Execution : time Time a command $ time wc test.c > wc.test real 2.0 user 0.4 sys 0.3 156 nohup Protecting a process from hanging and quit signals or interrupts The standard output is sent to nohup.out e.g., $ nohup du / & 820 Sending output to nohup.out $ logout TCS Confidential 1 156 nice * * Executes at specified priority Default priority is 24 $ nice +n Raise priority $ nice -n Lower priority $ nice n set157 priority at Executes process at specified time $ at 5 pm echo ^G ^G Time to logoff >/dev/tty04 ^D $ TCS Confidential 1 157 User Backup Utilities : tar tape archiver Copies files on backup medium such as floppy or tape in tar format tar [options] devicename filelist Options : 158 c r create a new tape backup tape old files are overwritten append files to the tape t x list the names of files from backup tape extract files from the backup medium u update a tape, if the named files are not present or have been modified later on verbose ; provides informational messages, such as the name of each file as and when it is encountered v f devicename use device for backup medium TCS Confidential 1 158 tar (example) $ tar cvf /dev/fd096 * copies all files from the current directory onto the backup medium /dev/fd096 $ tar xvf /dev/fd096 Extracts all the files from the backup medium /dev/fd096 onto the current directory 159 Creates necessary directories tar xvf /dev/fd096 try.c Extracts the file try.c from /dev/fd096 tar tvf /dev/fd096 Generates a file list of /dev/fd096 find / -mtime -7 -exec tar uf /dev/fd096 {} \; All the files modified last week is backed up onto /dev/fd096 TCS Confidential 1 159 cpio * Copy files archives in and out * Bundles all the files into one package cpio [options] file list devicename Options -o Copy these files onto tap -i -p Extract files from tape 160 Read a list of file or path names from the standard input -v display a verbose set of cpio action -c character information in ASCII for portability considerations -t For listing files from the backup medium -O Append files to the end of tape to be used with -A -d Creates necessary directories -k In case of error , ignore the error and continue TCS Confidential 1 160 cpio (Examples) $ ls *.c | cpio -ocBv > /dev/rmt/0n Lists the files *.c and copy onto the device /dev/rmt/0n 161 $ cpio -ocBv < filelst > /dev/rct/c0s0 Copies all the files mentioned in filelst onto the catridge tape $ cpio -icBdv < /dev/rct/c0s0 Extracts all the files from device c0s0 and creates necessary required directories $ cpio -itv < /dev/rmt/0m List the files from the tape /dev/rmt/0m TCS Confidential 1 161 dd convert, reblock, translate, copy a tape file. dd {option = value } Options Values if of ibs obs cbs skip seek count conv input file name output file name input block size (Default 512) 162 output block size(Default 512) conversion block size n records before copying from I/P file write after n output records in O/P file n records To ASCII, EBCDIC, lcase, ucase (separated by a comma) e.g., $ dd if=/dev/rmt/0n of=x ibs=800 cbs=80 conv=ascii,lcase TCS Confidential 1 162 doscp Copy a UNIX file in DOS format or Copy a DOS file in UNIX format doscp source file target file e.g., 163 $ doscp /usr/trg/test.c a: copies the file test.c onto the floppy $ doscp a:try.c . copies the file try.c from the floppy onto the current working directory TCS Confidential 1 163 Output Related Commands : pr prepares a file for printing Options -k K col. Output +k from page k -lk set length of page to k lines -p 164 page pause after each -h take next argument as header -wk set width to k characters -d double space e.g., $ pr -3 d h “file list” f1 f2 Generates a header as file list 3 Column output Double spacing $ pr -5 wordlist Generates 5 column output TCS Confidential 1 164 pr (contd.) $ pr myfile prepare myfile nov 24 : 11 : 31165 1987 myfile page 1 $ pr -t myfile suppresses the header This file can be printed using $ pr myfile | lpr TCS Confidential 1 165 lpr Print a file on the line printer $ lpr myfile prints myfile on the printer $ lpr -c myfile make a copy to the spool directory 166 $ lpr -r myfile myfile is removed from the spool lp Combines the actions of pr and lpr $ lp myfile prints myfile with header, date, time and page numbering. TCS Confidential 1 166 Other Utilities sleep Suspend execution $ sleep 5 prompt appears after 5 seconds $ sync * updates super block * writes all 167 disc buffers * calls sync before stopping system to ensure file system integrity * saves all modifications $ wait $du waits till all background jobs are over determines disk usage -s total blocks -a size of each file TCS Confidential 1 167 df displays no. of free blocks df [option] file system clear clears the screen and the prompt goes to top of screen tr replaces specified characters with other characters 168 e.g., $ tr “[a-z]” “[A-Z]” < lfile > ufile Replaces all small case letters to upper case letters $ tr -d’ ‘ < tstfl > tstfl2 Deletes all the blank characters $ tr -s ‘\012’ < try > try2 Removes adjacent blank lines in file try TCS Confidential 1 168 make - An Introduction Making & Maintaining large programs What is make? Program for maintaining large number of programs Need for make * Difficulties in remembering the dependencies * Simple mechanism for maintaining an up-to-date 169 version of Programs Characteristics * Helps maintain large systems * Specifies dependencies of files in the system and t the actions to make them * Uses creation date to determine the actions to be taken TCS Confidential 1 169 make : AN INTRODUCTION …Contd... make program takes the file named makefile or Makefile as its input. makefile details: the names of the files that make up the program system their interdependencies 170 how to regenerate the program system Example : final mod1 mod2 mod3 TCS Confidential 1 170 make : AN INTRODUCTION …Contd... Example (contd): Makefile or makefile final mod1.o mod2.o mod3.o : mod1.o mod2.o mod3.o cc -o final mod1.o mod2.o mod3.o : mod1.c cc -c mod1.c 171 : mod1.c cc -c mod2.c : mod3.c cc -c mod3.c Run the command as .... $ make final use -f urflname option If the name of the file is not Makefile or makefile TCS Confidential 1 171 Using the Macro facility and Inference rules QSRC = qp1.c qp2.c qp3.c INCLUDES = lim.h com.h OBJECTIF = qp.o qex.o libs.a query : $(OBJECTIF) 172 cc -o query $ (OBJECTIF) qp.o : $ (QSRC) cc -o qp.o $ (QSRC) libs.a : libs(lib1.o) libs(lib2.o) qex.o lib1.o lib2.o : $ (INCLUDES) LIBES = libs(libs1.o) libs(libs2.o) prog : $(OBJECTIF) cc $(OBJECTIF) $(LIBES) -o prog TCS Confidential 1 172 Lint - C Program Checker * * Reports the various inconsistencies that can exist in a program in the following areas . Syntax errors . Unused variables . Unused arguments . Unused functions 173 . Unused return values . Unused external variables . Unintialized variables . Type checking features . Portability considerations . Statement not reached . Excludes goto statements Invoked as lint [options] filename TCS Confidential 1 173 Lint - C Program Checker …Contd... Example : test.c #include <stdio.h> main(argc,argv) int argc; char **argv; { int i,j,k,l,number,num2; int int1, int2; int arr[20]; 174 scanf(“ %d”,num2); printf(“The square root of %d is %f \n”, number , sqrt(number) ); i=arr[j++]; add(int1,int2); } add (int1,int2,res) int int1,int2, *res; { printf("The value %d", int 1 + int 2); } TCS Confidential 1 174 Lint - C Program Checker …Contd... Generate an output after invoking lint as lint test.c -lm test.c warning : k , l unused in function main warning : j may be used before set warning : main() returns random to invocation environment 175 Function argument (number) type inconsistency scanf (arg2) int :: format int * : test.c Value type declared inconsistently sqrt llib-lm : test.c Function argument ( number ) used inconsistently sqrt ( arg1 ) : test.c Function called with variable number of arguments add : test.c Function returns value which is always ignored printf scanf TCS Confidential 1 175 Lex source * General tool for separating character strings patterns in an input * String patterns can be specified using rules 176 syntax definitions % % rules % % user subroutines TCS Confidential 1 176 Lex …Contd... Example (on rule) Expression ——> integer * integer Expression ——> integer + integer Expression ——> integer - integer Expression ——> integer / integer 177 The lex specification file %% [0-9]+ [-*+/] [\t ] ; { return (INT); } { return (OPR); } { printf(“Lexical analyser error\n”); exit(-1); } TCS Confidential 1 177 Lex …Contd... %% yywrap() { return(1); } yywrap() indicates no further input 178available Invoked as $ lex filename $ lex lex.l Generates lex.yy.c TCS Confidential 1 178 Yacc : What is it? Yet Another Compiler Compiler Features * General tool describing input to a program 179 * Specify input structure * Code for each structure * Outputs a subroutine which handles the input * User supplied routine to supply next basic item TCS Confidential 1 179 Yacc …Contd... Specifications Declarations %% rules %% Example %token INT OPR %start expr %% 180 expr : INT OPR INT { printf(“The input expression is correct\n”); } | error { printf (“The input expression is wrong \n”); } %% Invoked as Generates $ yacc filename $ yacc yac.y y.tab.c TCS Confidential 1 180 A Program using both lex & yacc Consider this program... #include <stdio.h> main() { yyparse(); } 181 #include “y.tab.c” #include “lex.yy.c” The whole program can be compiled as ... $ cc main.c -ll -ly -ll and -ly to link lex and yacc library $ a.out TCS Confidential 1 181 Chapter 6 182 Communication Features TCS Confidential 1 182 Communication Features 1. User communication commands mail write talk post news mesg 2. Networking commands 183 uucp uuto rlogin telnet ftp rcp uux ct cu TCS Confidential 1 183 User Communication Commands : * Mail * Write * Talk * Post * News * Mesg 184 TCS Confidential 1 184 Mail Sends and receives mail messages between users Sending mail on local system At the end of the message press ^D to exit $ mail <loginnames> 185 user is trg) Example : (Assume the current $ mail trg1 Sending mail on remote system $ mail recipient@remote_system.domain_info TCS Confidential 1 185 Receiving mail $ mail Shows the message received ? q s [file] w [file] d n or + r m [user] ! cmd Prompt quit save (defalut mbox) write without header delete next message 186 previous reply to the sender mail to user run the shell command * The message could be saved or deleted * Environment could be setup in .mailrc TCS Confidential 1 186 User Communication commands …Contd... Write Allows sending messages to a user terminal from another terminal The message is ended by pressing ^D. 187 $ write <username> Talk Allows interactive dialogue between users at two different terminals. $ talk <username> TCS Confidential 1 187 Post users. Messages can be posted on a “bulletin board” for previewing by all News mesg * * Keeps the user informed of current events Refers file /usr/news e.g., $ news Prints all the news , latest first $ news reorg Prints the news named reorg 188 The terminal can be write protected to prevent other users from writing onto your terminal e.g., $ mesg [option] option : - y write allowed -n remove write permission on the terminal $ mesg prints the current status whether mesg is y or n TCS Confidential 1 188 Networking Commands * uucp * uuto * telnet * rlogin * ftp * rcp * uux * ct * cu 189 TCS Confidential 1 189 uucp * allows copying files from one unix system to another * Refers file uuname and uulog uucp [options] source-files destination files options -d make necessary directory 190 -c do not copy local file to spool directory -m send mail to the requester when the copy is completed $ uucp -d -m file? sys_2!/usr/spool/uucppublic Copies the files file? creating appropriate directories onto system sys_2 onto /usr/spool/uucppublic TCS Confidential 1 190 uuto * * Simplified version of uucp Sends file to /usr/spool/uucppublic $ uuto filename(s) system!login 191 $ uuto -m test.c sys2!trg2 Sends a mail to the sender when the job is over rlogin * * Allows remote login over the network Similar to telnet, but a much more flexible facility. $ rlogin system_name!username TCS Confidential 1 191 Networking Commands …Contd... ftp * * File transfer protocol Transfers files from one system to another $ ftp ftp > open sys1 Name login: passwd 192 ftp > ? Gives you the help ftp > get filename Gets the file onto the current system ftp > put filename Puts the file onto the \ sys1 ftp > close $ TCS Confidential 1 192 rcp Similar to ftp, but a much more flexible facility uux * Unix to Unix command execution on remote systems * Gather files from various computers, run a command on a specified computer and redirect the 193a specified computer stdout to a file on $ uux [options] commandstring All shell special characters must be quoted $ pr minutes | uux -p host!lp Command line queues the file minutes to be printed on the area of the computer host TCS Confidential 1 193 ct * * Connect terminal to remote terminal Connects your terminal to remote terminal which is equipped with a modem This command dials the phone number of the modem 194 $ ct [options] telno $ ct -h -w5 -s1200 9=5553497 ct will call modem using a dialer operating at a speed of 1200 baud , wait for 5 minutes before quitting if dialer is not available and use the -h option not to disconnect the local terminal TCS Confidential 1 194 cu * * connects local system to remote system allows you to be logged on to both simultaneously Examples cu -s2400 9=5557867 Connected login : 195 ~%take proposal Takes proposal from remote computer to local computer ~%put minutes minutes.bak Vice versa of take and puts the file minutes on the remote computer as minutes.bak ~%!comma run the command on local system ~%$command run the command on local and send the output to the remote terminal ~%. terminate the link TCS Confidential 1 195 Chapter 7 196 System calls TCS Confidential 1 196 System Calls : 1. Introduction 2. File related system calls open() read() write() creat() chmod() chown() 197 lseek() 3. Process related system calls fork() getpid(), getppid(), getpgrp() wait() execl() TCS Confidential 1 197 * Library functions and system calls * Both are C - functions * Difference lies in their incorporation in the UNIX System * Library functions are referred to as add-ons * System calls are part of the UNIX Kernel * Library functions themselves use system calls and can be expanded by the user * System Calls generally common across UNIX versions * System calls share the concept of fd -file descriptor * File descriptor is an integer used to identify a file 198 TCS Confidential 1 198 Basic tasks in file operations opening files reading files writing in files creating files 199 changing the permission of files changing the owner and group of files seeking to file-location closing files TCS Confidential 1 199 open () * open() an existing file int open (filename,mode) char *filename; int mode; filename - character pointer to the name of the file mode - integer signifying the mode 0 for read 1 for write 200 2 for read and write open() returns the file descriptors on success and returns -1 on error Example : Opening the file “test” in read mode and checking for the error condition fd = open(“/usr/trg/test”,0); if ( fd == -1 ) { printf(“error in opening file test”); exit(1); } TCS Confidential 2 200 read () * * File should be opened in read mode read() an opened file int read (filedesc,buffer,nbytes) int filedesc; char *buffer; int nbytes; 201 filedesc - File descriptor indicating which file to be read buffer read - An area of buffer storage for holding the Characters nbytes - number of characters to be read at a time read() returns the number of characters read and 0 in case ofend of file (EOF) and returns -1 on error TCS Confidential 2 201 Example : Reading the file “test” 100 characters at a time while ( ( n = read(fd,buff,100)) > 0 ) { printf(“file test has contents %s “,buff); }if ( n == 0 ) printf ( “ End of file “); if ( n == -1 ) printf (“Error in reading202 file test”); When each read is finished the pointer advances by 100 bytes so that the next read picks from there * If the number of characters left are less than nbytes ( in this example - 100 ) then read() will pick up what is left over TCS Confidential 2 202 write () * File should be opened in write mode * write() to an opened file int write (filedesc, buffer, nbytes) int filedesc; char *buffer; int nbytes; 203 filedesc - File descriptor indicating which file to be written buffer file - The function takes from buffer and writes them to indicated nbytes - number of characters to be written at a time write() returns the number of characters written and returns -1 on error TCS Confidential 2 203 Example : while ( ( n = read (fd,buff,100)) > 0 ) { n1 = write (1,buff,100); /* writing to standard output */ /* file id - 1 is for stdout */ if ( n1 == -1 ) printf (“Error in writing on stdout"); } 204 if ( n == -1 ) printf (“Error in reading file test "); TCS Confidential 2 204 creat () * creat() creates a new file or overwrites on the existing file int creat(filename, mode) char *filename; int mode; 205 filename - character pointer to the name of the file mode Integer signifying the mode The mode is specified in octal code - creat returns the file descriptor on success and returns -1 on error TCS Confidential 2 205 creat () …Contd... Example : umask(0000); fd = creat(“newfile”,0666); if ( fd == -1 ) { printf(“error in creating file newfile “); 206 } Creates a file called “newfile” in mode 0666 i.e., read and write permissions for owner, group, and others Note : while creating a new file ensure umask is set to zero Otherwise, If umask had been 0022 in the environment variable, then the effective permission would be mode & ~0022 TCS Confidential 2 206 chmod () * chmod() set permissions for the file int chmod (filename, mode) char *filename; int mode; 207 filename - character pointer to the name of the file mode Integer signifying the mode The mode is specified in octal code - chmod() returns 0 on success and returns -1 on error TCS Confidential 2 207 Example : ret = chmod(“test.c”,0600); if ( ret == -1 ) { printf(“error in changing the file permission”); } 208 Changes the permission of file test.c i.e., read and write permissions for owner TCS Confidential 2 208 chown () * System call chown() * chown() set ownership for the file int chown (filename,owner ,group) 209 char *filename; int owner , group ; filename - owner - character pointer to the name of the file owner id group group id chown() returns 0 on success and returns -1 on error TCS Confidential 2 209 chown () …Contd…. Example : ret = chown(“test.c”,0,1); if ( ret == -1 ) { 210 printf(“error in changing the owner and group of the file”); } Changes the owner and group of the file test.c as root and others respectively TCS Confidential 2 210 lseek () * lseek() changes the position of read-write pointer for the file descriptor int lseek(filedes, offset, origin); int filedes, origin; long offset; 211 on success lseek() returns new-value of the pointer returns -1 on error The value of the pointer depends on origin : 0 set the pointer to offset bytes from the begining of the file 1 increment the current value of the pointer by offset 2 set the pointer to the size of the file plus offset bytes TCS Confidential 2 211 lseek () …Contd... Example : ret = lseek(fd,1000,0); if ( ret == -1 ) { 212 printf(“error in seeking to the 1000’th byte of the file “); } lseek(fd,1000,0) skips the first 1000 bytes of the file and starts reading from the 1001’th byte TCS Confidential 2 212 fork () * fork() creates a new process which is a child process * Child process is a logical copy of the parent process * Parent’s return value is the process id of the child * Child’s return value is 0 213 TCS Confidential 2 213 getpid () getppid () getpgrp () * getpid() returns the process id of the calling process * getppid() returns the parent process id of the calling process 214 * getpgrp() returns the process group of the calling process TCS Confidential 2 214 EXAMPLE fork () , getpid () , getppid (), getpgrp () #include <stdio.h> main() { int id ; int pid , pgrp ; 215 int ppid ; id = fork(); printf (“PPID - %d PID- %d id - %d \n”, getppid() , getpid() ,id ); printf (“PGRP - %d \n “, getpgrp() ); } TCS Confidential 2 215 EXAMPLE fork () , getpid () , getppid (), getpgrp () Output PPID - 371 PID - 372 id - 0 - From child Process PGRP - 136 PPID - 136 PID - 371 id - 372 - From Parent Process PGRP - 136 216 PID - Process-ID PPID - Parent Process - ID ID - Returned value from fork() PGRP - Process-group-ID If PID is equal to the process PGRP then the process is the group leader TCS Confidential 2 216 wait () & execl () * wait() causes a parent to stop running and await the termination of a child process * execl() overlays the original process with a new set of instructions Example on execl() #include <stdio.h> 217 main() { int id; printf ( “Parent process \n”); if ( ( id = fork() ) == 0 ) { printf(“Statement from child process\n”); execl(“/bin/date”,”date”,0); } printf(“ Parent process again \n”); } TCS Confidential 2 217 EXAMPLE ON execl() Output Parent Process Statement from child process Parent process again Tue Sep 10 11:34:17 1991 Process forked two processes and parent process avoided execl() to print the final statement i.e parent process did not wait for the child to finish To make the parent wait for the child to finish - wait() can be used The example on execl() gets modified 218 as #include <stdio.h> main() { int id; printf ( “Parent process \n”); if ( ( id = fork() ) == 0 ) { printf(“Statement from child process\n”); execl(“/bin/date”,”date”,0); } wait(); printf(“ Parent process again \n”); } TCS Confidential 2 218 OUTPUT OF THE MODIFIED EXAMPLE Parent Process Statement from child process 219 waits for the child to finish Tue Sep 10 11:34:17 1991 Parent Process again TCS Confidential 2 219 220 TCS Confidential 2 220