Instructor: Prof. Michael P. Harris ITSC 1405 – Intro to PC Operating Systems Chapter 9 Pipes, Filters, and Redirection Chapter 9 Pipes, Filters, and Redirection LECTURE NOTES CHAPTER OUTLINE Chapter Overview Will use redirection to redirect standard input and standard output. Learn how pipes can be used to connect programs. How filters can be used to manipulate data will be explained. What shell extensions are and how they may be used will be discussed. Learn how to combine commands using pipes, filters, and redirection. REDIRECTION OF STANDARD I/O (INPUT/OUTPUT) Redirection of Standard I/O (Input/Output) What is keyed in is input. What is written to screen is output. Input/output called I/O. OS recognizes three standards: Standard input: Keyboard. Standard output: Screen. Standard error: place from which OS writes error messages to the screen. Not all commands deal with standard input and standard output. See PowerPoint slide #10 – Results of Copy Command I/O redirection allows standard I/O to be changed. Read from file, not keyboard. Write to printer, not screen. Carolyn Z. Gillay, Bette A. Peat, Windows XP Command Line Franklin, Beedle & Associates ©2003 ISBN: 1-887902-82-1 TEACHING SUGGESTIONS Slides 2-6 Chapter Overview duplicated in PowerPoint slides. After completion of lecture, suggest to students that they review Objectives and Outcomes found on first page of chapter as a check to see if they have mastered concepts. SECTION 9.1 (pp. 440-441) Slides 7-13 Discussion Question (1) - Explain redirection. Discussion Question (2) - Explain the terms standard input, standard output, and standard error. Discussion Question (3) - Does every operating system command use standard input and standard output? If not, why not? Discussion Question (4) - What is the difference between > and >> when redirecting output? Discussion Question (5) - Explain how the symbol < is used. Page 1 Instructor: Prof. Michael P. Harris ITSC 1405 – Intro to PC Operating Systems Three redirection operators. The > redirects output of command to device or file. The < asks for input from source other than the keyboard. The >> redirects standard output and appends but does not overwrite a file. ACTIVITY—USING > TO REDIRECT STANDARD OUTPUT Using > to Redirect Standard Output DATA disk in Drive A, A: \> displayed. Activity steps. Key in: DIR C:\WUGXP\*.TXT DIR C:\WUGXP\*.TXT > Chapter 9 Pipes, Filters, and Redirection TXTFILES.TXT TYPE TXTFILES.TXT Activity completed. SECTION 9.2 (pp. 441-443) Redirection is either/or process - See on screen, or prints to printer. Redirection can be used with commands that writes its results to standard output device (screen). Cannot use COPY DIR *.TXT - COPY copies files, not commands. ACTIVITY—USING < TO REDIRECT STANDARD INPUT Using < to Redirect Standard Input DATA disk in Drive A, A: \> displayed. Activity steps. Key in: MD TEST COPY C:\WUGXP\*.NEW TEST DEL TEST\*.* N then DIR TEST TYPE Y.FIL DEL TEST\*.* < Y.FIL DIR TEST Activity completed. Carolyn Z. Gillay, Bette A. Peat, Windows XP Command Line Franklin, Beedle & Associates ©2003 ISBN: 1-887902-82-1 Slide 14 Discussion Question (6) – Keying in COPY DIR filename will not give you a file containing the directory display. Why? SECTION 9.3 (pp. 443-445) Slide 15 DEL *.* needs keyboard response. Useful to lock up the system - i.e., write a file with X <Enter>, then write command as DEL \TEST\*.* < X.FIL. System will lock up. Be careful with redirection of input. - Tell OS to take input from a file; any keyboard input will be ignored. . Page 2 Instructor: Prof. Michael P. Harris ITSC 1405 – Intro to PC Operating Systems ACTIVITY—USING >> TO ADD REDIRECTED OUTPUT TO A FILE Using >> to Add Redirected Output to a File DATA disk in Drive A, A: \> displayed. Activity steps. Key in: COPY C:\WUGXP\*.TXT TYPE JUPITER.TXT TYPE MERCURY.TXT TYPE MERCURY.TXT >> Chapter 9 Pipes, Filters, and Redirection SECTION 9.4 (pp. 445-447) Slide 16 Can only use >>. There is no <<. >> between files – adds contents of first file to end of second file. To append to end of existing file, use double redirection symbol (>>) JUPITER.TXT TYPE JUPITER.TXT Activity completed. FILTERS Filters SECTION 9.5 (p. 447) Manipulate information. Slides 17-20 Read information from keyboard. Change input in specified way. Compare filters to filters in water purification system. Write results to the screen. - Extract unwanted elements. Three OS filters - external commands. - Send pure water on its way. SORT - arranges lines in ascending or descending order. FIND - searches for particular group Discussion Question (7) - What are filters? of characters, called a character string. MORE - temporarily halts screen display after each screenful. OS creates temporary files while it "filters" data. Important that there be access to the disk and the filters. If a disk is write-protected, the OS will not be able to execute filter commands. THE SORT COMMAND The SORT Command Sorts/arranges lines of text. Sends output to screen unless redirected. Default sorts in ascending order. Carolyn Z. Gillay, Bette A. Peat, Windows XP Command Line Franklin, Beedle & Associates ©2003 ISBN: 1-887902-82-1 SECTION 9.6 (p. 447) Slides 21-22 Sorts A to Z or from lowest to highest Page 3 Instructor: Prof. Michael P. Harris ITSC 1405 – Intro to PC Operating Systems SORT syntax: See PowerPoint slide #22. /R - reverses sort order- sorts Z to A, 9 to 0. /+n - sorts file according to characters in column n. M kilobytes – amount of main memory to use for sort /T – (Temporary) – path of directory to hold sort’s working storage in case does not fit into main memory. /O – Output – if not specified data written to standard output. Specifying output file faster than redirecting standard output to same file. [drive1:] [path1] filename1 - specifies file (s) to be sorted. [drive2:] [path2] filename2 - specifies a file where the sorted input is to be stored. ACTIVITY—USING SORT Using SORT DATA disk in Drive A, A: \> displayed. <F6> means to press the <F6> key. Activity steps. Key in the following commands. Each command followed by <Enter> SORT then MECURY then VENUS then EARTH then 3 then MARS JUPITER then < F6> then SORT then 333 then 3 then 22 then 124 then <F6> Discuss results of this sort– sorting by units not numerically. Key in the following commands. Each command followed by <Enter> SORT then 333 <Spacebar> <Spacebar > 3 then <Spacebar> 23 then 124 then <F6> Discuss result of this sort – using spaces forces lines to be same length – placing number digits in proper position. Activity completed. Carolyn Z. Gillay, Bette A. Peat, Windows XP Command Line Franklin, Beedle & Associates ©2003 ISBN: 1-887902-82-1 Chapter 9 Pipes, Filters, and Redirection numbers (starting in first column). Discussion Question (8) – What do the three SORT parameters covered in this chapter—/n, /O, and /R—represent? SECTION 9.7 (pp. 448-450) Slides 23-27 <F6> same as pressing <Ctrl> + Z. Numbers can be character data – - Phone numbers or zip codes. Numbers - only when mathematical operation is performed on them. Character data sorted from left to right. Numeric data sorted by units. ASCII sort sequence: - Punctuation marks (including spaces). - Numbers. - Letters (lowercase then uppercase). Carolyn Smith before Robert Nesler because SORT command looks at entire line and Carolyn comes before Robert. Left justify character data and right justify numeric data. Discussion Question (9) -Explain how the SORT command works. Describe any limitations of the SORT command. Page 4 Instructor: Prof. Michael P. Harris ITSC 1405 – Intro to PC Operating Systems Chapter 9 Pipes, Filters, and Redirection FILTERS AND REDIRECTION Filters and Redirection SECTION 9.8 (p. 450) Standard output of filters is screen display. Slide 28 Can redirect both the output and the input of the filter commands. Filter commands not usually used with actual keyboard input. Filter commands used with input redirected from a file, a device, or another command. ACTIVITY—USING THE SORT COMMAND WITH REDIRECTION Using the SORT Command with Redirection DATA disk in Drive A, A: \> displayed. Activity steps. Key in: COPY C:\WUGXP\STATE.CAP SORT < STATE.CAP SORT STATE.CAP SORT /R < STATE.CAP SORT /+17 STATE.CAP SORT /+17 STATE.CAP > SORTED.CAP TYPE SORTED.CAP SORT/ +17 STATE.CAP /O BYCITY.CAP TYPE SORTED.CAP TYPE BYCITY.CAP Activity completed. SECTION 9.9 (pp. 450454) Slides 29-30 /R – sorts in reverse or descending order. WXP – SORT does not require < prior to file being sorted. /+n – sorts by column number . Column – on screen is place occupied by one character. Column number really means character number. +17 – seventeenth position in list. /O – stores sorted data in a file (faster than redirection). Till now, changed data not been saved to a file. Now creating a new file. Break down SORT < STATE.CAP - Can be written to screen (standard output) - Therefore can be directed to file. The FIND Filter The FIND Filter. Searches for specific character string by enclosing it in quotation marks. Command is looking for exact match, therefore, is case sensitive unless use /I parameter. Carolyn Z. Gillay, Bette A. Peat, Windows XP Command Line Franklin, Beedle & Associates ©2003 ISBN: 1-887902-82-1 Discussion Question (10) -Identify one place that standard output can be written. SECTION 9.10 (p. 454) Slides 31-34 Example - Use TYPE command, FIND command can tell if the word “indictment” is in a file. Search option available in Start menu - can search files for text. Command line not as reliable in WXP as it was in W2K Page 5 Instructor: Prof. Michael P. Harris ITSC 1405 – Intro to PC Operating Systems FIND syntax: FIND [/V] [/C] [/N] [/I] "string" [[drive:] [path] filename [ …]] /V - displays all lines not containing specified string /C - displays only the count of lines containing a string. /N - displays line numbers with displayed lines. /I - ignores the case of characters when searching for the string. /OFF [LINE] - Do not skip files with offline attribute set "string" - specifies the text string to find. [drive:] [path] filename - specifies a file or files to search. If pathname is not specified, FIND searches the text types at the prompt or piped from another command. FIND command at command line can help find a file based on content. ACTIVITY—USING THE FIND FILTER Using the FIND Filter DATA disk in Drive A, A: \> displayed. PERSONAL.FIL on DATA disk. Must use double quotes. Activity steps. Key in: Chapter 9 Pipes, Filters, and Redirection FIND "Smith" PERSONAL.FIL FIND /V "Smith" PERSONAL.FIL FND /N "Smith" PERSONAL.FIL FIND /C "Smith" PERSONAL.FIL FIND /I "Jones” PERSONAL.FIL FIND “Jones” PERSONAL.FIL /I\I Activity completed. Carolyn Z. Gillay, Bette A. Peat, Windows XP Command Line Franklin, Beedle & Associates ©2003 ISBN: 1-887902-82-1 Discussion Question (11) - What is the purpose of the FIND command? Discussion Question (12) - What are four parameters that are used with the FIND command and what do they represent? Discussion Question (13) - Why must the character string be enclosed in quotation marks when using the FIND command? SECTION 9.11 (pp. 454-457) Slides 35-36 Character string – enclosed in quotes FIND – case sensitive. Parameters. /I – ignores case. /V- search file for anything except what is in quotes. /N – finds specific line # of each occurrence. /C – numeric count of # of times specific character string in file. Page 6 Instructor: Prof. Michael P. Harris ITSC 1405 – Intro to PC Operating Systems PIPES Pipes Pipes - output from one program becomes input to next program. Not limited to two programs. Pipes used with filter commands. Used to further refine data. Symbol is broken bar | Location of pipe not standard Filter commands. Commands read and write temporary files to the disk. Filters are external commands. OS must be able to access the commands. All files (even temporary ones) must be named. Temporary files "hold" data until the next command can process it. Temporary files are deleted when pipes are done filtering. Filter commands will not work if a disk is write-protected. THE MORE FILTER The MORE Filter MORE displays one screenful of data at a time. Useful when want to read long text file. Pauses after screen is full. - Press any key - next screen is displayed. No more data in file – returns to system prompt. Carolyn Z. Gillay, Bette A. Peat, Windows XP Command Line Franklin, Beedle & Associates ©2003 ISBN: 1-887902-82-1 Chapter 9 Pipes, Filters, and Redirection SECTION 9.12 (pp. 457-458) Slides 37-41 Term “pipe” refers to flow of info. from one command to the next. Pipes move information the way water pipes move water. Filter commands. - Used with any command that uses standard output or input. - Transforms data to meet needs. Discussion Question (14) - What are pipes? Discussion Question (15) - Are there any restrictions on the use of pipes? What are they? SECTION 9.13 (pp. 458-459) Slides 42-44 MORE can be both redirected and used with a pipe. Extended features Pn Display next n lines Sn Skip next n lines F Display next file Q Quit = Show line number ? Show help line <space> Display next page <ret> Display next line Page 7 Instructor: Prof. Michael P. Harris ITSC 1405 – Intro to PC Operating Systems MORE syntax: See PowerPoint slide #43. Chapter 9 Pipes, Filters, and Redirection [drive:] [path] filename - specifies file (s) to display one screen at a time. command-name - specifies a command whose output will be displayed. /E – enable extended features. /C – clear screen before displaying page. /P – expand FormFeed character. /S – squeeze multiple blank lines into a single line. Tn – expand tabs to n space (default 8). Switches can be present in MORE environment. +n – Start displaying the first file at line n. files – List of files to be displayed. Files in the list are separated by blanks. Discussion Question (16) - How is the MORE command used? If extended features are enabled – following commands accepted at – More—prompt See PowerPoint slide #44. Right column describes features ACTIVITY—USING THE MORE FILTER Using the MORE Filter DATA disk in Drive A, A: \> displayed. Activity steps. Key in all bold commands. DIR | MORE then Return to System Prompt DIR | MORE then Return to System Prompt DIR | MORE then Return to System Prompt DIR | SORT / +39 | MORE then Return to System Prompt MORE PERSONAL.FIL then Return to System Prompt TYPE PERSONAL.FIL | MORE then Return to System Prompt MORE PERSONAL.FIL /C +20 MORE SORTED.CAP BYCITY.CAP /C Press <SPACEBAR> SECTION 9.14 (pp. 459-466) Slides 45-46 By continuing to press <spacebar> will return to system prompt. With extended features, pressing Q will break command and return to system prompt. Can connect several commands with pipes and filters. MORE can be used with any ASCII file. /P parameter can only be used with DIR. Demonstrate by using TYPE PERSONAL.FIL /P. /P not a valid TYPE parameter. Records are lines of information in a data file. Carolyn Z. Gillay, Bette A. Peat, Windows XP Command Line Franklin, Beedle & Associates ©2003 ISBN: 1-887902-82-1 Page 8 Instructor: Prof. Michael P. Harris ITSC 1405 – Intro to PC Operating Systems Chapter 9 Pipes, Filters, and Redirection MORE allows viewing file at a specified line or record number. MORE SORTED.CAP BYCITY.CAP Press <SPACEBAR> Close all open windows and return to desktop environment (GUI). Activity completed. OTHER FEATURES OF MORE Other Features of MORE If extended features are enabled – have more choices available to you with MORE command. See PowerPoint slide # 45. Command Prompt by default enables shell extensions CMD /X - Enables shell extensions CMD /Y -Disables shell extensions. Open Command Prompt window are running a shell. Shell is command interpreter used to pass commands to operating system. ACTIVITY—USING THE EXTENDED FEATURES OF MORE Using the Extended Features of MORE DATA disk in Drive A, A: \> displayed. Activity steps. Click Start/Run. Key in: CMD.EXE /X Click OK. Key in: A: then MORE PERSONAL.FIL. Press <Enter> twice then Q. Key in: MORE PERSONAL.FIL Key in: P then 5 Press <Enter> then = sign then S Key in: 3 then press <Enter> Close all open windows. Activity completed. More filter can be used with more than one file. /C parameter clears the screen before each display. SECTION 9.15 (pp. 466-467) Open Command Prompt window – running a shell. Commands that use shell extensions: DEL, COLOR, CD, MD, PROMPT, PUSHD, POPD, SET, SETLOCAL, ENDLOCAL, IF, FOR, CALL, SHIFT, GOTO, STARTS, ASSOC, and FTYPE Command name with /? - full details on what can be done with command. Discussion Question (17) – What are two useful features that extended features provides for the MORE command? SECTION 9.16 (pp. 467-471) Slides 51-52 Run remembers last command keyed in. /X – ensures that you are going to be able to Carolyn Z. Gillay, Bette A. Peat, Windows XP Command Line Franklin, Beedle & Associates ©2003 ISBN: 1-887902-82-1 Slides 47-50 use extensions to commands. With extensions enabled Pressing <Spacebar>(referred to as <Space> in syntax) - displays next page Pressing <Enter>(referred to as <ret> in syntax diagram) - displays next line in file To exit MORE command key in Q (returned to system prompt). Press P where MORE stopped – can request how many lines you want to display. = sign – displays which line number you are Page 9 Instructor: Prof. Michael P. Harris ITSC 1405 – Intro to PC Operating Systems Chapter 9 Pipes, Filters, and Redirection on. S – asked how many lines you want to skip in your display. COMBINING COMMANDS WITH PIPES AND FILTERS Combining Commands with Pipes and Filters Use pipes so standard output from one command is standard input to next command (filters). Use pipes to connect two or more programs and create a flow of data. Redirecting output from a command is an “instead of” process. When combine use of pipes and > redirection becomes end of pipeline. Last step in process. ACTIVITY—COMBINING COMMANDS Combining Commands Open Command Prompt window. DATA disk in Drive A, A: \> displayed. Activity steps. Key in: SECTION 9.17 (p. 471) When pipe symbol used – must be command on both sides of symbol. Redirection used with “pipeline,” command does not have to be on either side of > or >> SECTION 9.18 (pp. 471-474) DIR | SORT /+39 | MORE Return to system prompt. Key in: Key in: DIR | FIND “<DIR>“ SORT +39 Quotation marks around <DIR> so command line would not use < and > as redirection. Activity completed. Carolyn Z. Gillay, Bette A. Peat, Windows XP Command Line Franklin, Beedle & Associates ©2003 ISBN: 1-887902-82-1 Slides 56-57 Data in PERSONAL.FIL not changed – FIND "Teacher" PERSONAL.FIL | FIND "CA" FIND "Teacher" PERSONAL.FIL | FIND "CA" > TEACHER.FIL TYPE TEACHER.FIL FIND "PrOfeSSor" PERSONAL.FIL | FIND "AZ" | SORT Press up arrow once Use left arrow key until cursor in under P in PERSONAL.FIL Key in /|<SpaceBar> Press <End> key then <Enter> Slides 53-55 searched data so display only those lines/records that met your requirements. Because FIND command sends output to screen can redirect output. Can use same filter more than once in same command line. Can use filters in combination in same command line. Pipe. - Command on either side of pipe. - Taking standard output of a command and using it as standard input to next command. Redirection. - An “instead of” action. - Only get one output place. - Output goes to last place it is directed to go. Primary use of pipes and filters is to manipulate the standard output and standard input of commands. Rarely use pipes and filters to sort or find data in text or data files. Page 10 Instructor: Prof. Michael P. Harris ITSC 1405 – Intro to PC Operating Systems Chapter 9 Pipes, Filters, and Redirection Discussion Question (18) – How can combining filters be useful? Carolyn Z. Gillay, Bette A. Peat, Windows XP Command Line Franklin, Beedle & Associates ©2003 ISBN: 1-887902-82-1 Page 11