ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 Please note. Many of the examples below are just that, examples. They may or may not be valid based on the files that are currently in your home directory. Use the examples as a guide and enter appropriate commands based on the files/data that you have. The COMMAND Line When you enter the ls command, the shell executes the utility program named ls. The line that contains the command, including any arguments, is called the command line. Remember, when describing syntax (punctuation of command), square brackets are used to refer to optional items: $ ls [file] In above command you may or may not need to list a file to ls Arguments Arguments are the words listed after the command name (after the first word entered at the prompt). The following as two arguments: l and hi. $ ls –l hi PATH Very important!!!!!!!!!!!!!!!!! What’s the difference between the following two commands: $ ls $ /bin/ls The first command gives the command name without a path to that command. The second command gives a path to the command. Remember: Think of a command as a program. You can issue the command with a path. If so, unix knows where to find the program. ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 However, if you don't issue a path with the command, how does unix know where this program resides? Does it search every directory on the unix system???? NO!!! When you do not issue a path with the command, as shown below, then unix searches your PATH. $ ls That is, it searches the PATH variable. To view your PATH variable: $ echo $PATH Please note, that first dollar symbolizes the prompt … do not type it in. The second dollar however, is part of the variable PATH so you must enter it. ) The following command will display the contents of your PATH variable. It should contain a list of directories separated by colons. It should look something SIMILAR to this: $ echo $PATH /bin:/usr/bin:/sbin The above contains THREE directories: /bin /usr/bin /sbin How many directories are listed in YOUR path???? ______________________ If you issue a command without a path, unix will search those three directories. If the command is not located in one of those three directories, then unix will not run the command. What if the command is located in your current directory? ANSWER: if your current directory is not in your PATH, then it will not find the command!!! Please note, some commands that you enter are built into the shell, so it doesn't matter if you enter the command with a path or not. This is only for commands that are separate programs on your ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 system. For now, assume all are separate programs. What if directory is not in your PATH, but you need to run program? 1: Add directory to PATH (we will do this later). 2. Or just put a path name to your command. All of the following have some type of path associated with the command: $ /bin/ls $ /users/shah_a/myprogram $ ~/myprogram $ ./myprogram In the above examples, the PATH variable is not checked! Errors Get comfortable with error messages: xx: FSUM7351 not found The xx is the command you entered. Problem is probably that you had a typo in the name, the command doesn't exist, or the directory in which command is located is not in your PATH. xx: FSUM9209 cannot execute: … permission denied Program exists, you don't have permissions to run it. More on this one later. Standard Input/Output When you first log in, standard output is defined to be your terminal (screen). All output from your commands will go to your screen. Standard input is defined to be your keyboard. All input will come from the user … typing it in. More on cat Command Let's use the cat command to practice this. The cat command sends the data you give it to standard output. ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 Remember, if you gave the data in a file, it sent that data (in the file) to the screen: $ cat file1 We were giving it data from a file (in the above example it assumes you have a file1). By default, however, cat assumes you are giving it the data from standard input. $ cat If you enter the cat command WITHOUT arguments, it seems as if nothing happens. Since you don't see a prompt, it is telling you that it is waiting for some input. Type in a line. Whatever you type, should get displayed back to the screen. Keep doing this. When you are done, press [ctrl][d] It should look similar to this $ cat hi bye [ctrl][d] hi bye $ cat displayed the data it received. This is how we normally use the cat command: $ cat filename The filename basically becomes the input. It then displays that data (in the filename) to the screen. Output Rather than sending the output to the screen, Unix has a utility that can send the output to a file. The > character is used to send the output to a file (REDIRECTION) Output goes straight to the screen: ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 $ ls Output goes to the file called “junk”: $ ls > junk Important info on > Look at the following command: $ ls > junk The output is in a file called “junk” which is in your current directory (since you used a relative path name) By default if the file junk DOES NOT exist, it will be created. IF the file junk DOES exist, it gets wiped out!!! To view output Look at the following command: $ ls > junk The output is in a file called “junk” which is in your current directory (since you used a relative path name) To view the output you can do any of the following: $ more junk $ cat junk $ vi junk WHY? Output redirection is GREAT for many reasons: Save output for future reference Take output and manipulate it further More examples You can save information on your files (size, name, etc…): $ ls -l > junk You can save top 10 lines of the file “stuff” into a new file called “top10”: $ head stuff > top10 You can concatenate two files called “one” and “two” into one new file called “total”: $ cat file1 file2 > total WARNINGS on > Consider the command: $ head info > junk If there is already a file called “junk”, it will be ERASED before the concatenation begins. Then Unix issues the “head info” command, gets the output and puts it in a file called “junk”. ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 So … what do you think happens here: $ head junk > junk Answer to WARNINGS on > So … what do you think happens here: $ head junk > junk 1. Unix looks at the file name to the right of the >, if it exists, then it wipes it out 2. Unix looks to the left of the >, and issues the “head junk” … HOWEVER, “junk” was just wiped out earlier, so we have no more data. MORAL of the story: NEVER use the same file name on the left and right side of the > sign. Explanation on WARNINGS on > Why would people even try this: $ head junk > junk Some people, especially programmers, might think like this: I have a huge file called junk, and I only need the top ten lines, so grab the top 10 lines with “head junk” and put that answer back into the file called “junk”. It’s a good idea because it saves having to create a new file, but it WON’T work!!!!!! The >> Unix also allows you to append the output: The >> character is used to append output to a pre-existing file Output goes straight to the screen: $ ls Output gets added/appended to the file called “junk”: $ ls >> junk Important info on >> Look at the following command: $ ls >> junk The output is now added to a file called “junk” which is in your current directory (since you used a relative path name) By default if the file junk DOES NOT exist, it will be created. IF the file junk DOES exist, the output from the ls command gets ADDED to the bottom of the file!!! The < Rather than getting output from the keyboard, let's just get it from a ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 file. This way we don't have to type it in. Input has to come from user typing it in: $ cat Input comes from a file called junk $ cat < junk Important note on < Yes, the following two commands are the same: $ cat junk $ cat < junk Some commands will require you to use the latter (using < ) in order to specify that input is coming from a file, rather than keyboard. NO CLOBBER Remember how we said that if we issued the following command $ head stuff > junk and if the file “junk” already existed … then it would be wiped out. That is very dangerous. Unix has a way for us to prevent this from happening: NO CLOBBER (think of it as not clobbering the file) More on NO CLOBBER It varies how you turn on the NO CLOBBER option … depending on the shell you are using. We are using the bash shell so this is how we can turn on the no clobber option: $ touch xxx $ set –o noclobber $ ls > xxx $ date > xxx After you issue that command, you will no longer be able to accidentally wipe out files with the > utility. To turn noclobber back off: $ set +o noclobber ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 $ date > xxx How long does NO CLOBBER last? After you turn on the NO CLOBBER option, it will last as long as that session remains open. If you close your session (i.e., close your window, etc…) then you will be set back to the default, which is NO CLOBBER turned OFF. There is a way to set noclobber on permanently. We will talk about this later. NO CLOBBER only for output redirection The NO CLOBBER option only saves you from overwriting files when using output redirection, specifically It doesn’t prevent you from overwriting files when using the cp command or the mv command, etc… Also, when using NOCLOBBER with >>, the >> works slightly differently: If NOCLOBBER is on, then when you use the >>, the file to the right of the >> MUST exist. Think of it as saying, if you are using append symbol, then the file must exist. Yes, this can be a pain, but it's how it works! PIPING A pipe sends the output from one command straight into another command. Pipes aren’t really necessary, but they do save a lot of time. For example, say you want to view the listing of the /etc directory: $ ls /etc If you try listing the contents of the /etc directory you will notice that many files just fly by. Why? Because the listing is too long for the screen. So, one trick is: $ ls /etc > mylist $ more mylist That trick allows you to save the output from the ls command into a ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 file called “mylist” and you can then use the more command to slowly view it! Listing the /etc directory with pipes Of course, that previous trick created a file called “mylist” which you will probably have to delete. You can do the same thing with pipes: $ ls /etc | more The vertical bar | is the pipe. The output from the left side of the pipe becomes input to the right side of the pipe. The item on the right side of the pipe must be a command that TAKES input. Another pipe example How would you display the 3rd through 6th lines of a file with a command on one line??? $ Review on pipe A pipe is a vertical bar The output from the left side is sent to the command on the right side The right side must be a command that can accept pipes. The right side must be a command that REQUIRES a file as the argument, but we don't have to list the filename For example, more is never used by itself. We always use more with an argument “more file”. When it is on the right side of a pipe, we leave off the argument since it automatically gets that info from the left side of the pipe. Commands that lend themselves to PIPES There are certain commands that we tend to use on the RIGHT side of pipes. The commands I’m referring to are: grep, head, tail, sort, more, cat, tr, (awk) What do the above commands have in common?? ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 tr I love this command. It translates/converts characters in a file. For example, you can change every letter “A” to a “Z”. Or you can chance every lowercase “a” to a capital “A” The following changes every A to a Z and changes every B to a Q and displays the output to the screen: $ cat data | tr AB ZQ You must send the data to the command from a pipe! One of the few commands that REQUIRE pipes It also doesn’t change the contents of the original file, as with other UNIX commands. If you want to save this info, you must redirect the output: The following changes every A to a Z and changes every B to a Q and saves output to a new file called newdata: $ cat data | tr AB ZQ > newdata Cool trick with tr The following changes lowercase letter to an uppercase letter $ cat data | tr a-z A-Z > newdata The a-z is the same as: abcdefghijklmnopqrstuvwxyz It's just a shortcut way to right it. ps Used to display a detail list of current processes. Running this command without any options will list all the processes you are currently running in your shell (i.e., window). $ ps PID TTY TIME CMD 492 ttyp0000 00:00:00 /bin/sh ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 More on ps and SHELLS This is an explanation of the output : PID: each process has a number associated with it. This is the process id # TTY: which terminal is running the process. If you opened up another telnet session/window, you would see that it had a different TTY associated with it. Type the tty command to view your terminal name TIME: duration in seconds that the process has been running CMD: The name of the command $ ps PID TTY 492 pts/2 493 pts/2 TIME CMD 00:00:00 bash 00:00:00 ps The above example is from a different unix system. I was running two processes. The bash command is one. The ps is the other one (since I just ran it). When we first log in to Unix, we are actually put in a shell. The shell is a program that is waiting to translate/interpret your commands and send them off to the kernel. The kernel is the main part of the Unix operating system that is always running. Bourne Shell (/bin/sh) Bash Shell (/bin/bash) Csh (/bin/csh) Korn Shell (/bin/ksh) How could you find the different shells available on this system?!?!?!What command could you list: ______________________________________________ How could you find the default shell on our system?!?!?!What command could you list: ______________________________________________ ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 $ /bin/tcsh Type commands and then press enter and use the up arrow keys $ exit ps and kill $ ps PID TTY 492 pts/2 TIME CMD 00:00:00 bash If one of the processes you were running was hung up, you could use the ps command, find the PID of that process, and kill it. Right now we don’t have enough processes running to do this. If we killed the bash command we would be kicked of the Unix box, since that is what is controlling our commands (we don’t want to do that). More on ps and kill $ ps PID TTY TIME CMD 492 pts/2 0:00 bash However, if we needed to kill process 492 we would issue the following: $ kill 492 ps -fu This will show you all the processes for a particular user. If I was logged on, you could check all my jobs by issuing: $ ps –fu kc02123 The –f tells ps to show a full listing on most unix systems The –u (above) tells ps to list only the processes from a particular username, and that username must be the argument that follows the –u option! ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 Background Process What if you are running a command that takes a few minutes to execute? You would have to wait for the command to finish before you could enter another command (basically you have to wait for the prompt to be displayed). Unix allows you to run commands/processes in the background so that you do not have to wait for that job to finish before continuing on. More on background Process Use the & sign after the command to indicate you want it to run in the background, versus running it in the normal way, in the foreground! When you are running a process in the background, and it is a process that produces output, you will want to make sure that you redirect that output to some file so that it doesn’t interfere with your foreground session. You will be able to practice this in the lab. Something Extra What if you wanted to type these two commands one after the other. $ cd $ pwd Being the lazy person I am, I would type them in on one line, using the ; as a separator: $ cd ; pwd Pathname Expansion More wildcards * ? [] ITSC1316 Fourth Class Notes Tuesday, February 23, 2010 Pathname Expansion Note The following command: $ ls cat* Will list all the files that start with the letters cat. Suppose we had two files: catalog1 and catx. Unix actually matches (expands) the files when there are wildcards before running the command. It figures out the files/dirs it matches first, then runs command It's as if the command were run as follows: ls catalog1 catx Pathname Expansion Note 2 $ ls catalog* cat* Will list all the files that start with the letters cat and catalog. It expands the names of the files that match “catalog*” (catalog1 in our case) and then expands the names of the files that match “cat*” (catalog1 and catx). The following command is then issued behind the scenes: ls catalog1 catalog1 catx Notice how the catalog1 file will be listed twice. Try it out with files in your directories.