COMPUTING FUNDAMENTALS
COMPUTING FUNDAMENTALS
What is an Operating System?
An Operating System is a system software that manages computer hardware, software resources and
provides common services for computer programs.
What is Abstraction?
Abstraction is filtering / hiding the internal implementation or concept and only showing what is
relevant.
Basics:
OS is the interface between the end user and the computer hardware.
Hardware is controlled by Firmware, It is built into the electronics. Firmware is the OS for the
hardware.
BIOS is the firmware used to load an operating system and provides configuration options.
Architecture Hierarchy
User
Application
Operating System
Hardware
Kernel is the core of Operating System. It coordinates individual components of a computer.
Everything running on top of OS is software, such as processes and programs.
These user process interface with OS, which then works with Kernel to ensure the processes work
properly.
Responsibilities Of OS:
1. Processor Management : OS prioritizes which processes are run by the CPUs at what time and
ensures smooth execution.
2. Memory Management : OS prioritizes what goes into RAM for CPUs to access. If the arrival of
information is late in RAM then the computer runs slowly.
3. Device Management : OS manages I/O devices and passes them to different processees as
required.
4. File Management : OS keeps track of files and organizes them and passes access to different
processes.
5. Security : If defenses are programmed in, the OS attempts to identify unauthorized uses of files
and memory and attempts to defends itself.
6. Logging and Error Detecting : OS saves information as it runs to logs and enable troubleshooting
of processes.
7. System Performance : OS keeps track of how processes and the OS as a whole are running, and
will kill or pause processes as required for smooth operation.
Computer Hardware:
Central Processing Unit (CPU) :
The circuitry which executes the instructions that the OS gives it.
There can be multiple CPUs in a computer, and multiple processes on a CPU.
Random Access Memory (RAM) :
RAM are chips that provide fast, temporary storage for information and is required for
computers to run quickly. The OS uses RAM to ensure that the CPUs get the information they
need.
Input/Output (I/O) Devices :
Storage Devices : These storage devices are slow, but can contain a great deal of information.
Data from 'file systems' is loaded into RAM by the operating system when needed.
Mice/Keyboards : The OS listens for mouse movements and key presses and responds so
that a user can drive the OS.
Monitors : The OS displays what is going on so the user can see what is happening.
Virtualization:
Virtualization is when we run an operating system on top of another operating system.
Using Virtualization software on the Host OS, the Guest OS or OSs are able to pass through to the
Hosts hardware.
Virtualization software is known as a hypervisor.
Hypervisors are programs used for running and operating the Guest OSs, which we refer to as
Virtual Machines.
There are two primary types of hypervisors :
bare-metal : These hypervisors run directly on top of a server and act like their own OS,
managing all the things an OS does.
hosted : These hypervisors are programs that run on an OS like Linux, and have the OS
handle the responsibilities.
Hypervisors are able to run as many OSs as the underlying hardware can support, which is usually
limited by RAM.
Benefits of Virtualization:
The primary benefit of virtualization is the ability to share resources between multiple OSs.
To test OSs before installing them to bare-metal, Because the hardware is abstracted there is no
concern that we will have compatibility issues.
When we use a VM, we can be confident that we will not break the host environment with whatever
we do in there.
Containers:
A Container is a standard unit of software that packages up code and all its dependencies so the
application runs quickly and reliably from one computing environment to another.
A Docker container image is a lightweight, standalone, executable package of software that
includes everything needed to run an application: code, runtime, system tools, system libraries and
settings.
Container images become containers at runtime and in the case of Docker containers - images
become containers when they run on Docker Engine.
Shells:
In computing, a shell is a computer program which exposes an operating system's services to a
human user or other programs.
The way users modify the OS is by accessing either a Graphical User Interface (GUI) shell or a
Command Line Interface (CLI) shell.
While the kernel is the base of the OS, the shell is what interfaces with it and is the most visible part
of the OS.
sudo - ["substitute user do"]
used to escalte privileges up to the level of the username specified in the command.
If no username is specified, sudo defaults to the administrator access, ie. root.
Packages:
Packages are what Linux OSs use to distribute and maintain software.
They are custom file formats that usually contain an archive of compressed files that are used to
install, as well as some metadata that describes how to install and what the dependencies are.
Dependencies are a list of what other programs a piece of software need to be installed in order for
it to run.
Comand Line Basics:
whoami : tells us who we are / current username.
pwd : present working directory , tells the path within the directory.
ls -l : long list, list the long names of everything in current working directory.
mkdir : make directory, to create a new directory.
cd : change directory, to change the access or enter a directory.
cd .. : The ".." is a strange concept but basically means "go up one level". If you wanted to go up
two levels, you would type cd ../..
echo : to printout the given text.
">" : redirector, used to redirect the output of the commands to files. If there is no file, then it
creates a file.
cat : concatenate, to print the contents onto the terminal.
">>" : append, to add the output to the back of files.
cp : copy/rename, used to copy the contents of a file to another. It can also be used to rename a
file.[cp oldfile newfile].
rm : used to remove a file.
rm -r : delete file/folder recursively.
touch : used to create a new file and update timestamp.
mv : used to move a file.
!! : used to rerun the last command.
uniq : checks for matching lines adjacently.
sort : used to sort the contents.
Pipe : Pipes are a form of redirection, but instead of going to files, the redirected streams go to other
commands.
Pipes are represented with " | " and are one of the more powerful tools we have as we can chain
together commands.
It is important to note that pipes are unidirectional and only pass data from left to right.
Networking:
Routing
tracepath : used to trace the route in internet. In windows it is tracert.
Netcat :
the most common use is that you can use it to send and receive arbitrary data across a
network.
Also know as the "Swiss Army Knife" of networking tools.
Whatever you want to do, you can probably do it with netcat.
curl : Curl is super useful to quickly grab resources if you have the URL.
netstat
lsof
telnet : Telnet is an old fashioned protocol that usually provides nice and easy shell access to
anyone running a telnet server, which is often unauthenticated.
SSH : Secure SHell or SSH. They are basically the secure successor to Telnet for providing remote
shells.
Text-Fu:
Viewing Streams:
head : head will print the top few lines of a larger file.
tail : tail will print the last few lines of a file.
less : less will bring you to the top of a file and allow you to scroll or search through it.
String Modification :
cut : Remove sections from each line of files. Print selected parts of lines from each FILE to
standard output.
awk : The awk utility shall execute programs written in the awk programming language, which is
specialized for textual data manipulation. An awk program is a sequence of patterns and
corresponding actions. When input is read that matches a pattern, the action associated with that
pattern is carried out.
sed : Sed is a stream editor. A stream editor is used to perform basic text transformations on an
input stream (a file or input from a pipeline).
tr : Translate, squeeze, and/or delete characters from standard input, writing to standard output.
grep : grep searches the named input FILEs (or standard input if no files are named, or if a single
hyphen-minus (-) is given as file name) for lines containing a match to the given PATTERN. By
default, grep prints the matching lines.
grep - Global Regular Expression Print.
Search an entire directory by using the "*" symbol, often known as a wildcard. This wildcard will
open every file in a directory.
eg:- $ grep "dog" * .
If you only want to know which files in a directory have the target word, use "-l".
eg:- $ grep -l "dog" * .
If you want to know the file name and line number, use "-n".
eg:- $ grep -n "dog" * .
If you only want to see the lines in the file which don't have the term you are greping for, use "v".
eg:- $ grep -v "dog" animals .
This defaults to showing line numbers and file names, so if you only want to see the filenames,
use "-l".
eg:- $ grep -l "dog" animals.
find : Only shows about file information.
Useful Linux Commands
Tab key : Hit the tab key to autocomplete words/paths/anything and save yourself time. Double tap
it to get a listing of possible options.
cd : cd ~ which takes you to your home directory, and cd - which takes you back to the directory
prior.
History : look at your most recent command history using the command history, and combining
that with grep you can pipe the two together to search your most recent commands. But even
easier than that is typing ctrl-r, which kicks off a search in your history for you.
Line Navigation : If you have a super long command and don't want to arrow through the whole
thing, you can use "home" and "end" keys on your keyboard. Even more efficiently than that
though, you can ctrl-a and ctrl-e to zoom to the front and back.
Reuse Commands : Use the "!!" operator to run the same command again. If you want to run the
most recent of a certain command, use the single "!" operator and the command name, such as
!ls, which will run the most recently executed ls command. Much easier than pressing the up arrow
until you find what you are looking for.
Kill Processes : use ctrl-c, if you are in the same window as it. If not, ctrl-x is a great standby.
New Terminal : ctrl-shift-t should pop open a new terminal . If you are already in a terminal, it
should pop up a new tab. To switch between these, ctrl-pagedown or ctrl-pageup.
To open a new window, ctrl-shift-n should open a new terminal window.
Machine Code :In order to be executed on a CPU, code must be written in a way that is understood by the CPU.
This is called 'Machine Code' and consists of 1s and 0s.
Linux Filesystems
dd - "disc distroyer".
df - displays all the filesystem in the computer.
sudo blockdev --getbsz /dev/DRIVENAME - prints the block size of the given drive.
Inodes are the pointers OSs use to give the location of a block of data. Whether it is for a file or a
directory, there will be an inode associated with it. These inodes are all stored together in a table
towards the front of a file system on a drive. By reading the data located in the inode table, the OS
is able to calculate the correct offset into the Data Blocks required to find the file. This is why inode
stands for Index Node, because it helps us find where we put all of our files.
Each file has its own inode number, we can get it with ls -i fileName.
"debugfs" is a powerful tool for working directly with the drive, and we can use this particular
incantation to print out the information of the inode we have selected.
eg. "sudo debugfs -R "stat <1521431>" /dev/DRIVENAME" .
Metadata :Metadata is the information about a file, rather than the data contained in the file. Most file formats
have room for metadata towards the front of the file, which means that the metadata for the file is
stored inside of the file, but is not considered to be part of the file itself. It is possible to remove the
metadata for the file without affecting the way the file displays.
Another type of metadata is filesystem metadata. Rather than being included in the file itself, it is
stored in the inode table. Use ls -i to get the inode information for a file and then use debugfs -R
"stat like we did above to get all the information for the file, including the filesystem metadata.
A third type of metadata is information about who sends and receives information. Even if the
content can't be decrypted, just knowing who sends to who is useful information and can be
dangerous depending on the context.
Disk Partition
Now using the name you found for the filesystem, run the following command with resize2fs. This
command will automatically match the size of the partition to the size of the available disc space.
Linux Permissions
OS basically segregates user and root process privileges using something known as "Protection
Rings".
Modifying Permissions :
We use the command chmod and either "u" (for user), "g" (for group), "o" (for others), or "a" (for all
3) to do this on Linux. It's fairly straightforward, we either add ("+") or remove ("-") permissions with
"r", "w", and "x".
Here are some examples.
$ chmod g+x file2
adds the executable bit to the group permission for file2.
$ chmod g-w directory2
removed the write bit for a directory2 for group.
$ chmod a+x file2
added the executable bit for everyone.
We can use chmod to modify any permission on the file we own, as long as we have the
permission to modify permissions. The most common use of chmod is when you download a file
from somewhere, it is not given execute privs by default due to security policy. So anytime you find
an executable not executing, you should check a chmod.
If we want to change permissions on something we don't own, you're going to need root access for
that.
Now when we ls -l we see that file2 is owned by root. When the sudo command is finished running
we drop back into our normal permissions (run whoami to check). If we try to change permissions
now with chmod we will get a permission denied error. However, using sudo we can elevate privs
again and chmod the permission around because we are the owner. This works for a file owned by
any user, if you are root you can do what you want.
Similar to chown is chgrp which, you guessed it, changes the group. Basically the same usage as
chown.
A final thing we can do is set something known as "the sticky bit".
$ chmod +t file2
$ chmod +t directory2
This "+t" sets a file or directory as not-deletable by anyone other than the owner and root. This is
great for shared folders.
Bits and Permissions :
Because of the magic of binary, if we have three bits, those bits can range from "111" to "000" which
represents 7-0. This allows us to represent permissions as numbers as well.
The read bit is "4"
The write bit is "2"
The execute bit is "1"
Because we can add up the bits in binary, the "rwx" we were getting used to can also be
represented by "111", "421", or "7".
Now that you get it, this also allows us to set perms in chmod in a much cooler way.
$ chmod 755 file2
All perms for everybody is represented by "777" but you should avoid doing that, it's the opposite of
the Principle of Least Privilege.
You can also remove permissions by using lower numbers.
$ chmod 744 file2
Permissions and Umask :
When a file is created in Linux, the default permissions for it is "666". For a directory, the default
permissions are "777". That is basically always fine, but if you need to for some reason, you can
modify the default permissions using a command named umask.
Masks are all over computing and generally can be defined as strings of bits that set other bits
based on logic.
Depending on the required implementation, different logic gates can be used for different masks.
This means that the logic for a gate can be "AND", "OR", "XOR".
The most common place where you will see permissions masks is with the umask command.
In order to change your permissions that files default to for your shell:
$ umask 002
This umask of "002" converts over to being a mask of "775".
When applied to This will change permissions so that files come out RWX for user and group and R
for other.
You might have noticed that when you ran umask the first time there was a leading "0" in front of
the other numbers. That first bit is the special bit we talked about earlier in terms of the "sticky bit".
There are other special types of files that can be indicated in the special bit location.
/etc/passwd :
$ cat /etc/passwd
Here you will see a printout of all the users with accounts on your computer. The top one will be
'root', and will look like generally like this.
root:x:0:0:root:/root:/bin/bash
dennis:x:1337:1337:Dennis:/home/dennis:/bin/bash
There will be many more entries than this, but let's ignore them for now.
We will learn about how this file is broken down using "man passwd"
To break it down, /etc/passwd stores data about users separated by colons (:).
1. User name
2. Encrypted password
On modern systems, the password will not be stored here and is indicated by an 'x'. We will
learn about where the passwords are stored next.
3. User ID number (UID)
4. User's group ID number (GID)
5. User full name and other data(GECOS)
This is an arbitrary set of data also known as the "comment" field. Sometimes name, email, phone
number, separated by commas. Most of the time, there is nothing here but your name.
6. User home directory
7. Default Login shell
/etc/group
Similar to /etc/passwd, this prints out information about the various groups on the computer.
roppers:*:1337:dennis, grace
1. Group name
2. Group password - Using an elevated privilege like sudo is standard, but a separate password can
be added. "*" will be put in place as the default value.
3. Group ID (GID)
4. List of users - Manually specified users in the group.
Add a User and Set the Password :
$ sudo useradd -c "User's Full Name" account_name
$ sudo passwd account_name
Delete an Account :
$ userdel -r account_name
Userdel with the "-r" flag is final and will delete the user along with the user's home directory.
Set Password :
While usually you will use passwd to set passwords, there are many other uses. To check the
status of a user account, use this format.
$ passwd -S account_name