Introduction to Computing: FSL & UNIX
1
How is FSL used?
•
Mixture of GUI and command line tools
• typed commands at the terminal are “command line”
•
GUI is easiest to learn first
•
•
Command line is often more flexible/powerful
Scripting (automation/batching) can be done with
command line tools
•
•
•
In-house development of tools
Lots of experience around FMRIB and the University
Potential for novel analysis methods via collaboration
2
Image Files
• Supported image formats:
• NIfTI & Analyze (although Analyze is becoming outdated)
• Files for nifti normally end with .nii or .nii.gz
• Files can come in pairs of .hdr and .img but rare (like Analyze)
• Note that the .gz is a compression (gzip) to keep size small
• Data is stored in binary format (not text) and header
information (voxel size etc.) is also in binary format
• Other formats exist for other software tools
• Scanners do not write nifti directly - needs to be converted
• e.g. from DICOM
• Filenames should NEVER contain subject name or initials
(use anonymous identifiers - e.g. numbers - and keep one
record of ID relation to name)
3
Example Network (FMRIB)
These are all in a very
noisy room
Compute Farm
'Cluster'
University and the
Internet
Multimedia Mac
jalapeno
pepper
Firewall
jalapeno.fmrib.ox.ac.uk
Printers...
Storage
/usr/people
/Scratch
/Volumes/Data
Windows PCs
Apple Macs
Linux PCs
You sit here
4
Why Use Remote Access?
Advantages of using a remote access cluster:
• Speed
• e.g. fMRI analysis of 30 subjects would take ≈ 30 x 1 = 30 hours
but on a cluster ≈ 1 hour + queueing time ≈ 1 to 6 hours
• Flexibility
• can work from anywhere and it works exactly the same
e.g. college, department, home
• Updates
• professional IT staff keep all software working and up-to-date
• Back-ups
• automatically performs back-up on all/important files
Disadvantages
• Learning to use UNIX (but don’t need to be an expert)
• Not as quick and configurable for interactive/viewing tasks
(so keep using your laptop/desktop for these)
5
Remote Access
• Can run jobs on pepper in interactive mode
• Each user needs to be able to login to the cluster (e.g. course accounts)
• The local machine (computer) can be any type
• e.g. Windows/Mac/Linux : Desktop/Laptop
• the local machine does not do much “work” so it
•
doesn’t matter if it is powerful or not
really just a glorified keyboard/monitor
• Interact with the remote machine via a terminal window
• can sometimes also start up other interactive windows
(this is what X11 & ssh -Y or Putty & VNC are for)
• Access may be possible from anywhere, or restricted to within the centre/
University/country
• speed varies from 100Mb to over 10Gb (c.f. 8Mb broadband)
6
Getting Help
• Best source of help is often the person next to you!
• Also try:
• the person next to the person next to you
• the web
• your supervisor
• computing staff
• email first to computing-help@fmrib.ox.ac.uk
• only go to the office for urgent or special requests
• books/documentation
... and now for something hands-on!
7
Introduction To UNIX
8
Practical 1
Using pepper and background jobs
In this practical we will run the FSL GUI several times,
showing how to have it running in the background (and
at the same time being able to use the terminal).
•
Connect to pepper and get a terminal (run a “shell”)
•
•
•
In the terminal/shell type fsl and press return
You should see an FSL GUI window pop-up
Notice that you cannot do anything useful in the
terminal (e.g. try starting another fsl GUI)
Now just hit Exit on the fsl GUI
•
•
•
Now try this again but type: “fsl &”
Notice how both the FSL GUI and the terminal can
still be used. Finish by exiting the GUI again.
9
Practical 1
Using pepper and background jobs
•
•
•
•
•
Now try “fsl” again (no “&” this time)
When it’s running, press “control z” in the terminal
This gives you the terminal back but you cannot use
the fsl GUI anymore (try it)
To keep the GUI running at the same time type “bg”
in the terminal (for background)
To stop (kill) a job while it is in the foreground
use “control c”: e.g. try “fsl” then control c
Notes:
• A foreground job is one that runs in the terminal and
keeps the terminal busy (unusable) until the job finishes.
• A background job runs at the same time as the terminal.
• Many background jobs can be run at the same time.
10
Files and Directories
• Files are stored in directories/ (folders) and arranged into a tree
hierarchy
home
Scratch
mark
fs0
fs1
fs2
• The terminal is always “in” a directory
‣ we call this the “present working directory” or pwd
• / (forward slash) is a directory separator and the top directory
•
•
•
is called just “/”
The directory “above” is called .. (i.e. dot dot)
The directory you are now in is called . (i.e. dot)
Your own home directory is called ~ (i.e. tilde)
11
Files and Directories
• Files are stored in directories/ (folders) and arranged into a tree
hierarchy
home
Scratch
mark
fs0
• You can refer to files and directories via
fs1
fs2
‣ an absolute path (starting at the root, / , and specifying all
•
subdirectories along the way)
‣ a relative path (starting at the pwd & possibly using . or .. )
Examples (at FMRIB):
‣ Home directory (absolute) /home/fs0/mark
‣ Temp space in /vols/Scratch/mark
‣ A file in the docs directory (inside my home directory):
‣ /home/fs0/mark/docs/list.txt or
~/docs/list.txt or just docs/list.txt (relative)
12
Files and Directories
• The terminal is always “in” a directory
‣ we call this the “present working directory”
File-related commands (basic):
command
ls
ls blah
cd blah
pwd
cp file1 file2
cp file dir
mv blah1 blah2
mkdir blah
description
lists the files in the working directory
lists the files in directory blah
changes to a new directory (blah)
show name of present working directory (where am I?)
copy files (file2 = copy of file1)
copy files into a destination directory
move (or rename) files or directories (blah2 = new name)
make a new (empty) directory
Note: often several files and a directory can be used
e.g. cp file1 file2 ... fileN dir (similarly for mv)
13
Practical 2
In this practical we will learn the basics of dealing
with files: listing files, navigating around directories,
copying and renaming files.
•
In the terminal, try “ls” to see what is in your home
directory (you always start in the home directory)
•
•
•
Look at the MRdata directory with “ls MRdata”
Try just “ls” again - note you are still in your home
See where home is by doing “pwd”
•
•
•
•
•
Now change directory with “cd MRdata”
• Note that when it works nothing is shown on the screen
Check with “pwd” and do “ls”
Look at the directory above with “ls ..”
Change to the directory above and do ls and pwd to check
You can get to the home directory at any time by doing
“cd” on its own
14
Practical 2
•
Copy the file /home/fs0/mark/Graduate/timing.txt to the
home directory:
cp /home/fs0/mark/Graduate/timing.txt ~
•
•
•
Change directory to your home directory
Check the copy worked with ls
Rename the file to timing1.txt
•
•
Make a directory called “stim”
Move timing1.txt into stim and check it worked with ls
Notes:
• Many commands take extra options (see full list using “man”)
• These are flags preceded with a minus sign
• e.g. cp -r dir1 dir2
• this specifies that the copy is to be recursive - copying a
whole directory and all its subdirectories (the whole tree)
15
Files and Directories (2)
• File-related commands (create/destroy/look):
command description
touch blah
create a new (empty) file
rm blah
removes (deletes) file(s)
rmdir blah
removes (deletes) an empty directory
less blah
show contents of a text file - one screen at a time
(type “q” to exit)
cat blah
show contents of a text file - all at once
history
see a list of previous commands
(can rerun one with !N - e.g. !55 runs command number 55)
16
Files and Directories (2)
• File-related commands (create/destroy/look):
command description
touch blah
create a new (empty) file
rm blah
removes (deletes) file(s)
rmdir blah
removes (deletes) an empty directory
less blah
show contents of a text file - one screen at a time
(type “q” to exit)
cat blah
show contents of a text file - all at once
history
see a list of previous commands
(can rerun one with !N - e.g. !55 runs command number 55)
DON’T DO IT! It’s the UNIX equivalent of the atomic bomb!!
rm -rf
17
Practical 3
•
•
•
•
Go to “stim” directory (where you put timing1.txt)
Look at the contents of this file with “less” or
“cat”
Use “touch” to create a new file called timing2.txt
Look at its contents (there should be none)
•
•
Delete the file timing2.txt (and check it is gone)
Delete any other files, move to the home directory
and delete the directory “stim”
•
Without copying the file, look at the contents of:
/usr/local/fsl/data/atlases/HarvardOxford-Subcortical.xml
using first “less” and then “cat”
•
•
See a list of previous commands with “history”
Also try scrolling through with the up/down arrows
18
Files and Directories (3)
• File-related commands (permission and space):
‣ special directories: “~” is home & “.” is pwd
‣ permissions shown with “ls -l”
(-l is one of many options)
permissions
Total 42
drwxr-xr-x
drwxr-xr-x
drwxr-xr-x
drwxr-xr-x
drwxr-xr-x
-rwxr-xr-x
-rw-r--r--
2
2
8
3
2
1
1
owner
group
duncan
duncan
duncan
duncan
duncan
duncan
duncan
analysis
analysis
analysis
analysis
analysis
analysis
analysis
date
size modified
name
8192
8192
8192
8192
8192
138
133
c
gear
perl
scripts
test_scripts
vncserver
vncserver.b
Feb
Apr
Apr
Mar
Apr
Mar
Jan
19
11
11
10
11
27
24
2003
2002
2002
2003
2002
2002
2002
owner group everyone
d = directory ; r = read ; w = write ; x = execute
19
Files and Directories (3)
• File-related commands (permission and space):
command
description
ls -la blah
chmod flags
blah
du -sh blah
df -h .
quota -H
gzip blah
gunzip blah
diff file1 file2
wc blah
list files (+hidden ones) with extra info
changes permissions for files/directories
show disk usage of file/directory blah (space taken up)
show free space left on disk (the one that stores the pwd)
show how much disk space you are allowed
compress (zip) a file
have a guess... (hint: it’s not about weapons)
show differences between the two files
character/word/line count of file blah
Tip: when typing filenames, press tab and it will fill in what it can
20
Practical 4
•
•
Copy /home/fs0/mark/Graduate/timing.txt into your home
directory (again)
Do “ls -la” to see the size and permissions
Do “chmod u-w timing.txt” and then “ls -la” to see
the effect (note that u=user=owner)
Revert to the previous permissions with “chmod u+w”
•
•
See the disk usage here with “du -sh .”
Check the amount of total free space with “df -h .”
•
Copy the following file to the MRdata directory
/usr/local/fsl/data/standard/MNI152_T1_2mm.nii.gz
Look at the size with “ls -la” and “du -sh *.gz”
(note: the *.gz selects all files ending with .gz)
Run “gunzip” on this file and look at the size again
Run “gzip” again to get back to the original
•
•
•
•
•
21
Editing Files (on the server)
• Types of files:
‣ plain text
‣ pdf
‣ doc/tex
‣ images (nii.gz)
‣ etc.
• In UNIX, usually deal with plain text files
• It is not a good idea to use Word or anything similar
• A plain text editor is needed (on the server):
‣ nedit (simple and intuitive)
‣ like TextEdit or NoteBook
‣ emacs (very powerful, less easy, but very common)
22
Practical 5
•
•
Start nedit or emacs (in the background)
Open the file timing.txt (using File menu then Open)
•
Modify the values (or whatever) and save it as
timing3.txt
•
In the terminal look at the contents of the files
timing.txt and timing3.txt
•
Load the original file (timing.txt), modify this and
save it
Experiment with nedit (or emacs) some more...
... and prepare an example stimulus timing file for
the analysis of your experiment (3 columns per event):
onset time (in seconds)
duration (sec)
1
e.g.
48 16 1
(onset=48, duration=16)
•
23
Learning UNIX
•
•
•
•
•
•
Can get help for commands with:
‣ “man”, “help”, “apropos”, web
An intro/primer/easy reference and reading notes are at:
‣ https://www.fmrib.ox.ac.uk/internal/support/computing/
user-guides/unix/
‣ http://www.fmrib.ox.ac.uk/internal/support/computing/
user-guides/unixfordummies.html
Difficult to find a really good book
‣ Tend to be out-of-date
‣ Setup specific
Suggest reading:
‣ UNIX for Dummies, Levine & Levine Young, 4th Ed,
1998; ISBN: 0-7645-0419-3
‣ HARD-CORE: Unix in a Nutshell, 4th Ed, 2005, O’Reilly Press,
Arnold Robbins, ISBN: 0-596-10029-9
Honestly, the best way to learn is to practice with trial & error!
And ask others in the lab.
24