Python Crash Course
Intro, scripts
Bachelors
V1.0
dd 09-12-2014
Hour 5
Why Python?
• Modern scripting languages:
– Python, Perl, Ruby, IDL, Matlab, …
– High-level
– Interactive interpreter
• Ease of use
• Speed of development
• Encourages scripting, rather than one-off
analysis
• Permanent record
• Repeatability
Why not python?
• If you want fastest possible performance
• Highly parallel code
• Need low-level control
Why Python is great
•
•
•
•
•
•
•
•
•
•
Designed to be easy to learn and use – clear syntax
Well documented
Powerful, flexible, fully-featured programming language
‘Batteries included’
Comprehensive scientific tools
Fast
Interpreter, introspection
Runs everywhere
Completely free
You already have it
Why learn Python?
•
•
•
•
Less stress
Get more science done
Widely used and growing popularity
Throughout academia and industry
– NASA, AstraZeneca, Google, Industrial Light & Magic, Philips,…
– Web services, engineering, science, air traffic control,
quantitative finance, games, education, data management, …
• Python programmers in demand
• Easy introduction to general programming concepts
Why not?
• Existing code for your project in another language, but
still…
Python in optical astronomy
• STScI PyRAF (IRAF) + additional Python
only routines
• ESO PyMIDAS (MIDAS)
• STScI PyFITS (access to FITS files)
• Astro-WISE (widefield imaging system)
• Pyephem - solar system ephemeris
• LSST will use Python/C+
• …
Python in radio astronomy
• CasaPy (Casa) - AIPS++, default system for EVLA and ALMA data
analysis.
• ParselTongue - call AIPS tasks from Python
• PYGILDAS (GILDAS) - IRAM data analysis software ported to
Python
• BoA (Bolometer Analysis Package) for LABOCA on APEX and other
bolometers
• APECS (APEX control software)
• KAT-7 CMS is in Python
• Presto - pulsar search and analysis suite; most recent routines in
Pytho
Python in physics
• CERN
– PyROOT (research engine for high energy physics)
– PyMad (simulate particle accelerators)
• Computational physics
• ALPS (Algorithms and Libraries for Physics Simulations)
• …
Introduction to language - start
Linux
• At command line: python myscript.py
• With script: chmod, #!/usr/bin/env python
• At python prompt: execfile(’somefile.py’)
• At ipython prompt: %run somefile.py
Introduction to language - start
Windows
Files that have the
extension .py are known as
Python scripts. In Windows and
Mac OS, these files will appear
to be "clickable", i.e. will appear
to be files that you can open by
clicking them with the mouse.
It is not recommended that you
open these files by clicking on
them. Why? Because quite often
the result can be unpredictable.
Instead, start IDLE and open
Python scripts inside an IDLE
session.
Introduction to language - startup
pczaal2: python
Python 2.7.5 (default, Nov 3 2014, 14:26:24)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 2+2
4
>>> # This is a comment
... 2+2
4
>>> 2+2.0 # and a comment on the same line as code
4.0
>>> (50-5*6)/4
5
>>> width = 20 # assignment, no type declaration
>>> height = 5*9
>>> width * height
900
>>> x = y = z = 0 # zero x, y and z
>>> y
0
>>> n
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'n' is not defined
Introduction to language - scripts
2+2
# This is a comment
2+2
2+2.0 # and a comment on the same line as code
(50-5*6)/4
width = 20 # assignment, no type declaration
height = 5*9
width * height
x = y = z = 0 # zero x, y and z
y
• Can write in a text editor and copy and paste into
interpreter
• Can save and execute from command line:
$ python test.py
• Can save and use interactively in future sessions
(import)
Python script
– Start an editor and type some python commands. Save your file as: myfirst.py
– On the Unix command line type: python myfirst.py
– We want to be able to start the program by only typing its name. To make it
executable use chmod u+x myfirst.py or chmod +x myfirst.py if you want
to allow everybody on the system to execute your program
– Run it with: ./myfirst.py The dot slash part is necessary to force execution if
your current directory is not included in the settings for your path!
– If you type command echo $path on the command line, then a list is displayed
with directories with the paths to these directories included. If you type the name
of an executable file on the command line then all the directories in the list are
visited to look for the wanted executable. If the name is found, it will be executed.
If not, then you get a warning.
– Now we get an error message. Remember that we created a script and the first
line of a script should contain the so called shebang line which defines the path
to the application that has to execute the script
– Add in your script a shebang line as the first line: #!/usr/bin/env python
– Run it with: ./myfirst.py It should give you the right answer.
Introduction to language - numbers
>>> 10 + 3
13
>>> 10 - 3
7
>>> 10 * 3
30
>>> 10 / 3
3
>>> 10 // 3
3
>>> 10 % 3
1
>>> 10**3
1000
>>> 10 + 3 * 5 # *,/ then +,25
>>> (10 + 3) * 5
65
>>> -1**2 # -(1**2)
-1
>>> 10.0 + 3.0
13.0
>>> 10.0 - 3.0
7.0
>>> 10.0 * 3
30.0
>>> 10.0 / 3
3.3333333333333335
>>> 10.0 // 3
3.0
>>> 10.0 % 3.0
1.0
>>> 10.0**3
1000.0
>>> 4.2 + 3.14
7.3399999999999999
>>> 4.2 * 3.14
13.188000000000001
Introduction to language - numbers
Integer division is weird!
Mixing Integer and Floating
•Integer division truncates
•When you perform an operation where one
operand is an integer and the other operand is
a floating point the result is a floating point
•Floating point division produces floating
point numbers
>>> 10 / 2
5
>>> 9 / 2
4
>>> 99 / 100
0
>>> 10.0 / 2.0
5.0
>>> 99.0 / 100.0
0.99
•The integer is converted to a floating point
before the operation
>>> 99 / 100
0
>>> 99 / 100.0
0.99
>>> 99.0 / 100
0.99
>>> 1 + 2 * 3 / 4.0 - 5
-2.5
Arithmetic Operators
Assume variable a holds 10 and variable b holds 20 then:
Operator
Description
Example
+
Addition - Adds values on either a + b will give 30
side of the operator
-
Subtraction - Subtracts right
hand operand from left hand
operand
*
Multiplication - Multiplies values a * b will give 200
on either side of the operator
/
Division - Divides left hand
operand by right hand operand
b / a will give 2
%
Modulus - Divides left hand
operand by right hand operand
and returns remainder
b % a will give 0
**
Exponent - Performs exponential a**b will give 10 to the power
(power) calculation on operators 20
//
Floor Division - The division of
9//2 is equal to 4 and 9.0//2.0 is
operands where the result is the equal to 4.0
quotient in which the digits after
the decimal point are removed.
a - b will give -10
Arithmetic Operators Precedence
•
•
•
•
•
•
Highest precedence rule to lowest precedence rule
Parenthesis are always respected
Exponentiation (raise to a power)
Multiplication, Division, and Remainder
Addition and Subtraction
Left to right
Parenthesis
Power
Multiplication
Addition
Left to Right
1.
2.
3.
4.
5.
1 + 2 ** 3 / 4 * 5
1+8/4*5
1+2*5
1 + 10
11
Numerical types
Integers:
Long integers:
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> 2**1000
>>> 2L, 3l
>>>
1111111111111111111111111111111111111
11111
>>> float(2), float(2**1000)
>>> int(2.3), int(-2.3)
>>> int(2**1000), long(2), str(2)
2
0
-4711
07, 022 # Octal tuple
0x9, 0xa, 0XF # Hexadecimal tuple
17 + 4 # Expression
0xa - 2
23 ** (2+3) # Power
7 / 2, 7 / -2 # Int division
from __future__ import division
7/2
Floats:
>>>
>>>
>>>
>>>
>>>
>>>
>>>
2.3
-4.
0.1, .1
2.99E10, 6.62607e-27, -1e10
1.7 + .4
17. + 4
7./2., 7./2, 7/2.
Complex numbers:
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
2.+3j, 2-3J # complex literals
j # will not work
1J # but this will
complex(1,2)
# Watch operator precedence:
1+1j*2, (1+1j)*2
(2.+3j).real, (2+3j).imag
type(2-3j)
Introduction to language - variables
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
x
x
x
y
x
x
x
x
= 2 # Assign variable
# Display
+ 3 # Use variable
= x + 3 # New variable
= x + 1 # Assign new value
+= 1 # Shorthand; but no x++
= 12.3 + 98.7j # Change type
**= 2j
Some tricks:
>>> x, y = 2, 3
>>> x, y = y, x # No temporary
variables needed
>>> x = y = z = 1
>>>
>>>
>>>
>>>
>>>
xy, Xy
9x = 2
x9 = 2
_x = 2
if = 2
=
#
#
#
#
2, 3 # Case sensitive
Not allowed, must begin w. letter
ok
ok, but special
must not be keyword
Reserved keywords:
and del from not while
as elif global or with
assert else if pass yield
break except import print
class exec in raise
continue finally is return
def for lambda try
None
as with
Introduction to language - type
•
•
•
•
Python knows what “type” everything is
Some operations are prohibited
You cannot “add 1” to a string
We can ask Python what type something is
by using the type() function.
>>> eee = 'hello ' + 'there‘
>>> eee = eee + 1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: cannot concatenate 'str' and 'int' objects
>>> type(eee)
<type 'str'>
>>> type('hello')
<type 'str'>
>>> type(1)
<type 'int'>
Assignment Operators
Assume variable a holds 10 and variable b holds 20 then:
Operator
=
+=
-=
*=
/=
%=
**=
//=
Description
Example
Simple assignment operator, Assigns values
c = a + b will assigne
from right side operands to left side operand
value of a + b into c
Add AND assignment operator, It adds right
c += a is equivalent to c =
operand to the left operand and assign the
c+a
result to left operand
Subtract AND assignment operator, It
c -= a is equivalent to c =
subtracts right operand from the left operand c - a
and assign the result to left operand
Multiply AND assignment operator, It multiplies c *= a is equivalent to c =
right operand with the left operand and assign c * a
the result to left operand
Divide AND assignment operator, It divides left c /= a is equivalent to c =
operand with the right operand and assign the c / a
result to left operand
Modulus AND assignment operator, It takes
modulus using two operands and assign the
result to left operand
Exponent AND assignment operator, Performs
exponential (power) calculation on operators
and assign value to the left operand
c %= a is equivalent to c
=c%a
c **= a is equivalent to c
= c ** a
Floor Dividion and assigns a value, Performs
c //= a is equivalent to c =
floor division on operators and assign value to c // a
the left operand
Assignment Operators
A variable is a memory location used to
store a value (0.6)
x
0.6
0.6
0.6
x = 3.9 * x * ( 1 - x )
0.4
Right side is an expression.
Once expression is evaluated,
the result is placed in (assigned
to) X..
0.93
Assignment Operators
A variable is a memory location used to
store a value. The value stored in a
variable can be updated by replacing the
old value (0.6) with a new value (0.93).
x
0.6
0.93
x = 3.9 * x * ( 1 - x )
Right side is an expression.
Once expression is evaluated,
the result is placed in (assigned
to) the variable on the left side
(i.e. x).
0.93
Python environment
• PYTHONSTARTUP
$ls /usr/lib64/python2.7/site-packages
– Personal startup file defined in startup file:
abrt_exception_handler.py
setenv PYTHONSTARTUP /home/personal/mystartup.py
abrt_exception_handler.pyc
abrt_exception_handler.pyo
– all code in startup file will be executed upon start
abrt.pth
acutilmodule.so*
audit.py
– tells the Python interpreter where to locate the module files you import into a
audit.pyc
program
audit.pyo
setenv PYTHONPATH
_audit.so*
auparse.so*
“/usr/lib64/python2.7/site-packages:/home/personal/python/site-packages”
Avogadro.so*
basemap-1.0.6-py2.7.egg-info
_blueman.so*
Brlapi-0.5.6-py2.7.egg-info
brlapi.so*
cairo/
...
• PYTHONPATH
ipython
• What is it
– interactive shell for the Python programming language that offers
enhanced introspection, additional shell syntax, tab completion and rich
history.
• Why
– default interactive Python shell can sometimes feel to basic
– gives you all that you get in the basic interpreter but with a lot extra (line
numbers, advanced editing, more functions, help functions etc)
Python 2.7.5 (default, Nov 3 2014, 14:26:24)
Type "copyright", "credits" or "license" for more information.
IPython 1.1.0 -- An enhanced Interactive Python.
?
-> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help
-> Python's own help system.
object?
-> Details about 'object', use 'object??' for extra details.
From .ipython starup env
In [1]:
Python environment
.ipython/:
.ipython/profile_default:
db/
– history
Personal startup file defined in startup history.sqlite
file:
profile_default/
log/
setenv PYTHONSTARTUP /home/personal/mystartup.py
pid/
– .ipython/db:
all code in startup file will be executed security/
upon start
shadowhist/
startup/
shadowhist_idx
– ~/.ipython directory structure
.ipython/profile_default/db:
– .ipython/db/shadowhist:
all scripts in .ipython/profile_default/startup
are executed upon start
rootmodules
– 05new profile can be created using:
21
.ipython/profile_default/log:
$ ipython profile create profile_name
2e
– 61and used:
.ipython/profile_default/pid:
7a $ ipython --profile=profile_name
90
.ipython/profile_default/security:
95
b4
.ipython/profile_default/startup:
f3
README
• PYTHONSTARTUP
db/
• Profile
ipython
• TAB completion
– especially for attributes, is a convenient way to explore the structure of
any object you’re dealing with
– besides
Python
In [4]:
x.__objects and keywords, tab completion also works on file
x.__abs__names
x.__hash__
x.__reduce__
and directory
x.__add__
x.__class__
In [1]: from sys
import std
x.__coerce__
stderr stdin x.__delattr__
stdout
x.__init__
x.__int__
x.__le__
x.__long__
x.__div__
x.__lt__
In [1]: from urllib2
import
url
x.__divmod__
x.__mod__
url2pathname x.__doc__
urlopen
urlparse
x.__mul__
x.__eq__
x.__ne__
x.__float__
x.__neg__
x.__floordiv__
x.__new__
x.__format__
x.__nonzero__
x.__ge__
x.__pos__
x.__getattribute__ x.__pow__
x.__getformat__
x.__radd__
x.__getnewargs__
x.__rdiv__
x.__gt__
x.__rdivmod__
x.__reduce_ex__
x.__repr__
x.__rfloordiv__
x.__rmod__
x.__rmul__
x.__rpow__
x.__rsub__
x.__rtruediv__
x.__setattr__
x.__setformat__
x.__sizeof__
x.__str__
x.__sub__
x.__subclasshook__
x.__truediv__
x.__trunc__
ipython
• Magic
IPython -- An enhanced Interactive Python - Quick Reference Card
================================================================
obj?,
– built in commands
obj??
: Get help, or
– %quickref
more help for object (also works as
?obj, ??obj).
In [57]: lsmagic
?foo.*abc*
: List names in 'foo' containing 'abc' in them.
%magic
: Information about IPython's 'magic' % functions.
Available line magics:
%alias %alias_magic %autocall %autoindent %automagic %bookmark %cd %colors %config
Magic functions are prefixed by % or %%, and typically take their arguments
%cpaste %debug %dhist %dirs %doctest_mode %ed %edit %env %gui %hist %history
without parentheses, quotes or even commas for convenience. Line magics take a
%install_default_config %install_ext %install_profiles %killbgscripts %load %load_ext
single % and cell magics are prefixed with two %%.
%loadpy %logoff %logon %logstart %logstate %logstop %lsmagic %macro %magic
%notebook %page %paste %pastebin %pdb %pdef %pdoc %pfile %pinfo %pinfo2 %popd
Example magic function calls:
%pprint %precision %profile %prun %psearch %psource %pushd %pwd %pycat %pylab
%quickref %recall %rehashx %reload_ext %rep %rerun %reset %reset_selective %run
%alias d ls -F
: 'd' is now an alias for 'ls -F'
%save %sc %store %sx %system %tb %time %timeit %unalias %unload_ext %who %who_ls
alias d ls -F
: Works if 'alias' not a python name
%whos %xdel %xmode
alist = %alias
: Get list of aliases to 'alist'
cd /usr/share
: Obvious. cd -<tab> to choose from visited dirs.
Available cell magics:
%cd??
: See help AND source for magic %cd
%%! %%bash %%capture %%file %%perl %%prun %%ruby %%script %%sh %%sx %%system
%timeit x=10
: time the 'x=10' statement with high precision.
%%timeit
%%timeit x=2**100
x**100
: time 'x*100' with a setup of 'x=2**100'; setup code is not
Automagic is ON, % prefix IS NOT needed for line magics.
counted. This is an example of a cell magic.
ipython
• Input caching system
– input is saved and can be retrieved as variables
– _i, previous, _ii, next previous, _iii …etc.
• Macros
– macros are great for executing the same code over and over
– associate a name with a section of Python
code so the code can be run
In [1]: a=2
later by referring to the name
In [2]: b=3
In [1]:
In [2]:
In [3]:
In [4]:
Out[4]:
In [5]:
Out[5]:
In [6]:
Out[6]:
In [7]:
Out[6]:
In [8]:
Out[8]:
a=2
b=3
c=a+b
_ii
u'b=3'
_ih[1]
u'a=2‘
In[3]
u'c=a+b‘
print c
5
exec _ih[7]
5
In [3]: c=a+b
In [4]: print c
5
In [5]: %macro xxx 1-2 4
Macro `xxx` created. To execute, type its
name (without quotes).
=== Macro contents: ===
a=2
b=3
print c
In [6]: xxx
5
ipython
• Useful help commands
–
–
–
–
%reset resets the interactive environment
%hist allows you to see any part of your input history
%hist -g somestring
Search (‘grep’) through your history by typing
In [55]: hist -g math
19: import math
55: hist -g math
– %paste use text that you have in the clipboard, for example if you have copied
code with Ctrl+C. The command cleans up certain characters and tries to find out
how the code should be formatted.
– %edit The %edit command (and its alias %ed) will invoke the editor set in your
environment as EDITOR.
– %who This function list objects, functions, etc. that have been added in the
current namespace, as well as modules that have been imported.
In [50]: who
Interactive namespace is empty.
ipython
• Shell access
– Any input line beginning with a ! character is passed verbatim (minus the !) to the
underlying operating system.
In [2]: !ping www.google.com
PING www.google.com (173.194.67.104): 56 data bytes
64 bytes from 173.194.67.104: icmp_seq=0 ttl=49 time=6.096 ms
64 bytes from 173.194.67.104: icmp_seq=1 ttl=49 time=5.963 ms
^C
– You can capture the output into a Python list, e.g.: files = !ls.
ipython
• Aliases
– All of your $PATH has been loaded as IPython aliases, so you should be able to
type any normal system command and have it executed.
In [9]: %alias
Total number of aliases: 12
Out[9]:
[('cat', 'cat'),
('cp', 'cp -i'),
('ldir', 'ls -F -o --color %l
('lf', 'ls -F -o --color %l |
('lk', 'ls -F -o --color %l |
('ll', 'ls -F -o --color'),
('ls', 'ls -F --color'),
('lx', 'ls -F -o --color %l |
('mkdir', 'mkdir'),
('mv', 'mv -i'),
('rm', 'rm -i'),
('rmdir', 'rmdir')]
| grep /$'),
grep ^-'),
grep ^l'),
grep ^-..x'),
ipython
• the four most helpful commands
–
–
–
–
?
%quickref
help
object?
Introduction and overview of IPython’s features.
Quick reference.
Python’s own help system.
Details about ‘object’, use ‘object??’ for extra details.
Assignment Operators
End