Software Carpentry Boot Camp
Python
V0.1
dd 07-04-2015
Boot Camp Introduction
Introductory session
•Install all necessary software
http://www.strw.leidenuniv.nl/python/1ejaar/
PythonBootCamp.ppt
•Get you at a basic level
http://www.strw.leidenuniv.nl/python/1ejaar/
or more in depth
http://www.strw.leidenuniv.nl/python/1ejaar/
Boot Camp What Software
Required software
•Python (2 or 3) https://www.python.org/downloads/
–
–
–
–
–
Numpy http://www.scipy.org/scipylib/download.html
Scipy http://www.scipy.org/scipylib/download.html
Matplotlib http://matplotlib.org/users/installing.html
AstroPy (FITS) http://www.astropy.org/
h5py (HDF) http://docs.h5py.org/en/latest/build.html
•Cython http://docs.cython.org/src/quickstart/install.html
•f2py (part of numpy)
•Git http://git-scm.com/book/en/v2/Getting-Started-Installing-Git
Optional:
•ipython http://ipython.org/install.html
Boot Camp Install
Different Operating systems
This document will help you out installing the required software for the
Python Software Carpentry Boot Camp on the following operating
systems:
MacOS
Linux
 FC
 Ubuntu
Windows
Boot Camp Install
Python 2 or Python3
python2 is legacy, python 3 is the present and future, but…
Reason http://docs.python.org/py3k/whatsnew/3.0.html
•
•
•
•
Language cleanup
Better unicode support strings
Several core aspects made more consistent
Class new-style
Why still use Python 2
• Legacy
• All required packages are python 2 and 3 compatible
• Many astronomical packages are python 2 ony
• Many OS-es still use version 2
Boot Camp Install MacOS
 MacOS https://www.python.org/downloads/mac-osx/
• If you have Yosemite (10.10.x):
$ python --version
Python 2.7.6
• Need Python 2.7.9 (latest version)
• install this next to the official OS related python
• Prerequisites:
• gcc compiler (for Cython)
• gfortran compiler (for f2py)
• Go to the AppStore, search Xcode and ‘Install App’, may
take some time as size is 1Gb
– use activity monitor to track progress…
Boot Camp Install on MacOS
• Go to the AppStore, search Xcode and ‘Install App’, may
take some time as size is 1Gb
– use activity monitor to track progress…
– will install gcc, but we need a more recent version and
$ which gcc
/usr/bin/gcc
$ gcc –v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-includedir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX
10.10.sdk/usr/include/c++/4.2.1
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin14.1.0
Thread model: posix
– still need to install gfortran
$ which gfortran
Boot Camp Install on MacOS
• Go to http://hpc.sourceforge.net/ and download the
appropriate version
gcc-5.0-bin.tar.gz,
gcc-4.9-bin.tar.gz,
gcc-4.8-bin.tar.gz,
gcc-4.7-bin.tar.gz,
gfortran-5.0-bin.tar.gz
gfortran-4.9-bin.tar.gz
gfortran-4.8-bin.tar.gz
gfortran-4.7-bin.tar.gz
(gfortran
(gfortran
(gfortran
(gfortran
only),
only),
only),
only),
updated
updated
updated
updated
Nov 2014 (Yosemite).
Nov 2014 (Mavericks & Yosemite).
Oct 2013 (M. Lion & Mavericks).
July 2012 (Lion & M. Lion).
• and install the newest version of gcc and gfortran
$
$
$
$
cd
cd Downloads
sudo tar –zxvf gcc-5.0-bin.tar.gz –C /
sudo tar –zxvf gfortran-5.0-bin.tar.gz –C /
• which should now be in your path
$ which gcc
/usr/local/bin/gcc
$ which gfortran
/usr/local/bin/gfortran
Boot Camp Install MacOS
 MacOS https://www.python.org/downloads/mac-osx/
• If you have Yosemite (10.10.x):
$ python --version
Python 2.7.6
• Need Python 2.7.9 (latest version)
• install this next to the official OS related python
• Download the appropriate install package
• and run the installer
$ which python
/usr/local/bin/python
$ python –version
Python 2.7.9
Boot Camp Install MacOS
 Additional packages
• use the pip utility for this
$ which pip
/usr/local/bin/pip
• load numpy
$ sudo pip install numpy
...
• load scipy
$ sudo pip install scipy
...
• load matplotlib
$ sudo pip install matplotlib
...
Boot Camp Install MacOS
• load astropy
$ sudo pip install astropy
...
• load h5py, as you can seen, this automatically installs Cython as
well
$ sudo pip install h5py
Downloading/unpacking h5py
Downloading h5py-2.4.0-cp27-nonemacosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_
x86_64.whl (4.3MB): 4.3MB downloaded
Downloading/unpacking Cython>=0.17 (from h5py)
Downloading Cython-0.22-cp27-nonemacosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_
x86_64.whl (3.7MB): 3.7MB downloaded
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.6.1 in
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy$ pip matplotlib
1.9.2-py2.7-macosx-10.6-intel.egg
(from h5py)
...
Installing
collected packages: h5py, Cython
Successfully installed h5py Cython
Cleaning up...
Boot Camp Install MacOS
 GIT http://git-scm.com/download/mac
• download of the .dmg should start automatically
• Double click the package and then double click the installer
$ which git
/usr/bin/git
• iPython install
$ sudo pip install “ipython[notebook]”
...
$ cd /usr/local/bin
$ sudo ln –s ../../../Library/Frameworks/Python.framework/Versions/2.7/bin/ipython
$ which ipython
/usr/local/bin/ipython
Boot Camp Install Linux
 Linux https://www.python.org/downloads/source/
• You can install a fresh version of python:
$ python --version
Python 2.7.8
• Need Python 2.7.9 (latest version)
• install this next to the official OS related python
• Prerequisites:
• gcc compiler (for Cython)
• gfortran compiler (for f2py)
Boot Camp Install on Linux
• Check version of gcc and gfortran in OS
$ which gcc
$ which gfortran
/usr/bin/gcc
/usr/bin/gfortran
$ gcc
–v
$
Using
built-in
specs.
Using built-in
specs.
COLLECT_GCC=gfortran
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.9.2/lto-wrapper
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.9.2/lto-wrapper
Target:
x86_64-redhat-linux
Target:
x86_64-redhat-linux
Configured
with:
../configure
--prefix=/usr
--mandir=/usr/share/man
--infodir=/usr/share/info
Configured
with:
../configure
--prefix=/usr
--mandir=/usr/share/man
--infodir=/usr/share/info
--with-bugurl=http://bugzilla.redhat.com/bugzilla
--enable-bootstrap
--enable-shared
--enable--with-bugurl=http://bugzilla.redhat.com/bugzilla
--enable-bootstrap
--enable-shared
--enablethreads=posix
--enable-checking=release
--enable-multilib
--with-system-zlib
--enablethreads=posix
--enable-checking=release
--enable-multilib
--with-system-zlib
--enable__cxa_atexit
--disable-libunwind-exceptions
--enable-gnu-unique-object
--enable-linker-build__cxa_atexit
--disable-libunwind-exceptions
--enable-gnu-unique-object
--enable-linker-build--with-linker-hash-style=gnu
--enable-languages=c,c++,objc,obj-c++,fortran,ada,go,lto
id id
--with-linker-hash-style=gnu
--enable-languages=c,c++,objc,obj-c++,fortran,ada,go,lto
-- -enable-plugin
--enable-initfini-array
--disable-libgcj
--with-isl=/builddir/build/BUILD/gccenable-plugin
--enable-initfini-array
--disable-libgcj
--with-isl=/builddir/build/BUILD/gcc4.9.2-20150212/obj-x86_64-redhat-linux/isl-install
--with-cloog=/builddir/build/BUILD/gcc4.9.2-20150212/obj-x86_64-redhat-linux/isl-install
--with-cloog=/builddir/build/BUILD/gcc4.9.2-20150212/obj-x86_64-redhat-linux/cloog-install
--enable-gnu-indirect-function
--with4.9.2-20150212/obj-x86_64-redhat-linux/cloog-install
--enable-gnu-indirect-function
--withtune=generic
--with-arch_32=i686
--build=x86_64-redhat-linux
tune=generic
--with-arch_32=i686
--build=x86_64-redhat-linux
Thread
model:
posix
Thread
model:
posix
version
4.9.2
20150212
(Red
4.9.2-6)
(GCC)
gccgcc
version
4.9.2
20150212
(Red
HatHat
4.9.2-6)
(GCC)
– still need to install gfortran
Boot Camp Install on Linux
• Install gcc and gfortran in OS
$ which gcc
$ sudo yum groupinstall "Development Tools and Libraries"
...
# Fedora Core
$ sudo apt-get install buildessentials
...
# Ubuntu
$ which gfortran
$ sudo yum install gcc-gfortran
...
# Fedora Core
$ sudo apt-get install gfortran
# Ubuntu
Boot Camp Install Linux
• Need Python 2.7.9 (latest version)
• install this next to the official OS related python
$ python --version
Python 2.7.6
• Download the source code
$ cd /data2
$ wget https://www.python.org/ftp/python/2.7.9/Python-2.7.9.tgz
Connecting to www.python.org (www.python.org)|23.235.43.223|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16657930 (16M) [application/octet-stream]
Saving to: âPython-2.7.9.tgzâ
Python-2.7.9.tgz
100%[===========================================================>]
in 0.3s
15.89M
46.5MB/s
2015-04-06 15:08:19 (46.5 MB/s) - Python-2.7.9.tgz saved [16657930/16657930]
Boot Camp Install on Linux
• unpack and install in alternative directory
$ tar xzvf Python-2.7.9.tgz
...
$ cd Python-2.7.9
$ ./configure --prefix=/net/bree/data2/mypython/
...
$ make
...
$ make install
...
and set your environment to use this version:
$ vi mypython.csh
setenv PYTHONPATH /net/bree/data2/mypython/site-packages
setenv PATH "/net/bree/data2/mypython/bin/":"$PATH"
$ source mypython.csh
$ which python
/net/bree/data2/mypython/bin/python
Boot Camp Install on Linux
• add python tools
# Let's download the installation file using wget:
wget --no-check-certificate https://pypi.python.org/packages/source/s/setuptools/setuptools1.4.2.tar.gz
# Extract the files from the archive:
tar -xvf setuptools-1.4.2.tar.gz
# Enter the extracted directory:
cd setuptools-1.4.2
# Install setuptools using the Python we've installed (2.7.9)
python setup.py install
and also install pip:
$ curl https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py |python ...
Boot Camp Install Linux
 Additional packages
• use the pip utility for this, note to get your personal one!
$ which pip
/net/bree/data2/mypython/bin/pip
• load numpy
$ pip install numpy
...
• load scipy
$ pip install scipy
...
• load matplotlib
$ pip install matplotlib
...
Boot Camp Install Linux
• load astropy
$ pip install astropy
...
• load h5py, as you can seen, this automatically installs Cython as
well
$ sudo pip install h5py
Downloading/unpacking h5py
Downloading h5py-2.4.0-cp27-nonemacosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_
x86_64.whl (4.3MB): 4.3MB downloaded
Downloading/unpacking Cython>=0.17 (from h5py)
Downloading Cython-0.22-cp27-nonemacosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_
x86_64.whl (3.7MB): 3.7MB downloaded
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.6.1 in
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy$ pip matplotlib
1.9.2-py2.7-macosx-10.6-intel.egg
(from h5py)
...
Installing
collected packages: h5py, Cython
Successfully installed h5py Cython
Cleaning up...
Boot Camp Install Linux
 GIT, with root access install
$ sudo yum install git
$ sudo apt-get install git
# Fedora Core
# Ubuntu
$ whcih git
/usr/bin/git
 Download http://git-scm.com/download/
$ wget https://www.kernel.org/pub/software/scm/git/git-2.3.5.tar.gz
...
$ tar zxvf git-2.3.5.tar.gz
...
$cd git-2.3.5
$ ./configure –prefix=/nen/bree/data2/mypython
...
$ make
...
$ make install
...
Boot Camp Install Linux
• iPython install
$ pip install “ipython[notebook]”
...
$ cd /usr/local/bin
$ sudo ln –s ../../../Library/Frameworks/Python.framework/Versions/2.7/bin/ipython
$ which ipython
/usr/local/bin/ipython
Boot Camp Install Windows
 Windows
– By default Windows does not have python installed
– so get the Windows x86-64 MSI installer
• Prerequisites:
– Also windows does not have any compilers installed either
– gcc compiler (for Cython)
– gfortran compiler (for f2py)
Boot Camp Install Windows
 We wil make use of Anaconda installer as this allows
easy installation pypi based packages
 Go to http://continuum.io/downloads
– and download the Windos 4-Bit Python 2.7 Graphical Installer
Boot Camp Install Windows
Featured Packages
NumPy
$ which pip
SciPy
/usr/local/bin/pip
Pandas
PyTables
h5py
$ sudo pip install
...
Matplotlib
IPython
Spyder
Qt/PySide
VTK
numpy
Numexpr
Astropy
Cython
LLVM, LLVMpy
Numba
Scikit-learn
Scikit-image
NLTK
NetworkX
MayaVi
Boot Camp Install Windows
 GIT http://git-scm.com/download/win
• download of the .exe should start automatically
• Double click the installer and follow the instructions
(allow use from Windows command line)
Data processing example
End