University of Tartu, Institute of Computer Science
Introduction to Scientific Computing
MTAT.08.025
eero.vainikko@ut.ee
Spring 2015
2 Practical information
Lectures: Liivi 2 - 403 WED 10:15
Computer classes: Liivi 2 - 205, THU 10:15 Eero Vainikko
3 eap
Lectures: 16h; Computer Classes: 16h; Independent work: 46h
Final grade forms from :
1. Active partitipation at lectures
2. Stand-up quiz(es)
3. Exam
4. Computer class activities
Course homepage (http://courses.cs.ut.ee/2015/isc)
3 Introduction
1
1.1
Introduction
Syllabus
Lectures:
• Python for Scientific Computing NumPy, SciPy
• Scientific Computing - an Overview
• Floating point numbers, how to deal with roundoff errors
• Large problems in Linear Algebra, condition number
• Memory hierarchies and making use of it
• Numerical integration and differentiation
• Numerical solution of differential and integral equations
• Fast Fourier Transform
1.1 Syllabus
4 Introduction
1.1 Syllabus
Computer Classes (preliminary plan)
1. Python & Sage; Fibonacci numbers; Collatz conjecture
2. Discretization and round-off errors
3. NumPy arrays, matrices; LU-Factorization with Gauss Elimination Method
(GEM)
4. UT HPC server; LU-Factorization and GEM on HPC cluster
5. Floating point numbers
6. Fractals
7. Fourier series and Fast Fourier Transform
8. Discrete-time models and ordinary differential equations (optional: Neural Network; Image denoising)
5 Introduction
1.2
1.2 Literature
Literature
General Scientific Computing:
1. RH Landau, A First Course in Scientific Computing. Symbolic, Graphic, and
Numeric Modeling Using Maple, Java, Mathematica, and Fortran90. Princenton
University Press, 2005.
2. LR Scott, T Clark, B Bagheri. Scientific Parallel Computing. Princenton University Press, 2005.
3. MT Heath, Scientific Computing; ISBN: 007112229X, McGraw-Hill Companies, 2001.
4. JW Demmel, Applied Numerical Linear Algebra; ISBN: 0898713897, Society
for Industrial & Applied Mathematics, Paperback, 1997.
6 Introduction
1.2 Literature
Python:
1. Hans Petter Langetangen, A Primer on Scientific Programming with
Python, Springer, 2009. Book webpage (http://vefur.simula.no/
intro-programming/).
2. Hans Petter Langtangen, Python Scripting for Computational Science. Third
Edition, Springer 2008. Web-site for the book (http://folk.uio.no/
hpl/scripting/).
3. Neeme Kahusk, Sissejuhatus Pythonisse (http://www.cl.ut.ee/
inimesed/nkahusk/sissejuhatus-pythonisse/).
4. Brad Dayley, Python Phrasebook, Sams 2007.
5. Travis E. Oliphant, Guide to NumPy (http://www.tramy.us), Trelgol
Publishing 2006.
7 Introduction
1.3
1.3 Scripting vs programming
Scripting vs programming
1.3.1
What is a script?
• Very high-level, often short, program
written in a high-level scripting language
• Scripting languages:
– Unix shells,
– Tcl,
– Scheme,
– Rexx,
– JavaScript,
– Perl,
– Python,
– Ruby,
• This course: Python (and Sage)
– VisualBasic,
– ...
8 Introduction
1.3.2
1.3 Scripting vs programming
Characteristics of a script
• Glue other programs together
• Extensive text processing
• File and directory manipulation
• Often special-purpose code
• Many small interacting scripts may yield a big system
• Perhaps a special-purpose GUI on top
• Portable across Unix, Windows, Mac
• Interpreted program (no compilation+linking)
9 Introduction
1.3.3
1.3 Scripting vs programming
Why not stick to Java, C/C++ or Fortran?
Features of Perl and Python compared with Java, C/C++ and Fortran:
• shorter, more high-level programs
• much faster software development
• more convenient programming
• you feel more productive
• no variable declarations ,
but lots of consistency checks at run time
• lots of standardized libraries and tools
10 Introduction
1.4
1.4 Scripts yield short code
Scripts yield short code
Consider reading real numbers from a file, where each line can contain an arbitrary
number of real numbers:
1.1 9 5.2
1.762543E-02
0 0.01 0.001
9 3 7
Python solution:
F = open(filename, ’r’)
n = F.read().split()
Perl solution:
open F, $filename;
$s = join "", <F>;
@n = split ’ ’, $s;
11 Introduction
1.5 Performance issues
Ruby solution:
n = IO.readlines(filename).join.split
...Doing this in C++ or Java requires at least a loop, and in Fortran and C quite
some code lines are necessary
1.5
Performance issues
1.5.1
Scripts can be slow
• Perl and Python scripts are first compiled to byte-code
• The byte-code is then interpreted
• Text processing is usually as fast as in C
• Loops over large data structures might be very slow
12 Introduction
1.5 Performance issues
for i in range(len(A)):
A[i] = ...
• Fortran, C and C++ compilers are good at optimizing such loops at compile time
and produce very efficient assembly code (e.g. 100 times faster)
• Fortunately, long loops in scripts can easily be migrated to Fortran or C (or
special libraries like numpy!)
13 Introduction
1.5.2
1.5 Performance issues
Scripts may be fast enough
Read 100 000 (x,y) data from file and write (x,f(y)) out again
• Pure Python: 4s
• Pure Perl: 3s
• Pure Tcl: 11s
• Pure C (fscanf/fprintf): 1s
• Pure C++ (iostream): 3.6s
• Pure C++ (buffered streams): 2.5s
• Numerical Python modules: 2.2s (!)
• Remark: in practice, 100 000 data points are written and read in binary format,
resulting in much smaller differences
14 Introduction
1.5.3
1.5 Performance issues
When scripting is convenient
• The application’s main task is to connect together existing components
• The application includes a graphical user interface
• The application performs extensive string/text manipulation
• The design of the application code is expected to change significantly
• CPU-time intensive parts can be migrated to C/C++ or Fortran
• The application can be made short if it operates heavily on list or hash structures
• The application is supposed to communicate with Web servers
• The application should run without modifications on Unix, Windows, and Macintosh computers, also when a GUI is included
15 Introduction
1.5.4
1.5 Performance issues
When to use C, C++, Java, Fortran
• Does the application implement complicated algorithms and data structures?
• Does the application manipulate large datasets so that execution speed is
critical?
• Are the application’s functions well-defined and changing slowly?
• Will type-safe languages be an advantage, e.g., in large development teams?
16 Python in SC
2
1.5 Performance issues
Python in Scientfic Computing
Slides we might use: Introduction to Scientific Computing with Python (http:
//www.physics.rutgers.edu/grad/509/python1.pdf)