Parallel and Distributed
Computing Overview and
Syllabus
Professor Johnnie Baker
Guest Lecturer: Robert Walker
Instructors
• Professor Johnnie W. Baker
– Primary Lecturer
• Professor Robert Walker
– Guest lectures on specific architectures
– Guest lecture on his groups VLSI work on
parallel architectures
• Possible guest lecturers from parallel
processing group
– Lecture in areas of expertise
– Occasionally cover classes when I am away
Prerequisites
• Designed to be accessible to all graduate
students in computer science
• Students who are not CS graduate
students may also be qualified to take
course
Textbook and References
• Textbook
– Parallel Programming in C with MPI and
OpenMP
– Michael Quinn, author
– Published by McGraw Hill in 2004
• References for Supplementary Reading
– Classroom Slides will also include additional
information from a wide range of sources
– Any additional needed reference material will
be handed out or posted on course website
Some Features of Course
• Includes coverage of fundamental
concepts of parallel computation, rather
than focusing only on latest trends.
– Often quickly outdated due to rapid
technological changes
• Also covers the currently popular cluster
architectures, shared memory processors,
and the MP language.
– This is the focus of the Quinn Textbook
Course Features (cont.)
• Covers the common techniques for
parallel computation by looking at three
key features for each technique:
– Typical architectural features
– Typical programming languages used
– Typical algorithm design techniques used.
Some Specific Topics
• Fundamental concepts applicable to all
parallel computation.
• Asynchronous (MIMD) distributed memory
computation
– Message passing communications
– Programming using the MPI Language
– Architectural features
– Examples of typical algorithms
Specific Topics (cont.)
• Asynchronous (MIMD) shared memory
computation
– Symmetric Multiprocessors or SMPs
– OpenMP language overview
• Synchronous Computation
– SIMD, vector, pipeline computing
– Associative and Multi-Associative Computing
– Programming using the ASC language
– Programming on WorldScape system with Cn
– Fortran 90 and HPF Language overviews
– Algorithm examples
Specific Topics (cont.)
• Interconnection Networks
– Specific Computer Examples including 2D
mesh, hypercube, etc.
– Synchronous and asynchronous
considerations
• Comparing MIMD & SIMD Computation
using a Real-Time Application
– ATC (Air Traffic Control)
Some Benefits of Course
• While principal focus is on parallel computation,
most information is applicable to distributed
computing.
• There is a wide choice of thesis and dissertation
topics in this area
• Several professors in department work in this
area or make major use of parallel computation
in their research
• Students working on a thesis or dissertation in
another area may benefit from being able to use
parallel computation in their work.
Benefits (cont.)
• Most large computational problems require
a parallel or distributed system to satisfy
the speed and memory requirements
• Parallel computation currently has major
advantages over both distributed
computation and grid computation for
computational intensive problems.
– Programs are normally much simpler
– Architectures are much cheaper
– Grid computing is currently fairly futuristic
Two Complementary Courses
• Parallel & Distributed Computing (Fall)
–
–
–
–
Architectures
Languages
Parallel Programming
Algorithm Examples for some architectures
• Parallel & Distributed Algorithms (Alternate
Springs)
– Important Parallel Models
– Designing Efficient Algorithms for Various Models
– Expect to be next offered in Spring 2009
• PDC and PDA can be taken in either order
– Preference is for PDC to be taken first
Real-Time Systems Course
• Will be offered this Spring as an online
course
• Co-taught with Professor Drew at Ohio
University
• Plan to have a project on implementing a
basic ATC system on a WorldScapeClearSpeed parallel computer.
• The PDC class should provide useful
background information for this course.
Assignments and Grading
• Homework assignments
– Problems assigned for most chapters
– Probably 5-7 different assignments
– Some assignments will involve programming
• Course Grade
– Based on homework, midterm, and final
– Approximate weights
• Homework
40%
• Midterm Exam 30%
• Final Exam
30%
Documented Disabilities
• If you have documented disabilities,
please contact me and make me aware of
your needs.
• For information on disability
accommodations, support, and verification
procedure, please see www.kent.edu/sds
Course Website
• Will be established quickly
• Class slides, assignments, and some
references will be posted on this website.
• Also, an online reference textbook and a
pointer to a second online textbook will be
available at this site.
• First Assignment – Read Chapter 1 in
textbook.