CIS4930/CDA5125: Parallel and
Distributed Systems
Instructor: Xin Yuan, 168 Love, xyuan@cs.fsu.edu, 4-9133
• Office hour: 10:30am-11:20am M, W, F, or by appointment
Course website:
http://www.cs.fsu.edu/~xyuan/cda5125/index.html
CIS4930/CDA5125 Parallel and Distributed Systems
Florida State University
Course objectives
• Systematically introduce parallel and distributed systems (PDS)
and programming
• Expose contemporary PDS technologies
 Processors, networking, system software, and programming
paradigms
• Study the trends of technology advances in PDS.
• At the end of the course, the students are expected to
 Explain general concepts and important issues in parallel architectures,
programming, and algorithms.
 Understand the challenges in parallel and distributed computing
 Program with different programming paradigms for different
architectures.
CIS4930/CDA5125 Parallel and Distributed Systems
Florida State University
Topics covered
• Architecture
– Architecture classification
– SIMD, MIMD, shared memory (SMP, multicore, SMT, heterogeneous multi-core,
many-core), distributed memory (cluster, massively parallel, grid and cloud).
• Programming
– Vector programming (SSE), Thread, OpenMP, MPI, CUDA, and OpenCL
• Algorithm
– PRAM, BSP
• Advanced
– Power and others
CIS4930/CDA5125 Parallel and Distributed Systems
Florida State University
Textbok
• Required: none
• Reference books:
– D. E. Culler and J. P. Singh, "Parallel Computer Architecture, A
Hardware/Software Approach," Morgan Kaufmann, 1999.
– Maurice Herlihy and Nir Shavit, "The Art of Multiprocessor Programming,"
Morgan Kaufmann, 2008.
• A significant portion of the course materials come
from online resources.
–
–
–
–
MPI: http://www-unix.mcs.anl.gov/mpi/
OpenMP: http://www.openmp.org
CUDA: NVIDIA CUDA Programming Guide
OpenCL: OpenCL Programming Guide for the CUDA Architecture
CIS4930/CDA5125 Parallel and Distributed Systems
Florida State University
Course schedule (tentative)
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Week 1: Introduction and architectures classification
Week 2: Performance
Week 3: SIMD/Vector architectures, SSE and SIMD intrinsics
Week 4: Shared memory architectures
Week 5: Shared memory programming, thread
Week 6: Shared memory programming, OpenMP
Week 7: MIMD, distributed memory architectures, MPI
Week 8: communication hardware and Interconnection networks
Week 9: Stream architectures, CUDA/OpenCL
Week 10: Spring break
Week 11: CUDA/OpenCL
Week 12: Performance monitoring and Optimization
Week 13: PRAM and BSP
Week 14: Advanced topics
Week 15: Project presentations
Week 16: Final exam
CIS4930/CDA5125 Parallel and Distributed Systems
Florida State University
Prerequisite
• COP 4610: Operating systems
– No requirement for background knowledge of
parallel and distributed computing
– The class is designed to minimized the
dependence on COP4610.
CIS4930/CDA5125 Parallel and Distributed Systems
Florida State University
Grading
• 25 % - homework, programming assignment, class participation.
– Homework includes paper critics
– Programming assignments can be done by groups of two.
• 25% - course project
– Research or development
– Groups of two
– End product: report, software, and presentation.
• 20% - Midterm
• 30% - Final
• Final target letter grade: half A’s and half B’s.
CIS4930/CDA5125 Parallel and Distributed Systems
Florida State University
Course project
• Development projects
– Parallelizing a fairly significant application (preferably some well known
efficient sequential code).
• Evaluation projects
– E.g. Evaluating SSE performance on different processors, evaluate
different synchronization algorithms on one or more processors
• Research projects
– Survey an emerging area (e.g. techniques to improve power efficiency in
multi-core machines, emerging programming models for multi-core
systems).
– Develop new techniques in any area of PDS (e.g. new communication
algorithms, new synchronization algorithms, new scheduling schemes,
etc)
CIS4930/CDA5125 Parallel and Distributed Systems
Florida State University
Course policies
• Attendance: required.
• Late assignments: not accepted without a valid
excuse.
• Missed exam: following the university rules.
– Let me know when you need to miss an exam ASAP.
• Incomplete grade:
– Miss the final with an accepted excuse
– Due to extraordinary circumstances with appropriate
documentation.
CIS4930/CDA5125 Parallel and Distributed Systems
Florida State University
• Academic Integrity
– No copying from anywhere
– Don’t ask others for solutions and don’t give
solutions to others.
• Violation
– The university requires all violations to be
reported.
– First violation with level 1 agreement:
• 0 for the particular assignment/exam and the lowering of
one letter (A->B) for course final grade.
– Second violation: resolved through the office of the
Dean and the Faculties
CIS4930/CDA5125 Parallel and Distributed Systems
Florida State University
• Accommodation for Disabilities
1.Register with and provide documentation to the
Student Disability Resource Center (SDRC).
2.Bring a letter to me from the SDRC `indicating you
need academic accomodations. This should be done
within the first week of class
CIS4930/CDA5125 Parallel and Distributed Systems
Florida State University