Optimization/Learning on the
GPU (supplement figure slides)
CIS 665
Joe Kider
Pictures/Slides thanks to…
• Jonathan Shewchuk
• Nico Galoppo
• Jeff Bolz
• (Most of this was a blackboard lecture, these slides supplement that,
since drawing the graphs of quadratic forms can be difficult.
For the most part the lecture came from the following 3 sources:
– Jonathan Richard Shewchuk, An Introduction to the Conjugate Gradient
Method Without the Agonizing Pain
– Nico Galoppo et Al., LU-GPU: Efficient Algorithms for Solving Dense
Linear Systems on Graphics Hardware
– Bolz et Al., Sparse Matrix Solvers on the GPU: Conjugate Gradients
and Multigrid
Gauss-Jordon
Graph of a quadratic form f(x)
The minimum point of this surface is the solution to Ax=b
Contours of the quadratic form
Gradient f’(x) of the quadric form
Gradient Descent
Problem graphs
Method of Orthogonal Directions
Conjugate Directions
Gram-Schmidt Conjugation
Conjugate Directions
Conjugate directions using the
Axial unit vectors, also know
As Gaussian Elimination
Conjugate Gradients
Conjugate Gradients
Conjugate Gradients
Conjugate Gradients on the GPU
Conjugate Gradients on the GPU
Example Applications
• Just a few uses:
– GPU sim demo
– Heart wave demo
– Flesh Simulation
– Water Simulation