Using Algebraic Geometry for
Solving Polynomial Problems
in Computer Vision
David Nistér and Henrik Stewénius
Very Rough Outline
•
•
•
•
•
14:00-15:30 “1st Round", administered by David.
- What is this Tutorial About?
- Motivation, RANSAC, Example Problems
- Gröbner Bases
- Action Matrix
• -15:30-16:00 Coffee break
•
•
•
•
•
16:00-18:00-“2nd Round“, administered by Henrik.
- Background Material
- Gröbner Bases
- Exercises in Macaulay2
-Q&A
What is this Tutorial About?
• If you are interested in solving polynomial
equations, this tutorial is for you
• This tutorial is a ‘real’ tutorial
• The main focus will be on Gröbner bases
and how the theory of Gröbner bases can be
used to solve polynomial equations
History
•
•
•
•
Euclid (325BC-265 BC),
Apollonius (262 BC-190BC),
Descartes (1596-1650),
Fermat (1601-1655)
• Bézout (1730-1783)
• Hilbert’s (1862-1943) Nullstellensatz
I(V( J ))  J
Wolfgang Gröbner (1899-1980) Bruno Buchberger
RISC
Research
Institute for
Symbolic
Computation
Linz, Austria
1966: Ph.D. in mathematics. University of Innsbruck,
Dept. of Mathematics, Austria.
Thesis: On Finding a Vector Space Basis of the Residue Class Ring
Modulo a Zero Dimensional Polynomial Ideal (German).
Thesis Advisor: Wolfgang Gröbner
Bezout’s Theorem
2x2=4
With two variables,
a solution according to
the Bezout bound
can typically be
realized with resultants.
With three or more
variables, things
are less simple.
Mixed Volume, see for example [CLO 1998]
provides a generalization for non-general polynomials.
Resultants
• Provides Elimination
of variables by taking
a determinant
a1x2+a2y2+a3xy+a4x+a5y+a6
b1x2+b2y2+b3xy+b4x+b5y+b6
x3
x3
det
a1


b1


x2
x
1
a3 y  a4
a2 y 2  a5 y  a6
a1
a3 y  a4
b3 y  b4
b2 y 2  b5 y  b6
b1
b3 y  b4


a2 y 2  a5 y  a6 


b2 y 2  b5 y  b6 
[0]

= det 
[0]


x2
x
1
[1] [2]

[0] [1] [2]
= [4]

[1] [2]

[0] [1] [2]
Various States of Mind You May Have, I:
• Skip the theory, what is this all about? –Answers
from David
Useful Theory
2st Quadrant
Useful and Easy
Super!
Easy Theory
1st Quadrant
Useful but Hard
Algebraic Geometry and
Gröbner Bases Difficult Theory
Useless and impossible
Useless and Easy
to penetrate – Unfortunately
survives
3rd Quadrant
4th Quadrant
Useless Theory
Proofs of the Power of Gröbner Bases
•
•
•
•
•
So far, approx. 600 publications and 10 textbooks have been devoted to
Gröbner Bases.
Gröbner Bases are routinely available in current mathematical software
systems like Mathematica, Maple, Derive, Magma, Axiom, etc.
Special software systems, like CoCoa, Macaulay, Singular, Plural, Risa-Asir
etc. are, to a large extent, devoted to the applications of Gröbner Bases.
Gröbner Bases theory is an important section in all international conferences
on computer algebra and symbolic computation.
Gröbner Bases allow, for the first time, algorithmic solutions to some of the
most fundamental problems in algebraic geometry but are applied also in such
diverse areas as functional analysis, statistics, optimization, coding theory,
cryptography, automated theorem proving in geometry, graph theory,
combinatorial identities, symbolic summation, special functions, etc.
Various States of Mind You May Have, II:
• Stop the handwaving, what is the rigorous
theory?
Answers from David and Henrik + books
Suggested Literature
• D. Cox, J. Little, D. O’Shea, Ideals, Varieties,
and Algorithms, Second Edition, 1996.
• D. Cox, J. Little, D. O’Shea, Using Algebraic
Geometry, Springer 1998.
• T. Becker and Weispfennig, Gröbner Bases, A
Computational Approach to commutative Algebra,
Springer 1993.
Suggested Literature
• B. Buchberger, F. Winkler (eds.) Gröbner
Bases and Applications
Cambridge University Press, 1998.
• Henrik’s Thesis: H. Stewénius, Gröbner
Basis Methods for Minimal Problems in
Computer Vision, PhD Thesis, 2005
• Planned scaffolding paper, keep a lookout
on the tutorial web page
Various States of Mind You May Have, III:
• Skip the theory, how do I use it? – Answers
from Henrik, Exercises in Macaulay 2.
RANSAC- Random Sample Consensus
Robust
RANSAC
Least Squares
RANSAC- Random Sample Consensus
Robust
Line Hypotheses
Points
RANSAC
Hypothesis
Generator
Observation
Likelihood
?
Hypotheses
500
Observations
1000
500 x 1000 = 500.000
Preemptive RANSAC
Hypothesis Generation
Observed Tracks
Estimate or
posterior likelihood
output
Hypothesis
Generator
Precise
Formulation
Probabilistic
Formulation
Data Input
Geometry Tools
3D-3D 2D-2D
Absolute Orientation
Triangulation
2D-3D
Pose
2D-2D
Relative Orientation
Bundle Adjustment
Robust Statistics
For Which Problems Did We Use
Gröbner Bases?
No
The 5-Point Relative
Pose Problem
10
Yes
Yes, you bet
Unknown Focal
Relative Pose
64
15
Generalized Relative
Pose
2048
The Generalized
3-Point Problem
3 View
Triangulation
8(4)
47
Microphone-Speaker
Relative Orientation
The 3 View 4-Point
Problem
0 (or thousands)
8-38-150-344-??
The Generalized 3-Point Problem
The Generalized 3-Point Problem
The Five Point Problem
Given five point correspondences,
What is R,t ?
The 5-point algorithm (Nistér PAMI 04)


 

2 EE E  trace( EE ) E
det
Sturm Sequences
for Bracketing
Root Polishing
by Bisection
R,t
E
The 5-point algorithm (Nistér PAMI 04)

 
2 EE  E  trace( EE  ) E
det
Sturm Sequences
for Bracketing
Root Polishing
by Bisection
R,t
E
The 5-point algorithm (Stewénius et al)

2 EE  E  trace( EE  ) E
10 x 10
Action Matrix
Eigen-Decomposition
R,t
E
Easy Conditions
Correct Calibration
Realistic Conditions
5-Point Matlab Executable
Recent Developments on Direct Relative Orientation,
Henrik Stewenius, Christopher Engels, David Nister,
To appear in ISPRS Journal of Photogrammetry and Remote Sensing
www.vis.uky.edu/~dnister
Further Examples of Solved Problems
6-point generalized relative orientation (64 solutions) (Stewenius,
Nistér, Oskarsson and Åström, Omnivis ICCV 2005)
6-point relative orientation with common but unknown focal length
(15 solutions) (Stewenius, Nistér, Schaffalitzky and Kahl,
to appear at CVPR 2005)
Triangulation
Triangulation, 2 Views
•
•
•
•
One parameter family – Balance the error
L2-Norm -> Sextic (Hartley & Sturm)
Max-Norm -> Quartic (Closed form, Nistér)
Directional Error -> Quadratic (Oliensis)
Optimal 3 View Triangulation
work with Henrik Stewenius and Fred Schaffalitzky
ICCV 2005
47 Stationary Points
Microphone-Speaker Location
work with Henrik Stewenius
The 3 View 4 Point Problem
Work with Frederik Schaffalitzky
Geometry-Algebra ‘Dualism’
• Hilbert’s Nullstellensatz
I(V( J ))  J
How Hard is this Problem?
Approximately This Hard
For Which Problems Did We Use
Gröbner Bases?
No
The 5-Point Relative
Pose Problem
10
Yes
Yes, you bet
Unknown Focal
Relative Pose
64
15
Generalized Relative
Pose
2048
The Generalized
3-Point Problem
3 View
Triangulation
8(4)
47
Microphone-Speaker
Relative Orientation
The 3 View 4-Point
Problem
0 (or thousands)
8-38-150-344-??
Camera Geometry
• Often leads to polynomial formulations,
or can at least very often be formulated in
terms of polynomial equations
Polynomial Formulation
• p1(x) , … , pn(x)= A set of input polynomials
(n polynomials in m variables)
x=[y1 … ym]
Main Point
• Gröbner Basis Gives Action Matrix
(because it provides the ability to compute unique ‘smallest’ possible
unique remainders)
• Action Matrix Gives Solutions
Polynomial
Equations
Gröbner
Basis
Action
Matrix
Intensively Studied
Solutions
Algebraic Ideal
• I(p1 , … , pn)= The set of polynomials
generated by the input polynomials
(through additions and multiplications by a polynomial)
p and q in I => p+q in I
p in I
=> pq in I
The ideal I consists of ‘Almost’ all the polynomials implied
by the input polynomials
(More precisely, the radical I of the ideal consists of all)
Basis (for Ideal)
• A basis for the ideal J is a set of polynomials
{p1 , … , pn} such that J=I(p1 , … , pn)
Algebraic Variety
• The solution set
(the vanishing set of the input polynomials)
V(I)={x:I(x)=0}
More precisely
p(x)=0 for all p in I
Quotient Ring J/I
• The set of equivalence classes of
polynomials when only the values on V are
considered (i.e. polynomials are equivalent iff p(V)=q(V))
V(I)
p in J/I
Action Matrix
• For multiplication by polynomial on finite
dimensional solution space
V(I)
Action Matrix
Action Matrix
Transposed
Companion Matrix
An ‘Equivalence’
Compute Companion
Matrix
Finding the Eigenvalues
Finding the Roots
of a Matrix
of a Polynomial
Compute
Characteristic Polynomial
Requires
Gröbner
Compute Action Matrix in Quotient Ring Basis for
(Polynomials modulo Input Equations) Input Equations
Finding the Roots
of a Multiple
Polynomial Equations
Finding the Eigenvalues
of a Matrix
Compute
Characteristic Polynomial
Companion Matrix
Multiplication by x modulo the seventh degree polynomial
x7+ a6x6 +a5x5+ a4x4+ a3x3+ a2x2+ a1x+a0
can be expressed as left-multiplication by the matrix
x6 x5 x4 x3 x2 x 1
-a6 -a5 -a4 -a3 -a2 -a1 -a0
1
1
1
1
1
1
Action Matrix
I
Action Matrix
I
V(I)
Action Matrix
p in J
I
V(I)
Action Matrix
p in J
I
p in J/I
V(I)
Action Matrix
I
p in J/I
Action Matrix
q in J/I
p in J/I
I
Action Matrix
pq in J/I
q in J/I
p in J/I
I
Action Matrix
Multiplication by a polynomial q is a linear
operator Aq
(αp+βr)q=α(pq)+β(rq)
The matrix Aq is called the action matrix for
multiplication by q
Action Matrix
b0
b1
b2
x0
x1
x2
I
Action Matrix
q in J
b0
b1
b2
x0
x1
x2
I
Action Matrix
q in J
q(x0)b0
q(x2)b2
q(x1)b1
I
Action Matrix
q in J
q(x0)b0
The values q(xi) of q at the solutions xi are the
eigenvalues of the action matrix
q(x2)b2
q(x1)b1
I
Action Matrix
The values q(xi) of q at the solutions xi are the
eigenvalues of the action matrix
If we choose q=y1 , the eigenvalues are the
solutions for y1
Action Matrix
b’=[r1 … ro]
b’(x)Aq p=q(x)b’(x)p
for all p in J/I and x in V(I)
b’(x)Aq =b’(x)q(x)
b(x) is a left nullvector of Aq corresponding to eigenvalue
q(x)
Monomial Order
• Needed to define leading term of a polynomial
• Grevlex (Graded reverse lexicographical) order
usually most efficient
y_2
y_1
Gröbner Basis
• A basis for ideal I that exposes the leading
terms of I (hence unique well defined remainders)
• Easily gives the action matrix for
multiplication with any polynomial in the
quotient ring
y_2
y_1
A Reduced Gröbner Basis is a Basis
in the normal sense
• A polynomial in the ideal I can be written as
a unique combination of the polynomials in
a reduced Gröbner basis for I
• The monic Gröbner basis for I is unique
Buchberger’s Algorithm
Buchberger’s Algorithm
Gaussian
Elimination
Euclid’s
Algorithm for the
GCD
Gaussian Elimination
Exposes all the leading terms, which are simply
all the variables in the case of general linear equations
xn
x1
Remember Row Operations:
• Multiplying a row by a scalar
• Subtracting a row from another
• Swap rows
Add:
• Multiplying a row by any polynomial
More General Elimination
With non-linear equations, there are relations between
the monomials that matter when multiplying
y2 xy x2 y x
y2
xy
x2
y x
Multiply by x
y2 xy x2 y x
Multiply by y
Multiplying by a Scalar
Transitions through zero remain
p(x)
3.8p(x)
Adding
Common transitions through zero remain
p1(x)
p2(x)
p1(x) + p2(x)
Multiplying
Transitions through zero remain
p(x)f(x)
p(x)
f(x)
Buchberger’s Algorithm
Compute remainders of S-polynomials until
all remainders are zero
Buchberger’s Algorithm
Compute remainders of S-polynomials until
all remainders are zero
Buchberger’s Algorithm
Compute remainders of S-polynomials until
all remainders are zero
Buchberger’s Algorithm
Compute remainders of S-polynomials until
all remainders are zero
Buchberger’s Algorithm
Compute remainders of S-polynomials until
all remainders are zero
Buchberger’s Algorithm
Compute remainders of S-polynomials until
all remainders are zero
Buchberger’s Algorithm
Compute remainders of S-polynomials until
all remainders are zero
Buchberger’s Algorithm
Compute remainders of S-polynomials until
all remainders are zero
Buchberger’s Algorithm
Compute remainders of S-polynomials until
all remainders are zero
Buchberger’s Algorithm
Compute remainders of S-polynomials until
all remainders are zero
Buchberger’s Algorithm
Compute remainders of S-polynomials until
all remainders are zero
Begin (offline)
Pose Problem. Port to Zp
Approach
Begin (online)
Pose Problem over R
Compute number of solutions Elimination Compute Gröbner basis
Schedule
Build matrix based
Gröbner basis code
Compute Action Matrix
Solve Eigenproblem
Port to R
End
Backsubstitute
End
Prime Field Formulation
• Reals => Cancellation unclear
• Rationals => Grows unwieldy
• Prime Field => Cancellation clear, size is
limited, only small risk of incorrect
cancellation if prime is large
Gaussian Elimination
• Expanding all polynomials up to a certain
degree followed by Gaussian elimination
allows pivoting
Unwanted Solutions
Can be removed by ideal quotients, or more generally saturation
Elimination Example
Elimination Example
Elimination Example
Elimination Example
Elimination Example
Elimination Example
Elimination Example
Elimination Example
Elimination Example
Elimination Example
Action Matrix