Are resultant methods numerically unstable for
multidimensional rootfinding?
University of Oxford, 16th June 2015
NA internal seminar
Alex Townsend
MIT
Joint work with Vanni Noferini
Betteridge’s law of headlines
“Any headline that ends in a question mark can be answered by the word no.”
Are resultant methods numerically unstable for multidimensional
rootfinding?
I do not know
What we do know: The two most popular resultant-based numerical
rootfinders are numerically unstable. Including the one in chebfun2v/roots...
Including Chebfun2’s competitors...
[Sommese & Wampler, 05], [Sorber, Barel, Lathauwer, 14], [Nakasukasa, Noferini, T., 14], [Bini & Mario, 13],
[Wallack, Emiris, Manocha, 98], [Bujnak, Kukelova, & Pajdla, 08], [Allgower, Georg, & Miranda, 92], [Strumfels, 98].
Alex Townsend @ MIT
1/12
Betteridge’s law of headlines
“Any headline that ends in a question mark can be answered by the word no.”
Are resultant methods numerically unstable for multidimensional
rootfinding?
I do not know
What we do know: The two most popular resultant-based numerical
rootfinders are numerically unstable. Including the one in chebfun2v/roots...
Including Chebfun2’s competitors...
[Sommese & Wampler, 05], [Sorber, Barel, Lathauwer, 14], [Nakasukasa, Noferini, T., 14], [Bini & Mario, 13],
[Wallack, Emiris, Manocha, 98], [Bujnak, Kukelova, & Pajdla, 08], [Allgower, Georg, & Miranda, 92], [Strumfels, 98].
Alex Townsend @ MIT
1/12
Betteridge’s law of headlines
“Any headline that ends in a question mark can be answered by the word no.”
Are resultant methods numerically unstable for multidimensional
rootfinding?
I do not know
What we do know: The two most popular resultant-based numerical
rootfinders are numerically unstable. Including the one in chebfun2v/roots...
Including Chebfun2’s competitors...
[Sommese & Wampler, 05], [Sorber, Barel, Lathauwer, 14], [Nakasukasa, Noferini, T., 14], [Bini & Mario, 13],
[Wallack, Emiris, Manocha, 98], [Bujnak, Kukelova, & Pajdla, 08], [Allgower, Georg, & Miranda, 92], [Strumfels, 98].
Alex Townsend @ MIT
1/12
Betteridge’s law of headlines
“Any headline that ends in a question mark can be answered by the word no.”
Are resultant methods numerically unstable for multidimensional
rootfinding?
I do not know
What we do know: The two most popular resultant-based numerical
rootfinders are numerically unstable. Including the one in chebfun2v/roots...
Including Chebfun2’s competitors...
[Sommese & Wampler, 05], [Sorber, Barel, Lathauwer, 14], [Nakasukasa, Noferini, T., 14], [Bini & Mario, 13],
[Wallack, Emiris, Manocha, 98], [Bujnak, Kukelova, & Pajdla, 08], [Allgower, Georg, & Miranda, 92], [Strumfels, 98].
Alex Townsend @ MIT
1/12
Betteridge’s law of headlines
“Any headline that ends in a question mark can be answered by the word no.”
Are resultant methods numerically unstable for multidimensional
rootfinding?
I do not know
What we do know: The two most popular resultant-based numerical
rootfinders are numerically unstable. Including the one in chebfun2v/roots...
Including Chebfun2’s competitors...
[Sommese & Wampler, 05], [Sorber, Barel, Lathauwer, 14], [Nakasukasa, Noferini, T., 14], [Bini & Mario, 13],
[Wallack, Emiris, Manocha, 98], [Bujnak, Kukelova, & Pajdla, 08], [Allgower, Georg, & Miranda, 92], [Strumfels, 98].
Alex Townsend @ MIT
1/12
Multidimensional rootfinding setting
Find all the solutions to:
¨
˛
p1 px1 , . . . , xd q
˚
‹
..
˝
‚ “ 0,
.
px1 , . . . , xd q P Ωd Ă Cd .
pd px1 , . . . , xd q
Polynomials are of maximal degree n (at most degree n in each variable)
Isolated solutions ùñ ď d!nd solutions (Bernstein’s Theorem)
SIAM digit challenge problem [Trefethen, 02]
1
Easiest possible scenario:
1. Simple solutions, i.e., the Jacobian
J px ˚ q´1 exists
2. No solutions at infinity
0.5
w
−0.5
−1
−1
Alex Townsend @ MIT
257s
0
−0.5
0
0.5
1
2/12
Inherit robustness from eigenvalue solver
Main idea: Inherit robustness from eigenvalue solver
˛
p1 px1 , . . . , xd q
˚
‹
..
˝
‚“ 0
.
¨
pd px1 , . . . , xd q
Av “ λv
For a simple root x ˚ , absolute
condition number is:
For a semisimple eigenvalue λ˚ ,
absolute condition number is:
κ2 px ˚ q “ }J px ˚ q´1 }2
κpλ˚ , A q
Alex Townsend @ MIT
3/12
Overview
d“1
dě2
p px q “ 0
p1 px q “ ¨ ¨ ¨ “ pd px q “ 0
Not much going
on here.
n
n
ď d!nd
v “ λv
companion, colleague, comrade
v “ λv
ď d!nd
linearization of a (Cayley) resultant matrix
Conditioning analysis:
[Van Dooren & Dewilde, 83], [Edelman & Murakami, 95],
[Terán, Dopico, & Mackey, 10], [Noferini & Pérez, 15]
Alex Townsend @ MIT
Conditioning analysis:
[Noferini & T., 15]
4/12
Overview
d“1
dě2
p px q “ 0
p1 px q “ ¨ ¨ ¨ “ pd px q “ 0
Lots going on
inside here!
Not much going
on here.
n
n
ď d!nd
v “ λv
companion, colleague, comrade
v “ λv
ď d!nd
linearization of a (Cayley) resultant matrix
Conditioning analysis:
[Van Dooren & Dewilde, 83], [Edelman & Murakami, 95],
[Terán, Dopico, & Mackey, 10], [Noferini & Pérez, 15]
Alex Townsend @ MIT
Conditioning analysis:
[Noferini & T., 15]
4/12
Hidden-variable resultant methods
p “ 0, q “ 0
”Hide” one of the variables, say x1 :
pj rx1 spx2 , . . . , xd q “
n
ÿ
i2 “¨¨¨“id “0
ci2 ,...,id px1 q
d
ź
φis pxs q
s “2
A resultant R is a polynomial such that (ignoring solutions at infinity):
Rpx1˚ q “ 0 ðñ Dpx2˚ , . . . , xd˚ q P Cd ´1
s.t. p1 px ˚ q “ ¨ ¨ ¨ “ pd px ˚ q “ 0,
Alex Townsend @ MIT
R “ resultant
?
? ?
5/12
Hidden-variable resultant methods
p “ 0, q “ 0
”Hide” one of the variables, say x1 :
pj rx1 spx2 , . . . , xd q “
n
ÿ
i2 “¨¨¨“id “0
ci2 ,...,id px1 q
d
ź
φis pxs q
s “2
A resultant R is a polynomial such that (ignoring solutions at infinity):
Rpx1˚ q “ 0 ðñ Dpx2˚ , . . . , xd˚ q P Cd ´1
s.t. p1 px ˚ q “ ¨ ¨ ¨ “ pd px ˚ q “ 0,
Alex Townsend @ MIT
R “ resultant
?
? ?
5/12
A linear example
¨
˛
x1
˚ ‹
A ˝ ... ‚` b “ 0
xd
¨ ˛
1
˚
“
‰ ˚ x2 ‹
‹
x
A
p
:,
1
q
`
b
A
p
:,
2
:
end
q
˚ .. ‹ “ 0
1
looooooooooooooooooomooooooooooooooooooon
˝.‚
“A1 `x1 A p:,1qe1T
xd
`
˘
det A1 ` x1 A p:, 1qe1T “ detpA1 q` x1 detpA q “ 0
Linear rootfinding problem
”Hide x1 ”
Matrix determinant lemma
Cramer’s rule:
x1 “ ´ detpA1 q{ detpA q
Alex Townsend @ MIT
6/12
A linear example
¨
˛
x1
˚ ‹
A ˝ ... ‚` b “ 0
xd
¨ ˛
1
˚
“
‰ ˚ x2 ‹
‹
x
A
p
:,
1
q
`
b
A
p
:,
2
:
end
q
˚ .. ‹ “ 0
1
looooooooooooooooooomooooooooooooooooooon
˝.‚
“A1 `x1 A p:,1qe1T
xd
`
˘
det A1 ` x1 A p:, 1qe1T “ detpA1 q` x1 detpA q “ 0
Linear rootfinding problem
”Hide x1 ”
Matrix determinant lemma
Cramer’s rule:
x1 “ ´ detpA1 q{ detpA q
Alex Townsend @ MIT
6/12
A linear example
¨
˛
x1
˚ ‹
A ˝ ... ‚` b “ 0
xd
¨ ˛
1
˚
“
‰ ˚ x2 ‹
‹
x
A
p
:,
1
q
`
b
A
p
:,
2
:
end
q
˚ .. ‹ “ 0
1
looooooooooooooooooomooooooooooooooooooon
˝.‚
“A1 `x1 A p:,1qe1T
xd
`
˘
det A1 ` x1 A p:, 1qe1T “ detpA1 q` x1 detpA q “ 0
Linear rootfinding problem
”Hide x1 ”
Matrix determinant lemma
Cramer’s rule:
x1 “ ´ detpA1 q{ detpA q
Alex Townsend @ MIT
6/12
A linear example
¨
˛
x1
˚ ‹
A ˝ ... ‚` b “ 0
xd
¨ ˛
1
˚
“
‰ ˚ x2 ‹
‹
x
A
p
:,
1
q
`
b
A
p
:,
2
:
end
q
˚ .. ‹ “ 0
1
looooooooooooooooooomooooooooooooooooooon
˝.‚
“A1 `x1 A p:,1qe1T
xd
`
˘
det A1 ` x1 A p:, 1qe1T “ detpA1 q` x1 detpA q “ 0
Linear rootfinding problem
”Hide x1 ”
Matrix determinant lemma
Cramer’s rule:
x1 “ ´ detpA1 q{ detpA q
Gabriel Cramer
Alex Townsend @ MIT
6/12
A linear example
¨
˛
x1
˚ ‹
A ˝ ... ‚` b “ 0
xd
¨ ˛
1
˚
“
‰ ˚ x2 ‹
‹
x
A
p
:,
1
q
`
b
A
p
:,
2
:
end
q
˚ .. ‹ “ 0
1
looooooooooooooooooomooooooooooooooooooon
˝.‚
“A1 `x1 A p:,1qe1T
xd
`
˘
det A1 ` x1 A p:, 1qe1T “ detpA1 q` x1 detpA q “ 0
Linear rootfinding problem
”Hide x1 ”
Matrix determinant lemma
Cramer’s rule:
x1 “ ´ detpA1 q{ detpA q
Gabriel Cramer
Alex Townsend @ MIT
6/12
Resultant method with matrix polynomials
[Nakatsukasa, Noferini, & T., 14]
Undo the determinant:
Rpx1 q “ detpR px1 qq
Matrix polynomial
Polynomial eigenvalue problem:
Consider
˜
¸
T7 px qT7 py qp2 ` xy q
“ 0,
T5 px qT5 py qp1 ` xy q
where Tk px q “ cospk cos´1 px qq.
R px1 q has semisimple eigenvalues
R px1 qv “ 0
Work by (and many others):
Leiba Rodman
Alex Townsend @ MIT
Israel Gohberg
Françoise Tisseur
Absolute error “ 8.2 ˆ 10´16
7/12
Cayley resultant matrix
Skipping the eye-watering tensor manipulations
RCAYLEY px1 q is a matrix polynomial of size at most pd ´ 1q!nd ´1 of degree dn.
It is the matricization of the tensor of expansion coefficients of
¨
˛
p1 rx1 sps1 , s2 , . . . , sd ´1 q . . . pd rx1 sps1 , s2 . . . , sd ´1 q
˚ p rx spt , s . . . , s q . . . p rx spt , s . . . , s q ‹ O d ´1
ź
d ´1
d 1 1 2
d ´1 ‹
˚ 1 1 1 2
‹
fCayley “ det ˚
psi ´ ti q.
..
..
...
˚
‹
.
.
˝
‚
i “1
p1 rx1 spt1 , t2 , . . . , td ´1 q . . . pd rx1 spt1 , t2 , . . . , td ´1 q
Devastating: In any degree-graded basis, there exists a polynomial system and
a simple root x ˚ such that
Absolute conditioning
can increase by an
exponential factor
Alex Townsend @ MIT
κpRCAYLEY , x1˚ q ě ||J px ˚ q´1 ||d .
[Noferini & T., 15]
8/12
Cayley resultant matrix
Skipping the eye-watering tensor manipulations
RCAYLEY px1 q is a matrix polynomial of size at most pd ´ 1q!nd ´1 of degree dn.
It is the matricization of the tensor of expansion coefficients of
¨
˛
p1 rx1 sps1 , s2 , . . . , sd ´1 q . . . pd rx1 sps1 , s2 . . . , sd ´1 q
˚ p rx spt , s . . . , s q . . . p rx spt , s . . . , s q ‹ O d ´1
ź
d ´1
d 1 1 2
d ´1 ‹
˚ 1 1 1 2
‹
fCayley “ det ˚
psi ´ ti q.
..
..
...
˚
‹
.
.
˝
‚
i “1
p1 rx1 spt1 , t2 , . . . , td ´1 q . . . pd rx1 spt1 , t2 , . . . , td ´1 q
Devastating: In any degree-graded basis, there exists a polynomial system and
a simple root x ˚ such that
Absolute conditioning
can increase by an
exponential factor
Alex Townsend @ MIT
κpRCAYLEY , x1˚ q ě ||J px ˚ q´1 ||d .
[Noferini & T., 15]
8/12
Cayley resultant matrix
Skipping the eye-watering tensor manipulations
RCAYLEY px1 q is a matrix polynomial of size at most pd ´ 1q!nd ´1 of degree dn.
It is the matricization of the tensor of expansion coefficients of
¨
˛
p1 rx1 sps1 , s2 , . . . , sd ´1 q . . . pd rx1 sps1 , s2 . . . , sd ´1 q
˚ p rx spt , s . . . , s q . . . p rx spt , s . . . , s q ‹ O d ´1
ź
d ´1
d 1 1 2
d ´1 ‹
˚ 1 1 1 2
‹
fCayley “ det ˚
psi ´ ti q.
..
..
...
˚
‹
.
.
˝
‚
i “1
p1 rx1 spt1 , t2 , . . . , td ´1 q . . . pd rx1 spt1 , t2 , . . . , td ´1 q
Devastating: In any degree-graded basis, there exists a polynomial system and
a simple root x ˚ such that
Absolute conditioning
can increase by an
exponential factor
Alex Townsend @ MIT
κpRCAYLEY , x1˚ q ě ||J px ˚ q´1 ||d .
[Noferini & T., 15]
8/12
Sylvester resultant matrix
A popular alternative in 2D is the Sylvester resultant
ř
řn
k
k
RSYLV px1 q associated with p1px1, x2 q “ m
k “0 akpx1 qx2 and p2px1, x2 q “ k “0 bkpx1 qx2 :
¨
a0 px1 q
˚
˚
˚
˚
˚
˚
RSylv px1 q “ ˚
˚b0 px1 q
˚
˚
˚
˚
˝
a 1 px 1 q
...
a m px 1 q
..
..
..
.
.
.
a 0 px 1 q
a 1 px 1 q
b1px1 q
...
bn px1 q
..
..
..
.
.
b 0 px 1 q
.
b 1 px 1 q
..
.
...
..
.
...
˛ ,
/
/
.
‹ /
‹
‹ n rows
‹ /
/
‹ /
a m px 1 q‹ ‹,
‹/
‹/
‹/
.
‹
‹ m rows
‚/
/
/
bn px1 q -
Devastating: For any degree-graded basis, there exists a polynomial system
and a simple root x ˚ such that
Absolute conditioning
can be squared
Alex Townsend @ MIT
κpRSYLV , x1˚ q ě ||J px ˚ q´1 ||2 .
[Noferini & T., 15]
9/12
Sylvester resultant matrix
A popular alternative in 2D is the Sylvester resultant
ř
řn
k
k
RSYLV px1 q associated with p1px1, x2 q “ m
k “0 akpx1 qx2 and p2px1, x2 q “ k “0 bkpx1 qx2 :
¨
a0 px1 q
˚
˚
˚
˚
˚
˚
RSylv px1 q “ ˚
˚b0 px1 q
˚
˚
˚
˚
˝
a 1 px 1 q
...
a m px 1 q
..
..
..
.
.
.
a 0 px 1 q
a 1 px 1 q
b1px1 q
...
bn px1 q
..
..
..
.
.
b 0 px 1 q
.
b 1 px 1 q
..
.
...
..
.
...
˛ ,
/
/
.
‹ /
‹
‹ n rows
‹ /
/
‹ /
a m px 1 q‹ ‹,
‹/
‹/
‹/
.
‹
‹ m rows
‚/
/
/
bn px1 q -
Devastating: For any degree-graded basis, there exists a polynomial system
and a simple root x ˚ such that
Absolute conditioning
can be squared
Alex Townsend @ MIT
κpRSYLV , x1˚ q ě ||J px ˚ q´1 ||2 .
[Noferini & T., 15]
9/12
Absolute versus relative conditioning
Quote: ”Usually, it is the relative condition number that is
of interest, but it is more convenient to state results for the
absolute condition number” [N. Higham]
Devastating example in absolute and relative conditioning:
Let u be a small real positive parameter and d ě 2. Take
¯
´?
?
2
2
2
p2i ´1 px q “ x2i ´1 ` u 2 x2i ´1 ` 2 x2i ,
Ωd “ r´1, 1sd ,
´?
¯
?
p2i px q “ x2i2 ` u 22 x2i ´1 ´ 22 x2i ,
1 ď i ď td {2u,
where if d is odd take pd px q “ xd2 ` uxd .
Alex Townsend @ MIT
d
2
3
5
10
u “ ||J px ˚ q´1 ||2
6.7 ˆ 107
1.6 ˆ 105
1.4 ˆ 103
36.7
10/12
Absolute versus relative conditioning
Quote: ”Usually, it is the relative condition number that is
of interest, but it is more convenient to state results for the
absolute condition number” [N. Higham]
Devastating example in absolute and relative conditioning:
Let u be a small real positive parameter and d ě 2. Take
¯
´?
?
2
2
2
p2i ´1 px q “ x2i ´1 ` u 2 x2i ´1 ` 2 x2i ,
Ωd “ r´1, 1sd ,
´?
¯
?
p2i px q “ x2i2 ` u 22 x2i ´1 ´ 22 x2i ,
1 ď i ď td {2u,
where if d is odd take pd px q “ xd2 ` uxd .
Alex Townsend @ MIT
d
2
3
5
10
u “ ||J px ˚ q´1 ||2
6.7 ˆ 107
1.6 ˆ 105
1.4 ˆ 103
36.7
10/12
Absolute versus relative conditioning
Quote: ”Usually, it is the relative condition number that is
of interest, but it is more convenient to state results for the
absolute condition number” [N. Higham]
Devastating example in absolute and relative conditioning:
Let u be a small real positive parameter and d ě 2. Take
¯
´?
?
2
2
2
p2i ´1 px q “ x2i ´1 ` u 2 x2i ´1 ` 2 x2i ,
Ωd “ r´1, 1sd ,
´?
¯
?
p2i px q “ x2i2 ` u 22 x2i ´1 ´ 22 x2i ,
1 ď i ď td {2u,
where if d is odd take pd px q “ xd2 ` uxd .
Alex Townsend @ MIT
d
2
3
5
10
u “ ||J px ˚ q´1 ||2
6.7 ˆ 107
1.6 ˆ 105
1.4 ˆ 103
36.7
10/12
Negative result with (hopefully) a positive impact
dě2
A BIG bag of numerical tricks
Numerical trick
Will it help?
Newton polishing
No
Homogenization
No
Change of basis
No
Domain subdivision
For low d
Regularization
For large solutions
Local refinement
Only sometimes
Rotate coordinates
Not always
Variable transform
It may or may not
Randomization
Not analyzed
Reformulation as min
Not analyzed
p1 px q “ ¨ ¨ ¨ “ pd px q “ 0
It’s got to happen
before here!
ď d!nd
v “ λv
ď d!nd
linearization of a resultant matrix
Conditioning analysis:
[Noferini & T., 15]
Alex Townsend @ MIT
11/12
Thank you
If you know of a practical and robust numerical multidimensional rootfinding.
Call me!
+1 (857) 204-3609
Thanks also to: Anthony Austin, Daniel Bates, John Boyd, Martin Lotz, Nick
Higham, Gregorio Malajovich, Yuji Nakatsukasa, Bor Plestenjak, Andrew
Sommese.
Alex Townsend @ MIT
12/12