Generating a d-dimensional
linear subspace efficiently
Raphael Yuster
SODA’10
Problem statement
• Computing the rank of a matrix, and, more generally,
generating a row-reduced equivalent matrix is a
fundamental computational algebraic problem.
• Standard approach: Gaussian elimination.
This takes O(nω) < O(n2.38) time [H-B 74’, I-M-H 82’] .
• No faster algorithm is known for computing the rank of
general n  n matrices.
• If matrix is sparse (only m << n2 non-zero entries), can
we compute a row reduced matrix faster?
Not known (and probably impossible via G.E.)
2
Problem statement and main result
• If we allow randomness, the rank can be computed faster.
O(m n) Monte Carlo [Wiedemann 86’].
O(n3-1/(ω -1)) if m=O(n) Las Vegas [Eberly et al. 07’].
• Recall: If B is a row-reduced matrix of A then:
B spans Rows(A) .
B has rank(A) rows.
Any d rows of B span a d-dim. subspace of Rows(A) .
• Our main result:
generating a d-dimensional subspace of Rows(A) requires
(in some cases significantly) less time than computing a
row-reduced matrix.
3
Main result
• Let A be an n  n over an arbitrary field, with m nonzero
entries, and let d  n:
There is an algorithm that computes a d  n matrix
with full row rank which is a subspace of Rows(A).
If rank(A) < d the algorithm generates a rank(A)  n
row-reduced matrix of A.
Dense case
• The running time is: Sparse case
O(min{ n2-2/ω m1/ωdω-2+1/ω , n2dω-2 } ) .
4
Implications and comparison
• Can be used to answer queries of the form “rank(A) > d ?”
• A row-reduced matrix of A, and thus rank(A), computable
in time which is a function of the rank (binary search).
O(min{ n2-2/ω m1/ω rank(A)ω-2+1/ω , n2 rank(A)ω-2 } )
• If the matrix is dense, the runtime is O(n2dω-2) < O(n2d0.38).
For d = o(n) this is faster than doing G.E. from scratch.
• Even more efficiency if matrix is sparse:
5
Implications and comparison
• If m = n2-ε then for d suff. small we get better than
O(n2dω-2) and sometimes even better than O(n2) :
d < (n2/m)0.528 < (n2/m) 1/(ω-1)2
d < (n2/m)
sub-quadratic runtime.
running time O(n2dω-2 (md/n2)1/ω).
o(1)
6
An application
• Given a graph G with n vertices what’s the cardinality
M(G) of the maximum matching?
• [Lovász 79’] proved that computing 2M(G) (randomly)
reduces to computing the rank of an n  n matrix over a
small finite field.
• Thus, his algorithm computes M(G) in O(nω) time.
• Our algorithm computes M(G) in O(n2 M(G)ω-2) time,
which is faster whenever M(G) = o(n) .
7
Proof of main result - outline
Proof consists of two parts:
1. Reducing the computation of a d-dim. subspace of a
sparse matrix to the computation of a d-dim. subspace
of a dense, but smaller, rectangular matrix.
2. Computing a d-dim. subspace of a (possibly dense,
possibly rectangular) matrix by repeatedly filtering out
non-essential rows and replacing other rows with a
smaller equivalent set of rows spanning the same
subspace.
8
Proof of main result – part 1
We need a result of Ibarra, Moran and Hui 82’:
Lemma 1: Let B be an l  n matrix. Then a row-reduced
matrix can be computed in O(nl ω-1) time.
We prove:
Sparse Lemma: Let A be an n  n matrix with m nonzeroes. Let d  n2/m. For any s with d  s  n there is a
matrix L with at most dn/s rows so that:
• If rank(A) < d then L is row-equivalent to A.
• If rank(A)  d then Rows(L) is a subspace of dimension
at least d of Rows(A).
• L is constructed in O(msω-1) time.
9
Proof of main result – part 1
Proof of Lemma 2:
Partition the rows of A into t = n/s rectangular matrices,
A1,…,At , each with s rows.
Let ci denote the number of non-zero columns of Ai.

t
c m
i 1 i
Compact Ai by discarding zero columns (but remember
their locations) thereby obtaining an s  ci matrix Bi.
Apply Lemma 1 to each Bi to obtain reduced equivalent
matrices Ci with ci columns and rank(Bi)=rank(Ai) rows.
10
Proof of main result – part 1
The time required to construct Ci is O(cisω-1).
So, to construct all C1,…,Ct takes O(msω-1).
Let Di be the matrix obtained from Ci by taking the first d
rows of Ci. Expand Di by reinserting the zero columns.
If rank(Ai) < d then Di is row-equivalent to Ai
Otherwise, Di spanning a d-dim. subspace of Rows(Ai).
11
Proof of main result – part 1
A1
A2
A3
B1
4
0
1
0
3
0
4
4
0
9
0
3
0
4
0
6
8
2
5
0
2
0
1
0
2
4
2
0
0
8
3
0
0
4
0
0
0
8
0
8
0
0
0
1
0
0
2
1
3
0
0
6
3
0
8
0
0
1
0
2
1
0
4
4
1
3
4
4
0
9
0
5
0
3
6
9
0
0
0
3/4
-11/4
-3
-1
2
0
0
7
9
4
0
1
0
7
4
1
3
4
4
0
9
3
4
6
8
2
5
2
1
2
4
2
C1
D1
4
0
1
0
3
0
4
4
0
0
0
3/4
0
-11/4
0
-3
-1
2
12
Proof of main result – part 1
Let L be the union of the Di. L has at most dt = dn/s rows.
If rank(A) < d then L is equivalent to A, as each Di is
equivalent to Ai.
If rank(A)  d then Rows(L) is a subspace of Rows(A) with
dimension at least d.
13
Proof of main result – part 2
Dense Lemma: Let L be an l  n matrix and let d  l /2. We
construct L* with at most l /2 rows so that:
If rank(L) < d then L* is row-equivalent to L.
If rank(L)  d then Rows(L*) is a subspace of Rows(L)
with dimension at least d.
The running time is O(lnd ω-2).
14
Proof of main result – part 2
Proof of the main result – the algorithm:
We are given an n  n matrix A containing m nonzero
entries, and a positive integer d.
•If d < n2/m apply the sparse lemma with a value s that
will be chosen later after optimization.
The result is a matrix L0 with n column and
l0  dn/s rows satisfying the conditions of the lemma.
•If d  n2/m we just set L0=A and l0= n.
In any case now Rows(L0) is a subspace of Rows(A) with
dimension at least d (or else L0 is equivalent to A).
15
Proof of main result – part 2
We check if d  l0 /2. If so, we apply the dense lemma to
L0 and obtain L1 with l1  l0 /2 rows.
We repeatedly apply the dense lemma to Li as long as
d  li /2 and obtain the next matrix Li +1.
We halt when d > li /2.
After halting, we perform a final Gaussian elimination on
Li and obtain a reduced matrix B, which is the output of
our algorithm.
16
Proof of main result – part 2
Proof of the main result- correctness:
At any stage of the algorithm, the rows of the current
matrix Li form a subspace of Rows(A), and hence also at
the end Rows(B) is a subspace of Rows(A).
Thus, if B has (at least) d rows then the final output is (at
least) a d-dimensional subspace of Rows(A).
If Rank(A) < d then each of the Li, as well as the final B, is
row-equivalent to A.
17
Proof of main result – part 2
Proof of the main result- running time:
•Consider first the case when d  n2/m. In this case l0= n
and the application of the dense lemma on Li requires
O(2-in2d ω-2) time.
After O(log n) applications the final G.E. to obtain B takes
O(nd ω-1) time by Lemma 1.
The overall runtime is O(n2d ω-2) as required.
•Consider next the case when d < n2/m.
The application of the sparse lemma requires O(msω-1)
time.
18
Proof of main result – part 2
As l0  dn/s, the dense lemma on Li requires:
O(2-is-1n2d ω-1) time.
After O(log n) applications the final G.E. to obtain B takes
O(nd ω-1) time by Lemma 1.
Thus, the overall running time is
O(msω-1 + s-1n2d ω-1)
Choosing optimal value for s which is s = (d ω-1n2/m)1/ω we
obtain a runtime of
O(n2-2/ω m1/ωdω-2+1/ω)
as required.
19
Thanks
20