Slides: Asaf Shapira & Oded Schwartz;
Sonny Ben-Shimon & Yaniv Nahum.
Notes: Leia Passoni, Reuben Sumner, Yoad Lustig &
Tal Hassner.
(from Oded Goldreich’s course lecture notes)
1
Introduction
This lecture covers:
 Space Complexity
 Non-Deterministic
space
2
Complexity Functions
4.1
Def: A function f is called constructible if it
satisfies the following conditions:

Positive: f: N+  N+

Monotone: f(n+1)  f(n) for all n

Constructive:  a Turing Machine Mf that,
on input x, outputs a string of size f(|x|),
in time O(|x|+f(|x|)), and in space
O(f(|x|))
3
Constructible functions



4.2
Many “popular” complexity functions, e.g. n,
log(n), n2, n! satisfy the above criteria
Odd things may occur, in regard to
relations between complexity classes, if we
don't choose these functions properly
Note: We therefore use time constructible
functions for time bound and space
constructible functions for space bound
4
4.3
Space Complexity - The model:
3-tape Turing machine:
1.
Input Tape – Read Only
2.
Enables sub
linear space
Output tape – Write only
[Omitted for decision problems]
[Usually unidirectional]
3.
Work tape – Read & Write
Length of which corresponds to space usage
5
What kind of TM should we
use?


Any multi-tape TM can be simulated by
an ordinary TM with a polynomial loss of
efficiency
Hence, from here on, a TM will refer to
the 3 tape TM described above
6
DSPACE - Definition
For every TM, M, and input x:
WM(x) = The index of the rightmost cell on the
work-tape scanned by M on x
SM(n) = max|x|=n WM(x)
cL(x) = 1 if xL, 0 otherwise
DSPACE(S(n)) =
{L| DTM M, x M(x)=
Maximal amount of
space used by M for
input of length n
cL(x) and n SM(n)  S(n) }
7
Sub-Logarithmic Space



4.4
DSPACE(O(1)) is equivalent to the set of
regular languages.
Do we gain any strength by having sublogarithmic sapce?? Or formally…
DSAPCE(o(log(n)))  DSPACE(O(1)) ?
or is it
DSPACE(o(log(n))) = DSPACE(O(1)) ?
8
DSPACE(o(log(n))) 
DSPACE(O(1))
Thm:
DSPACE(o(log(n))) is a proper superset of
DSPACE(O(1))
Proof: We will construct a language L
s.t. LDSPACE(loglog(n)), however,
LDSPACE(O(1)), which will prove the
theorem (log(log(n)) = o(log(n))) .
9
Proof (contd.) - Definition of L
L = {xk|kN,xk=Bk 0$Bk 1$Bk 2$…$Bk (2k-1)$}
’
’
’
’
where:
Bk i = Binary representation of i of length k.
’
For example:
x2 = 00$01$10$11$
10
Proof (contd.)
Claim 1: LDSPACE(O(1)) (RegularLanguages)
Proof: By using the “Pumping Lemma”
Claim 2: LDSPACE(loglog(n))
Proof: We will show an algorithm for
deciding L that uses loglog(n) space.
11
The wrong way of proving Claim 2
1) Check that the first block is all 0’s and
works if
that the last is all 1’s.
xL
2) For any two consecutive blocks, check
that the second is the binary
increment of the first.
Clearly (1) can be done in constant space,
and (2) in log(k) space which is
loglog(n) space, as n=|xk|=(k+1)2k
this might use more then loglog(n) if xk  L (e.g. 0m$1m$) m=n/2 - 1
12
The correct solution
m=1
while (true) {
 check that the last m bits of the first block are
all 0’s
 check that the last m bits of the Bk i blocks form
’
an increasing sequence mod 2m, and that
each
block has m bits.
 check that the last m bits of the last block are all
1’s
 if you found an error, return false
 if m is the exact size of Bk i return true.
’
 m = m +1
}
13
The correct solution – An
Example
input: 000$001$010$011$100$101$110$111$
m=1, 1 left bits increasing mod 21=2
000$001$010$011$100$101$110$111$
m=2, 2 left bits increasing mod 22=4
000$001$010$011$100$101$110$111$
m=3, 3 left bits increasing mod 23=8
000$001$010$011$100$101$110$111$
The entire series is increasing  return true
14
Sub-Logarithm
Conclusion:
L  Space(O(loglog(n)))\Space(O(1)) 
DSPACE(o(log(n)))  DSPACE(O(1))
In fact, the above claim does
not work for o(loglog(n)), that is:
DSPACE(o(loglog(n))=DSPACE(O(1))
15
Configuration - Definition
Def: A configuration of a TM M, is a
complete description of its state at a
computation stage, comprising:
1. the state of M (range |QM|)
2. contents of the worktape (range 2s(n))
3. the head position on the input tape
(range n)
4. the head position on the worktape
(range s(n)).
16
#Configuration – An upper
bound
Let C be the number of possible
configuarations of a TM M.
C  |QM|* 2s(n) * n * s(n)
number of
states
contents of
the
worktape
head
position on
worktape
head
position on
input tape
17
4.5
Relation between time and space
Thm: s(n) ≥ log(n)
DSPACE(s(n))  Dtime(2O(s(n)))
Proof:
For every L  DSPACE(s(n)),There is a TM M that
uses no more than O(s(n)) space on input x.
 the number of configurations of M  2O(s(n)) .
 if M does not stop after 2O(s(n)) steps, it must
pass through the same configuration twice, which
implies an infinite loop.
18
How to Make TMs halt?
4.6
Thm: For s(n) ≤ log(n), for any TM, MDSPACE(s(n)),
there is a TM, M’DSPACE(O(s(n)) s.t. L(M’)=L(M),
and M’ always halts.
Proof: By simulation. Given x, M’ computes the
maximal number of configurations C -- that takes
s(|x|) space.
Now M’ simulates M. If it arrives at an answer in
less than C steps, it returns it.
Otherwise (M is in an infinite loop) M’ returns ‘no’.
This stage also takes s(|x|), so the total is
O(s|x|).
19
Space Hierarchy
4.7
Thm: for any s1(n), s2(n), if s1(n)  log(n), s2(n)
is space-constructible and s1(n)=o(s2(n)) then
DSPACE(s1(n)) ≠ DSPACE(s2(n))
Proof: By diagonalization. We construct a
language L, such that L  DSPACE(s2(n)), but
L can’t be recognized by any TM using s1(n)
space:
Let c0 be a constant, 0 c0 1.
L = { x | x = <M> 01*, |<M>|  c0s2(|x|), M
rejects x using ≤ c0s2(|x|) space }
20
Space Hierarchy
Claim: L  DSPACE(s2(n))
Proof: By a straightforward algorithm.
1.
check if x is of the right form. (O(1) space)
2. compute S:= c0s2(|x|). (s2(|x|) space)
3. check that |<M>|  c0s2(|x|). (log(S) space)
4. simulate M.
if the bound S has been exceeded – reject.
if M rejects x – accept, else reject.
Altogether O(s2(|x|)) space as claimed.
21
Space Hierarchy
Claim: L  DSPACE(s1(n))
Proof: We show that for every TM M of space
complexity s1(n), L(M)  L.
s1(n)=o(s2(n))  n0. s1(n0)  c0*s2(n0).
Assume, by way of contradiction, a TM M0 of
space complexity s1(n), s.t. |<M0>|  c0*s2(n0),
accepting L.
Observe M0 result on the input string
n0  M0 1
M0 01
22
L  DSPACE(s1(n)) – Proof contd.
1.
2.
if M0 accepts x, then by definition x  L.
if M0 rejects x, then since |<M0>| 
c0s2(n0), and M0 on x uses at most s1(n0)
 c0*s2(n0) space, it must be that x  L.
In any case this is a contradiction to the
fact that M0 accepts L.
23
5.1
Non Deterministic Space


Def: A non-deterministic Turing
machine - NDTM is a TM with a nondeterministic transition function, having
a work tape, a read-only input tape, and
a unidirectional write-only output tape.
The machine is said to accept input x if
there exists a computation ending in an
accepting state.
24
Def: On-line / Off-line TM


An offline (online) non-deterministic
TM has a work tape, a read-only input
tape, a unidirectional write-only
output tape, and a two-way (one-way)
read-only guess tape.
The machine is said to accept input x
if there exists a content y to the
guess tape that causes the machine’s
computation to end in an accepting
state.
25
Nspaceon, Nspaceoff
Def: Nspaceon(S) = { L | there exist an
online-NDTM ML, which uses ≤ S(|x|)
space, that accepts x iff xL }
Def: Nspaceoff(S) = { L | there exist an
offline-NDTM ML, which uses ≤ S(|x|)
space, that accepts x iff xL }
26
NDTM = Online-NDTM
5.2
Claim: the NDTM model is equivalent to the
online-NDTM model
Proof: We show that a language L is
decidable by a NDTM in time O(T) and
space O(S) iff L is decidable by an onlineNDTM with same time and space bounds.
 Use the guess string y to determine which
transition function to take every step.
 Guess the content of the next cell in the
guess string when read. Remember the last
step’s guessed letter (using internal state)
when guess-tape-head doesn’t move.
27
Nspaceon vs. Nspaceoff
5.3
Thm: Nspaceon(S)  Nspaceoff(log(S))
We’ll simulate an online-NDTM Mon
that uses space S, using an offlineNDTM Moff that uses space log(S).
 Moff guesses a sequence of
configurations of Mon and then
validates it to be an accepting run

28
Nspaceon vs. Nspaceoff
The guess string has blocks, each
representing a configuration [of
length (O(S))]
guess string
doesn’t count in
the space of
Moff
with ≤ 2O(S) blocks
(any valid sequence of configurations
with more blocks, must have the same
configuration twice, therefore can be
replaced by a shorter guess)
29
Nspaceon vs. Nspaceoff
Moff will validate that:
 The first block is a legal
starting configuration.
 Last block is a legal accepting
configuration.
 Every block can result by a legal
move applied to previous block
(carried out two consequetive
blocks at a time)
30
Nspaceon vs. Nspaceoff

The (supposedly) configuration strings:
... $aaaabcaa$ aaaabchaa $ aaaabxaha $aaaabcaa$aaaab$...
O(log(|Con|))
$aaaabchaa$
$aaaabxaha$
1. Check (almost) all
symbols in strings are
identical, and string
lengths identical.
2. Check that symbols
marked with head
position in 1st
configuration
transforms to a legal
threesome on the 2nd.
O(1)
31
Nspaceon vs. Nspaceoff


The working tape holds a counter of the
location in the configuration checked log(O(S)), and O(1) additional space for
the validation.
A counter for the number of
configuration checked - O(S) - is not
necessary!
32
Nspaceon vs. Nspaceoff

Note that this simulation can’t be done
by the online machine, as it has to read
forwards & backwards on the guess tape
(block size being a function of n).
33
Nspaceon vs. Nspaceoff
Thm: Nspaceoff(S)  Nspaceon(2O(S))
Proof: The proof of this theorem
also uses simulation of one machine
using the other.
34
Savitch’s Theorem
5.4
Thm: NL = Nspace(log(n))  DSPACE(log2(n))
We later generalize the theorem to read:
S(n)log(n)  Nspace(S)  DSPACE(S2)
Def: a Configuration Graph is a graph that,
given a TM M which works in space S on an
input x, has one vertex for every possible
configuration of M’s computation on x, and
an edge (u,v) if M can change move from u
to v.
35
Savitch’s Thm - Reducing
Acceptance to Reachability


If there is more then one accepting
configuration, another vertex t is added,
with edges (u,t) for all accepting
configurations’ vertices u.
The starting configuration’s vertex is
named s.
The question of M accepting x reduces to an
s-t reachability problem on the
configuration graph.
We next show reachability in
DSPACE(log2(n)).
36
Savitch’s Theorem
The Trick:
If there is a path from vertex u to v
of size d>0, then there must be a
vertex z, s.t. there is a path from u to
z, shorter then d/2, and a path from
z to u, shorter then d/2.
Note: As we try to save space, we can
afford trying ALL possible z’s. Time
complexity does not matter.
37
The Algorithm
Boolean PATH(a,b,d) {
Both use
if there is an edge from a to b then
the
return TRUE
same
else {
space
if d=1 return FALSE
for every vertex v (except a,b) {
if PATH(a,v, d/2) and PATH(v,b, d/2)
then return TRUE
}
return FALSE
}
}
38
Why log2(n)?
1.
2.
3.
The binary representation of all
numbers used by the algorithm is at
most of size of O(log(n)).
As the d parameter is reduced to half
at each recursive call, the recursion
tree is of depth O(log(n)).
Therefore at each step of the
computation, we use at most O(log(n))
numbers of size O(log(n)) resulting in
O(log2(n)) total space.
39
2
Example of Savitch’s
algorithm
3
1
4
boolean
boolean PATH(a,b,d)
PATH(a,b,d) {
boolean
PATH(a,b,d)
{{
boolean
PATH(a,b,d)
{
if
if there
there is
is an
an edge
edge from
from a
to b
then
if
there
is
an
edge
from
aa to
to
bb then
then
if
there
is
an
edge
from
a
to
b
then
return TRUE
TRUE
return
return
TRUE
return
TRUE
return
TRUE
else
else {
else
{{
else
{
if (d=1)
(d=1) return
return FALSE
FALSE
if
if
(d=1)
return
FALSE
if
(d=1)
return
FALSE
if
(d=1)
return
FALSE
for
for every
every vertex
vertex v
(not a,b)
a,b) {
for
every
vertex
vv (not
(not
a,b)
{{
for
every
vertex
v
(not
a,b)
{
if
if PATH(a,v,
PATH(a,v, d/2)
d/2) and
and
if
PATH(a,v,
d/2)
and
if
PATH(a,v,
d/2)
and
PATH(v,b, d/2)
d/2) then
then
PATH(v,b,
PATH(v,b,
d/2)
then
PATH(v,b,
d/2)
then
PATH(v,b,
d/2)
then
return
return TRUE
TRUE
return
TRUE
return
TRUE
}
}}
}
return FALSE
FALSE
return
return
FALSE
return
FALSE
return
FALSE
}
}}
}
}
}
}}}
(a,b,c)=Is there a path from a to b, that takes no more
than c steps.
(1,4,3)(2,4,1)
(1,4,3)(1,3,2)(1,2,1)TRUE
(1,4,3)(1,3,2)(1,2,1)
(1,4,3)(3,4,1)TRUE
(1,4,3)(1,3,2)
(1,4,3)(1,2,2)TRUE
(1,4,3)(1,3,2)(2,3,1)
(1,4,3)(1,2,2)
(1,4,3)(2,4,1)FALSE
TRUE
(1,4,3)
(1,4,3)(1,3,2)(2,3,1)TRUE
(1,4,3)(3,4,1)
(1,4,3)(1,3,2)TRUE
Log2(3)
40
Applying s-t-reachability to
Savitch’s theorem.



Given a NDTM Mn working in space
log(n), we construct a DTM M working in
log2(n) in the following way:
Given x, M solves the s-t reachability on
the configuration graph of (Mn,x).
Note: the graph is generated “on
demand”, reusing space, therefore M
never keeps the entire representation
of the graph.
41
Appling s-t-reachability to
Savitch’s thm.
Since Mn works in log(n) space it
has O(2log(n)) configurations, its
configuration graph is of size
O(2log(n)) and reachability is
solved in log2(O(2log(n)) )= log2(n)
space.
42
Savitch’s theorem - conclusion

NL  DSPACE(log2(n))
This is not just a special case of Savitch
Thm but equivalent.

As we’ll see next

43
Generalization of the proof
Note that in the last argument, We could
have substituted the log(n) function by any
function, and thus derive the general Savitch
Theorem:
S(n)log(n)  Nspace(S)  DSPACE(S2).
We will next prove a lemma that will help us
generalize any theorem proved for small
functions to larger ones. Specifically, we will
generalize the NL  DSPACE(log2(n))
theorem
44
Translation Lemma-(Padding
argument)
5.5
For space constructible functions
s1(n), s2(n)  log(n), f(n)  n:

Nspace(s1(n))  DSPACE(s2(n))

Nspace(s1(f(n)))  SPACE(s2(f(n)))
45
Padding argument


Let LNPspace(s1(f(n)))
There is a 3-Tape-NDTM ML which
accepts L in NPspace (s1(f(n)))
|x|
Input
babba•
•
Work
••
O(s1(f(|x|)))
46
Padding argument


Define L’ = { x0f(|x|)-|x| | xL }
We’ll show a NDTM ML’ which decides L’ in
the same space as ML.
n’=f(|x|)
Input
babba00000000000000000000000000000000•
•
Work
••
O(s1(n’)) = O(s1(f(|x|))
47
Padding argument – ML’
NSpace(log(n’))
1. Count 0’s backwards, mark end of
x and check f(|x|)-|x| = 0’s length
n'
2. Run ML on x.
NSpace(s1(f(n))) =
NSpace(s1(n’))
Input
babba#0000000000000000000000000000000•
•
Work
••
O(s1(n’))
48
Padding argument
Total Nspace(O(s1(n’)))
n'
Input
babba#0000000000000000000000000000000•
•
Work
••
O(s1(n’))
49
Padding argument – M’L’
ML’  NPspace(s1(n))
using Nspace(s1(n))  DSPACE(s2(n)) :
there is a M’L’, deterministic TM,
which accept L’ in DSPACE(s2(n))
Given M’L’ , we will construct a DTM
M*L that accept L in O(s2(f(n))
space.
50
Padding argument – M*L
1.
2.
3.
4.
Run M’L’ on input x.
Whenever M’L’ head leaves the x part of
the input, use counter to simulate the
head position.
check that M’L’ doesn’t use more than
s2(f(|x|)) space.
This can be checked because s2 and f are
M’L’ can be simulated by
constructible.another
DTM which receives
the original input, by “imagining” the 0’s,
and counting the place of the imaginary head,
when “reading” to the right of the input.
51
Padding argument – particular
case
L  Nspace(n)

L’  Nspace(Log(n’)) = NL(n’)

L’  Space(Log2(n’))

L  Space(Log2(2n))

L  Space(n2)
n’ = 2n
by NL  DSPACE(log2(n))
n’ = 2n
52
Padding argument – particular
case
Therefore
NL  DSPACE(log2(n))

Nspace(n)  DSPACE(n2)
53