MASSACHUSETTS INSTI TUTE OF TECHNOLOGY
ARTI FI CI AL I NTELLI GENCE LABORATORY
A.I . Memo No. 1376
September, 1992
Localizati on and Posi ti oni ng usi ng
Combi nati ons of Model Vi ews
Ehud Rivli n and Ronen Basri
Abstract
Amet hod for l ocal i zat i on and posi t i oni ng i n an i ndoor envi ronment i s present ed.
i s t he act of recogni zi ng t he envi ronment , and
i s t he act of comput i ng t he e
coordi nat es of a robot i n t he envi ronment . The met hod i s based on represent i ng t he sce
as a set of 2D vi ews and predi ct i ng t he appearance of novel vi ews by l i near combi nat i on
t he model vi ews. The met hod accurat el y approxi mat es t he appearance of scenes under weak
perspect i ve project i on. Anal ysi s of t hi s proj ect i on as wel l as experi ment al resul t s demo
t hat i n many cases t hi s approxi mat i on i s suci ent t o accurat el y descri be t he scene. W
ort hographi c approxi mat i on i s i nval i d, ei t her a l arger number of model s can be acqui red o
i t erat i ve sol ut i on t o account f or t he perspect i ve di st ort i ons can be empl oyed.
The present ed met hod has several advant ages over exi st i ng met hods. I t uses rel at i vel y
represent at i ons, t he represent at i ons are 2Drat her t han 3D, and l ocal i zat i on can be done
a si ngl e 2Dvi ew onl y. The same pri nci pal met hod i s appl i ed bot h f or t he l ocal i zat i on as
as t he posi t i oni ng probl ems, and a si mpl e al gori t hmf or
, t he t ask of ret urni
a previ ousl y vi si t ed posi t i on dened by a si ngl e vi ew, i s deri ved f romt hi s met hod.
Localizati
posi t i oni ng
repos i t i oni ng
c Massachuset t s I nst i t ut e of Technol ogy (1992)
Thi s report descri bes research done at t he Massachuset t s I nst i t ut e of Technol ogy wi t hi
Art i ci al I nt el l i gence Laborat ory and t he McDonnel l - Pew Cent er f or Cogni t i ve Neurosci e
Support f or t he l aborat ory's art i ci al i nt el l i gence research i s provi ded i n part by t he Ad
Research Proj ect s Agency of t he Depart ment of Def ense under Oce of Naval Research cont ract N00014- 91- J- 4038. Ronen Basri i s support ed by t he McDonnel l - Pew and t he Rot hchi l d
post doct oral f el l owshi ps. Ehud Ri vl i n i s at t he Uni versi ty of Maryl and, Col l ege Park, MD
1 Introduction
Basi c t asks i n aut onomous robot navi gat i on are l ocal i zat i on and posi t i oni ng.
act of recogni zi ng t he envi ronment , t hat i s, assi gni ng consi st ent l abel s t o di erent l ocat
i s t he act of comput i ng t he coordi nat es of t he robot i n t he envi ronment . Posi t i
i s a t ask compl ement ary t o l ocal i zat i on, i n t he sense t hat posi t i on ( e. g. , \1. 5 met ers no
of t abl e T ") i s of t en speci ed i n a pl ace- speci c coordi nat e syst em( \i n room911") . I
paper we suggest a met hod of bot h l ocal i zat i on and posi t i oni ng usi ng vi si on al one. Avar
of t he posi t i oni ng probl em, ref erred t o as
, i nvol vi ng t he ret urn t o a prev
vi si t ed pl ace i s al so di scussed.
Previ ous st udi es have exami ned t he probl ems of l ocal i zat i on and posi t i oni ng under a var
of condi t i ons, dened by t he ki nd of sensor( s) empl oyed, t he nat ure of t he envi ronment ,
t he represent at i ons used. We can di st i ngui sh bet ween act i ve and passi ve sensi ng, i ndoor
out door navi gat i on t asks, and met ri c and t opol ogi cal represent at i ons. The met ri c appr
at t empt s t o ut i l i ze a det ai l ed geomet ri c descri pt i on of t he envi ronment , whi l e t he t opo
approach uses a more qual i t at i ve descri pt i on i ncl udi ng a graph wi t h nodes represent i ng p
and arcs represent i ng sequences of act i ons t hat woul d resul t i n movi ng t he robot f romone n
t o anot her.
I n t he paper we consi der a robot t hat uses a passi ve sensor, vi si on, i n an i ndoor envi ron
The envi ronment cannot be changed by t he robot t o i mprove i t s perf ormance; nei t her beacon
nor oor or wal l marki ngs are empl oyed. The paper addresses bot h t he l ocal i zat i on and t
posi t i oni ng probl ems. Sol ut i ons t o t hese probl ems are present ed based on obj ect recogn
t echni ques. The met hod, based on t he l i near combi nat i ons scheme of [17], represent s sce
by set s of t hei r 2D i mages. Local i zat i on i s achi eved by compari ng t he observed i mage
l i near combi nat i ons of model vi ews, and t he posi t i on of t he robot i s comput ed by anal yz
t he coeci ent s of t he l i near combi nat i on t hat al i gns t he model t o t he i mage. Al so, a si m
\qual i t at i ve" sol ut i on t o t he reposi t i oni ng probl em usi ng t he l i near combi nat i ons sch
present ed.
The rest of t he paper i s organi zed as f ol l ows. The next sect i on descri bes t he l ocal i zat
posi t i oni ng probl ems and surveys previ ous sol ut i ons. The met hod of l ocal i zat i on and posi t
usi ng l i near combi nat i ons of model vi ews i s descri bed i n Sect i on 3. The met hod assumes we
perspect i ve proj ect i on. An i t erat i ve scheme t o account f or perspect i ve di st ort i ons i s p
i n Sect i on 4. An anal ysi s of t he error resul t i ng f romt he proj ect i on assumpt i on i s prese
Sect i on 5. Const rai nt s i mposed on t he mot i on of t he robot as a resul t of speci al propert
i ndoor envi ronment s can be used t o reduce t he compl exi t y of t he met hod present ed here. Th
t opi c i s covered on Sect i on 6. Experi ment al resul t s f ol l ow.
Local i z at i on
pos i t i oni ng
repos i t i oni ng
1
2 The Problem
Local i zat i on and posi t i oni ng f romvi sual i nput are dened i n t he f ol l owi ng way: Gi ven a
mi l i ar envi ronment , i dent i f y t he observed envi ronment , and t hen nd your posi t i on i n t
envi ronment . Local i zat i on resembl es t he t ask of obj ect recogni t i on, wi t h obj ect s repl a
scenes. Once l ocal i zat i on i s accompl i shed, posi t i oni ng can be perf ormed.
One probl ema syst emf or l ocal i zat i on and posi t i oni ng shoul d address i s t he vari abi l
i mages due t o vi ewpoi nt changes. The i nexact ness of pract i cal syst ems makes i t di cul t f
robot t o ret urn t o a speci ed posi t i on on subsequent vi si t s. The vi sual dat a avai l abl e
robot between vi si t s vari es i n accordance wi t h t he vi ewi ng posi t i on of t he robot . Al ocal
syst emshoul d be abl e t o recogni ze scenes f romdi erent posi t i ons and ori ent at i ons.
Anot her probl emi s t hat of changes i n t he scene. At subsequent vi si t s t he same pl ace m
l ook di erent due t o changes i n t he arrangement of t he obj ect s, t he i nt roduct i on of newobj
and t he removal of ot hers. I n general , some obj ect s t end t o be more st at i c t han ot hers. W
chai rs and books are of t en moved, t abl es, cl oset s, and pi ct ures t end t o change t hei r po
much l ess, and wal l s are al most guarant eed t o be st at i c. St at i c cues nat ural l y are more re
t han mobi l e ones. Conni ng t he syst emt o st at i c cues, however, may i n some cases resul t
f ai l ure t o recogni ze t he scene due t o i nsuci ent cues. The syst emshoul d t heref ore at t em
rel y on st at i c cues, but shoul d not i gnore t he dynami c cues.
Sol ut i ons t o t he probl emof l ocal i zat i on f romvi sual dat a requi re a l arge memory and he
comput at i on. Exi st i ng syst ems of t en t ry t o reduce t hi s cost by usi ng sparse representa
and by expl oi t i ng cont ext ual i nf ormat i on. Sparse represent at i ons are i nt roduced i n [ 1
Mat ari c [ 10] represent s scenes as sequences of l andmarks ( such as wal l s, doors, et c. ) ex
by t raci ng t he boundari es of t he scene usi ng a sonar and a compass. Met ri c i nf ormat i on
and bet ween t he l andmarks i s not st ored. Sarachi k [ 14] recogni zes a roomby i t s di mensi
whi ch are measured by i dent i f yi ng and l ocat i ng t he t op corners of t he roomusi ng st ereo d
( obt ai ned f romf our cameras) . I n bot h cases t he represent at i on i s very sparse, and t he sc
t heref ore of t en ambi guous.
Ri cher represent at i ons are used i n [ 3, 5] where hi gher success rat es are report ed. Bra
[ 3] represent s t he scene by an occupancy t abl e, a 2D bi t array whi ch cont ai ns a 1 at ev
l ocat i on occupi ed by some obj ect . The t abl e i s const ruct ed by t aki ng st ereo pi ct ures cov
360 f romt he mi ddl e of t he roomand proj ect i ng t he obt ai ned 3D dat a ont o t he oor. The
met hod suers f roml oss of i nf ormat i on due t o t he proj ect i on ont o t he oor.
Engel son
[ 5] represent t he scene by a set of i nvari ant \si gnat ures". A si gnat
usual l y composed of l ow- resol ut i on gray- l evel or range dat a obt ai ned by bl urri ng an i mag
set of si gnat ures t aken f romdi erent vi ewpoi nt s are st ored. Ascene i s recogni zed i f t he
encount ers a si gnat ure si mi l ar t o one of t he st ored si gnat ures.
Syst ems t hat use t he f ul l i nf ormat i on provi ded by t he i mage ( e. g. , [ 6, 12] ) usual l
on cont ext ual i nf ormat i on t o avoi d scanni ng al l t he model s i n t he memory and t o reduce t
comput at i onal cost of compari ng a model t o t he i mage. The syst emf ol l ows a predet ermi ne
et al .
2
pat h, so t hat t he i dent i ty of each vi si t ed l ocat i on i s known i n advance, and l ocal i zat i on b
a veri cat i on probl em. Pat h cont i nui ty i n many cases i s essent i al , and t he so- cal l ed \dro
probl emi s not addressed. The emphasi s i n t hese syst ems i s on posi t i oni ng, whi ch i s use
keep t he robot on t he pat h. I t i s typi cal f or t hese syst ems ( e. g. , [ 1, 6, 12] ) t o use a
model of t he envi ronment .
Onoguchi
[ 12] , among ot hers, represent t he envi ronment by a set of l andmarks sel e
f rompai rs of st ereo i mages by a human operat or. These l andmarks are t ransf ormed by an i mag
processi ng program whi ch i s desi gned so as t o i dent i f y t he speci c l andmark usi ng spe
ext ract i on i nst ruct i ons ( such as what f eat ures t o l ook f or and at what l ocat i ons) . Local
i s achi eved by appl yi ng t he ext ract i on procedure speci ed f or t he next l andmark. Onc
l andmark i s i dent i ed, t he posi t i on of t he robot rel at i ve t o t hat l andmark i s det ermi n
compari ng t he di mensi ons of t he observed l andmark wi t h t hose of t he st ored model .
The met hod present ed i n t hi s paper represent s t he envi ronment usi ng a set of edge map
Local i zat i on and posi t i oni ng are achi eved by compari ng i mages of t he envi ronment t o l i n
combi nat i ons of t he model vi ews. The met hod uses ri ch vi sual i nf ormat i on t o represent
scene. The syst emi s exi bl e. I n many cases i t i s capabl e of recogni zi ng i t s l ocat i o
coverage i s not requi red) . When one i mage i s not suci ent , addi t i ona
one i mage onl y ( 360
i mages can be acqui red t o sol ve t he l ocal i zat i on probl em. Cont ext can be used t o det erm
t he order of compari son of t he model s t o t he observed i mage and t o i ncrease t he condence
a gi ven mat ch, but cont ext i s not essent i al : t he syst emcan al so, by perf ormi ng more ext en
comput at i ons, sol ve t he \drop- o" probl em.
et al .
3 The Method
The probl ems of l ocal i zat i on and obj ect recogni t i on are si mi l ar i n many ways. Bot h probl
requi re t he mat chi ng of vi sual i mages t o st ored model s, ei t her of t he envi ronment or of
observed obj ect s. Bot h probl ems f ace si mi l ar di cul t i es, such as varyi ng i l l umi nat i on con
and changes i n appearance due t o vi ewpoi nt changes. Si mi l ar met hodol ogi es t heref ore can
used f or sol vi ng bot h probl ems.
A part i cul ar appl i cat i on of an obj ect recogni t i on scheme, t he Li near Combi nat i ons (
scheme [ 17] , t o t he probl ems of l ocal i zat i on and posi t i oni ng i s di scussed bel ow. The envi r
i s represent ed i n t hi s scheme by a smal l set of vi ews obt ai ned f romdi erent vi ewpoi nt s an
t he correspondence bet ween t he vi ews. A novel vi ew i s recogni zed by compari ng i t t o l i n
combi nat i ons of t he st ored vi ews. Posi t i oni ng i s achi eved by recoveri ng t he posi t i on
camera rel at i ve t o i t s posi t i on i n t he model vi ews f romt he coeci ent s of t he al i gni ng
combi nat i on. I n t he rest of t hi s sect i on we revi ewt he l i near combi nat i ons approach and des
i t s appl i cat i on t o bot h l ocal i zat i on and posi t i oni ng. The sect i on concl udes wi t h a sol u
t he probl emof reposi t i oni ng, t hat i s, t he probl emof ret urni ng t o a previ ousl y vi si t ed p
by \l ocki ng" i nt o an i mage acqui red i n t hat posi t i on.
3
3.1
Localizati on
The probl emof l ocal i zat i on i s dened as f ol l ows: gi ven P , a 2Di mage of a pl ace, and M, a s
i. Local i zat i on i s t he recogni t i on
st ored model s, nd a model i M2 M such t hat P mat ches M
of a pl ace. I t can t heref ore pot ent i al l y benet f romusi ng an obj ect recogni t i on met hod
A common approach t o handl i ng t he probl emof recogni t i on f romdi erent vi ewpoi nt s i s b
compari ng t he st ored model s t o t he observed envi ronment af t er t he vi ewpoi nt i s recovered
compensat ed f or. Thi s approach, cal l ed
, i s used i n a number of st udi es of ob
recogni t i on [ 2, 7, 8, 9, 15, 16] . We appl y t he al i gnment approach t o t he probl emof l ocal i z
The syst emdescri bed bel owuses t he \Li near Combi nat i ons" ( LC) scheme, whi ch was suggest ed
by Ul l man and Basri [ 17] .
We begi n wi t h a bri ef revi ewof t he LCscheme. LCi s dened as f ol l ows. Gi ven an i mage, we
const ruct two vi ewvect ors f romt he f eat ure poi nt s i n t he i mage, one cont ai ns t he x- coordi
of t he poi nt s, and t he ot her cont ai ns t he y- coordi nat es of t he poi nt s. An obj ect ( i n our
t he envi ronment ) i s model ed by a set of such vi ews, where t he poi nt s i n t hese vi ews are orde
i n correspondence. The appearance of a novel vi ew of t he obj ect i s predi ct ed by appl y
l i near combi nat i ons t o t he st ored vi ews. The predi ct ed appearance i s t hen compared wi t h
act ual i mage, and t he obj ect i s recogni zed i f t he t wo mat ch. The advant age of t hi s meth
i s t wof ol d. Fi rst , vi ewer- cent ered represent at i ons are used rat her t han obj ect - cent er
namel y, model s are composed of 2Dvi ews of t he observed scene; second, novel appearances a
predi ct ed i n a si mpl e and accurat e way ( under weak perspect i ve proj ect i on) .
Formal l y, gi ven P , a 2Di mage ofk a scene, and M, a set of st ored model s, t he obj ect i ve i
nd a model Mi 2 M such t hat P = j j Mji f or some const ant sj 2 R. I t has been shown
t hat t hi s scheme accurat el y predi ct s t he appearance of ri gi d obj ect s under weak perspe
proj ect i on ( ort hographi c proj ect i on and scal e) . The l i mi t at i ons of t hi s proj ect i on mo
di scussed l at er i n t hi s paper.
More concret el y, l et
i =p( x i; yi; zi) , 10 0 i n, be a set of n obj ect poi nt s. Under weak
perspect i ve proj ect i on, t he posi0i =(
t i xoni; pyi) of t hese poi nt s i n t he i mage are gi ven by
x0i = sr xi + s r yi +s r zi +t x
yi0 = s r xi +s r yi +s r zi +t y
( 1)
where rij are t he component s of a 3 2 3 rot at i on mat ri x, and s i s a scal e f act or. Rewri t i ng t
i n vect or equat i on f ormwe obt ai n
x0 = s r x +s r y +s r z +t x1
y0 = s r x +s r y +s r z +t y 1
( 2)
where x; y; z; 0 ;x y0 2 Rn are t he vect ors ofi, xyi, zi, x0i and yi0 coordi nat es respect i vel y, and
1 =( 1; 1; : : : ; 1) . Consequent l y,
x0; y0 2 fx; y; z; 1g
( 3)
al i gnment
P
=1
11
12
13
21
22
23
11
12
13
21
22
23
s pan
4
n. (ofNotRi ce t hat
or, i n ot her words,0 and
x y0 bel ong t o a f our- di mensi onal l i near subspace
0
z , t he vect or of dept h coordi nat es of t he proj ect ed poi nt s, al so bel ongs t o t hi s subspac
f act i s used i n Sect i on 4 bel ow. ) A f our- di mensi onal space i s spanned by any f our l i ne
i ndependent vect ors of t he space. Two vi ews of t he scene suppl y f our such vect ors [ 13,
Denot e by x , y and x , y t he l ocat i on vect ors of t he n poi nt s i n t he t wo i mages; t hen t her
exi st coeci ent s; aa ; a ; a and b ; b ; b ; b such t hat
x0 = a x +a y +a x +a 1
y0 = b x +b y +b x +b 1
( 4)
( Not e t hat t he vect or aly ready depends on t he ot her f our vect ors. ) Si nce R i s a rot at i o
mat ri x, t he coeci ent s sat i sf y t he f ol l owi ng t wo quadrat i c const rai nt s:
a +a +a 0 b 0 b 0 b =2( b b 0 aa ) r +2( b b 0 aa ) r
a b +a b +a b +( a b +a b ) r +( a b +a b ) r =0
( 5)
To deri ve t hese const rai nt s t he t ransf ormat i on bet ween t he two model vi ews shoul d be recove
Thi s can be done under weak perspect i ve usi ng a t hi rd i mage. Al t ernat i vel y, t he const ra
can be i gnored, i n whi ch case t he syst emwoul d conf use ri gi d t ransf ormat i ons wi t h ane on
Thi s usual l y does not prevent successf ul l ocal i zat i on si nce general l y scenes are f ai rl y
f romone anot her.
A LC scheme f or t he probl emof l ocal i zat i on i s as f ol l ows: The envi ronment i s model
by a set of i mages wi t h correspondence between t he i mages. For exampl e, a spot can b
model ed by t wo of i t s correspondi ng vi ews. The correspondi ng quadrat i c const rai nt s may a
be st ored. Local i zat i on i s achi eved by recoveri ng t he l i near combi nat i on t hat al i gns t he
t o t he observed i mage. The coeci ent s are det ermi ned usi ng f our model poi nt s and t he
correspondi ng i mage poi nt s by sol vi ng a l i near set of equat i ons. Three poi nt s are suci e
det ermi ne t he coeci ent s i f t he quadrat i c const rai nt s are al so consi dered. Addi t i onal
may be used t o reduce t he eect of noi se.
The LC scheme uses vi ewer- cent ered model s, t hat i s, represent at i ons t hat are compos
of i mages. I t has a number of advant ages over met hods t hat bui l d f ul l t hree- di mensi o
model s t o represent t he scene. Fi rst , by usi ng vi ewer- cent ered model s t hat cover rel at i vel
t ransf ormat i ons we avoi d t he need t o handl e occl usi ons i n t he scene. I f f romsome vi ewpo
t he scene appears di erent because of occl usi ons we ut i l i ze a new model f or t hese vi ewpo
Second, vi ewer- cent ered model s are easi er t o bui l d and t o mai nt ai n t han obj ect - cent ered
The model s cont ai n onl y i mages and correspondences. By l i mi t i ng t he t ransf ormat i on betwe
t he model i mages one can nd t he correspondence usi ng mot i on met hods. I f l arge port i ons
t he envi ronment are changed bet ween vi si t s a newmodel can be const ruct ed by si mpl y repl aci
ol d i mages wi t h newones.
One probl emwi t h usi ng t he LCscheme f or l ocal i zat i on i s due t o t he weak perspect i ve a
proxi mat i on. I n cont rast wi t h t he probl emof obj ect recogni t i on, where we can general l y as
1
1
2
1
2
2
3
4
1
2
3
4
1
1
1
1
2
2
1
3
1
3
2
2
4
4
2
2
1
2
2
2
3
1 1
2
1
2 2
2
2
2
3
3 3
1 3
1 3
1 3
3 1
5
11
11
2 3
2 3
2 3
3 2
12
12
t hat obj ect s are smal l rel at i ve t o t hei r di st ance f romt he camera, i n l ocal i zat i on t he
ment surrounds t he robot and perspect i ve di st ort i ons cannot be negl ect ed. The l i mi t at
of weak perspect i ve model i ng are di scussed bot h mat hemat i cal l y and empi ri cal l y i n t he n
two sect i ons. I t i s shown t hat i n many pract i cal cases weak perspect i ve i s suci ent t o e
accurat e l ocal i zat i on. The mai n reason i s t hat t he probl emof l ocal i zat i on does not r
accurat e measurement s i n t he ent i re i mage; i t onl y requi res i dent i f yi ng a suci ent numbe
spot s t o guarant ee accurat e nami ng. I f t hese spot s are rel at i vel y cl ose t o t he cent er
i mage, or i f t he dept h di erences t hey creat e are rel at i vel y smal l ( as i n t he case of l oo
a wal l when t he l i ne of si ght i s nearl y perpendi cul ar t o t he wal l ) , t he perspect i ve di st
are rel at i vel y smal l , and t he syst emcan i dent i f y t he scene wi t h hi gh accuracy. Al so,
rel at ed by a t ransl at i on paral l el t o t he i mage pl ane f orma l i near space even when perspe
di st ort i ons are l arge. Thi s case and ot her si mpl i cat i ons are di scussed i n Sect i on 6.
By usi ng weak perspect i ve we avoi d st abi l i ty probl ems t hat f requent l y occur i n perspec
comput at i ons. We can t heref ore comput e t he al i gnment coeci ent s by l ooki ng at a rel at i v
narrowel d of vi ew. The ent i re scheme can be vi ewed as an accumul at i ve process. Rat her t h
acqui ri ng i mages of t he ent i re scene and compari ng t hemal l t o a f ul l scene model ( as i n
we recogni ze t he scene i mage by i mage, spot by spot , unt i l we accumul at e suci ent convi nci
i nf ormat i on t hat i ndi cat es t he i dent i ty of t he pl ace.
When perspect i ve di st ort i ons are rel at i vel y l arge and weak perspect i ve i s i nsuci e
model t he envi ronment , t wo approaches can be used. One possi bi l i t y i s t o const ruct a l a
number of model s so as t o keep t he possi bl e changes bet ween t he f ami l i ar and t he novel vi e
smal l . Al t ernat i vel y, an i t erat i ve comput at i on can be appl i ed t o compensat e f or t hese d
t i ons. Such an i t erat i ve met hod i s descri bed i n Sect i on 4.
3. 2
Posi t i oni ng
Posi t i oni ng i s t he probl emof recoveri ng t he exact posi t i on of t he robot . Thi s posi t i on
speci ed i n a xed coordi nat e syst emassoci at ed wi t h t he envi ronment ( i . e. , roomcoordi na
or i t can be associ at ed wi t h some model , i n whi ch case l ocat i on i s expressed wi t h respect t
posi t i on f romwhi ch t he model vi ews were acqui red. I n t hi s sect i on we di scuss an appl i ca
of t he LCscheme t o t he posi t i oni ng probl em.
The i dea i s t he f ol l owi ng. We assume a model composed of two i mages,
and PP ; t hei r
rel at i ve posi t i on i s gi ven. Gi ven a novel 0;i mage
we rst
P al i gn t he model wi t h t he i mage
( i . e. , l ocal i zat i on) . By consi deri ng t he coeci ent s of t he l i near combi nat i on t he robot '
rel at i ve t o t he model i mages i s recovered. To recover t he absol ut e posi t i on of t he robot
roomt he absol ut e posi t i ons of t he model vi ews shoul d al so be provi ded.
Assumi ng P i s obt ai ned f romPby a rot at i on R, t ransl at i on xt; =(
yt ) ,t and scal i ng s , t he
0
0
0
coordi nat es of a poi nt i; n(Px; y) , can be wri t t en as l i near combi nat i ons of t he correspondi ng
model poi nt s i n t he f ol l owi ng way:
x0 = a x +a y +a x +a
1
2
1
1 1
2 1
6
3
2
4
2
=
y0
b 1 x1 +b 2y1 +b 3 x2 +b 4
( 6)
Subst i t ut i ng f or wex obt ai n
x0 = a x +a y +a ( s r x +s r y +s r z +t x ) +a
y0 = b x +b y +b ( s r x +s r y +s r z +t x ) +b
( 7)
and rearrangi ng t hese equat i ons we obt ai n
x0 = ( a +a s r ) x +( a +a s r ) y +( a s r ) z +( a tx +a )
y0 = ( b +b s r ) x +( b +b s r ) y +( b s r ) z +( b tx +a )
( 8)
Usi ng t hese equat i ons we can deri ve al l t he paramet ers of t he t ransf ormat i on between t he mo
and t he i mage. Assume t he i mage i s obt ai ned by a rot at i on U , t ransl
at i on
scalt i ngn .s
n , and
Usi ng t he ort honormal i t y const rai nt we can rst deri ve t he scal e f act or
sn = ( a +a s r ) +( a +a s r ) +( a s r )
= a +a +a s +2 a s ( a r +a r )
( 9)
FromEquat i ons ( 8) and ( 9) , by deri vi ng t he component s of t he t ransl at i non, we
vectcan
or, t
obt ai n t he posi t i on of t he robot i n t he i mage rel at i ve t o i t s posi t i on i n t he model vi ews
1x = a tx +a
1y = b ty +b
( 10)
1
1
1z = f ( s 0 s )
2
1
1
2 1
1 1
1
2 1
3
1
3
11
3
11
2
3
1
11
2
1
3
2
2
1
13 1
12 1
12
3
11
12 1
1
3
2
1
2
1
11
12
2
2
2 2
3
3
3
13 1
1
3
1
3
3
4
12
1 11
13
13
4
1
3
1
2
3
4
3
13
4
2
2 12
4
3
4
n
Not e t hat 1z i s deri ved f romt he change i n scal e of t he obj ect . The rot at i on mat ri x U betw
P and P 0 i s gi ven by
a +a s r
a +a s r
a sr
u =
u =
u =
sn
sn
sn
( 11)
b +a s r
b +a s r
b sr
u =
u =
u =
s
s
s
1
11
21
1
3
11
1
3
21
n
12
22
2
3
12
2
3
22
n
13
23
3
13
3
23
n
As was al ready ment i oned, t he posi t i on of t he robot i s comput ed here rel at i ve t o t he posi t i
t he camera when t he rst model i mage,, Pwas acqui red. 1x and 1z represent t he mot i on of
t he robot f romPt o P0; and t he rest of t he paramet ers represent i t s 3Drot at i on and el evat i
To obt ai n t he rel at i ve posi t i on t he t ransf ormat i on paramet ers between t he model
andvi ews, P
P , are requi red.
1
1
1
2
3. 3
Repos i t i oni ng
An i nt erest i ng vari ant of t he posi t i oni ng probl em, ref erred t o as
, i s de
f ol l ows. Gi ven an i mage, cal l ed t he
i mage, posi t i on yoursel f i n t he l ocat i on f rom
r epos i t i oni ng
t ar get
7
t hi s i mage was observed.One way t o sol ve t hi s probl emi s t o ext ract t he exact posi t i on f rom
whi ch t he t arget i mage was obt ai ned and di rect t he robot t o t hat posi t i on. I n t hi s sect i o
are i nt erest ed i n a more qual i t at i ve approach. Under t hi s approach posi t i on i s not compu
I nst ead, t he robot observes t he envi ronment and ext ract s onl y t he di rect i on t o t he t a
l ocat i on. Unl i ke t he exact approach, t he met hod present ed here does not requi re t he reco
of t he t ransf ormat i on bet ween t he model vi ews.
We assume we are gi ven wi t h a model of t he envi ronment t oget her wi t h a t arget i mage.
The robot i s al l owed t o t ake new i mages as i t i s movi ng t owards t he t arget . We assume a
hori zont al l y movi ng pl at f orm. ( I n ot her words, we assume t hree degrees of f reedomrat her t
si x; t he robot i s al l owed t o rot at e around t he vert i cal axi s and t ransl at e hori zont al l
val i di ty of t hi s const rai nt i s di scussed i n Sect i on 6. ) Bel owwe gi ve a si mpl e comput at i o
det ermi nes a pat h whi ch t ermi nat es i n t he t arget l ocat i on. At each t i me st ep t he robot acqu
a new i mage and al i gns i t wi t h t he model . By compari ng t he al i gnment coeci ent s wi t h t he
coeci ent s f or t he t arget i mage t he robot det ermi nes i t s next st ep. The al gori t hmi s di
i nt o t wo st ages. I n t he rst st age t he robot xat es on one i dent i abl e poi nt and moves al
a ci rcul ar pat h around t he xat i on poi nt unt i l t he l i ne of si ght t o t hi s poi nt coi nci de
t he l i ne of si ght t o t he correspondi ng poi nt i n t he t arget i mage. I n t he second st age t he
advances f orward or ret reat s backward unt i l i t reaches t he t arget l ocat i on.
Gi ven a model composed of t wo i mages, and
P P , P i s obt ai ned f rom Pby a rot at i on
about t he Y - axi s by an angl e , hori zont al t ransl
i onscal
t e f act or s . Gi ven a t arget
x, atand
i mage Pt, Pt i s obt ai ned f rom Pby a si mi l ar rot at i on by an angl e , t ranslt, atand
i onscal
t e
st . Usi ng Eq. ( 4) t he posi t i on of a t arget poi
( x be expressed as
t ; yt )ntcan
xt = a x +a x +a
yt = b y
( 12)
( The rest of t he coeci ent s are zero si nce t he pl at f ormmoves hori zont al l y. ) I n f act , t he
ci ent s are gi ven by
s si n( 0 )
a = t
si n st si n a =
( 13)
s si n t s si n a = tt0 x t
s si n b = st
( The deri vat i on i s gi ven i n t he Appendi x. )
At every t i me st ep t he robot acqui res an i mage and al i gns i t wi t h t he above model . Assum
t hat i mage Pp i s obt ai ned as a resul t of a rot at i on by an angl e , t ransl
i on tep.s
p , andat scal
1
1
2
2
1
1
1 1
3 2
4
2 1
1
3
4
2
1
This problem can be c ons i de r e d as a var i ant of t he homi ng pr obl e m. A di s c us s i on of t he ge ne r al homi ng
pr obl e mwi t h a \s i gnat ur e - bas e d" s ol ut i on c an be f ound i n[11].
8
The posi t i on of a poi ntp; (ypx) i s expressed by
xp = c x +c x +c
yp = d y
( 14)
where t he coeci ent s are gi ven by
s si n( 0 )
c = p
si n sp si n ( 15)
c =
s si n t s si n c = tp0 x p
s si n d = sp
The st ep perf ormed by t he robot i s det ermi ned by
c a
= 0
( 16)
c a
That i s,
s si n( 0 ) s si n( 0 )
( 17)
=
si n 0 si n =s si n ( cot 0 cot )
The robot shoul d now move so as t o reduce t he absol ut e val ue of . The di rect i on of mot i
depends on t he si gn of . The robot can deduce t he di rect i on by movi ng sl i ght l y t o t he s
and checki ng i f t hi s mot i on resul t s i n an i ncrease or decrease of . The mot i on i s den
f ol l ows. The robot moves t o t he ri ght ( or t o t he l ef t , dependi ng on whi ch di rect i on reduce
by a st ep 1x .
A new i mage Pn i s now acqui red, and t he xat ed poi nt i s l ocat ed i n t hi s i mage. Denot
i t s newposi t i on byn . xSi nce t he mot i on i s paral l el t o t he i mage pl ane t he dept h val ues of t
poi nt i n t he two vi ews,p and
P Pn, are i dent i cal . We now want t o rot at e t he camera so as t o
ret urn t he xat ed poi nt t o i t s ori gi nal posi t i on. The angl e of rot at i on, , can be deduce
t he equat i on
xp = x n cos +si n ( 18)
Thi s equat i on has two sol ut i ons. We chose t he one t hat count ers t he t ransl at i on ( namel y
t ransl at i on i s t o t he ri ght , t he camera shoul d rot at e t o t he l ef t ) , and t hat keeps t he a
rot at i on smal l . I n t he next t i me st ep t he new pinctrepl
ure aces
P pPand t he procedure i s
repeat ed unt i l vani shes. The resul t i ng pat h i s ci rcul ar around t he poi nt of f ocus.
Once t he robot arri ves at a posi t i on f or whi ch = 0 ( namel y, i t s l i ne of si ght coi n
wi t h t hat of t he t arget i mage, and = ) i t shoul d nowadvance f orward or ret reat backwar
t o adj ust i t s posi t i on al ong t he l i ne of si ght . Several measures can be used t o det ermi
di rect i on of mot i on; one exampl e i s t he t=aermwhic ch sat i ses
sp
c
=
( 19)
a
st
when t he t wo l i nes of si ght coi nci de. The obj ect i ve at t hi s st age i s t o bri ng t hi s measure
1
1
3
2 1
1
3
4
2
1
1
3
3
1
1
1
9
1
2
4
4 Handl i ng Perspecti ve Di storti ons
The l i near combi nat i on scheme present ed above accurat el y handl es changes i n vi ewpoi nt assu
i ng t he i mages are obt ai ned under weak perspect i ve proj ect i on. Error anal ysi s and experi m
resul t s demonst rat e t hat i n many pract i cal cases t hi s assumpt i on i s val i d. I n cases wher
spect i ve di st ort i ons are t oo l arge t o be handl ed by a weak perspect i ve approxi mat i on, mat c
bet ween t he model and t he i mage can be f aci l i t at ed i n t wo ways. One possi bi l i t y i s t o av
cases of l arge perspect i ve di st ort i on by augment i ng t he l i brary of st ored model s wi t h addi
model s. I n a rel at i vel y dense l i brary t here usual l y exi st s a model t hat i s rel at ed t o t h
by a suci ent l y smal l t ransf ormat i on avoi di ng such di st ort i ons. The second al t ernat i ve
i mprove t he mat ch bet ween t he model and t he i mage usi ng an i t erat i ve process. I n t hi s sect
we consi der t he second opt i on.
The suggest ed i t erat i ve process i s based on a Tayl or expansi on of t he perspect i ve co
nat es. As descri bed bel ow, t hi s expansi on resul t s i n a pol ynomi al consi st i ng of t erms
of whi ch can be approxi mat ed by l i near combi nat i ons of vi ews. The rst t ermof t hi s ser
represent s t he ort hographi c approxi mat i on. The process resembl es a met hod of mat chi ng
poi nt s wi t h 2Dpoi nt s descri bed recent l y by DeMent hon and Davi s [ 4] . I n t hi s case, howev
t he met hod i s appl i ed t o 2Dmodel s rat her t han 3Dones. I n our appl i cat i on t he 3Dcoordi nat
of t he model poi nt s are not provi ded; i nst ead t hey are approxi mat ed f romt he model vi ews.
An i mage poi nt ( x; y ) =( f X=Z; f Y =Z) i s t he proj ect i on of some obj ect poi nt , ( X; Y;
t he i mage, where f denot es t he f ocal l engt h. Consi der t he f ol l owi ng Tayl or expansi on of
around some dept h val ue Z:
0
1 =
Z
1 f (k) ( Z )
0
X
k=0
k!
( Z 0 Z)k
0
( 0 1)k ( Z 0 Z)k
( 20)
k
k ( k 0 1) ! Z
1
k
k
= Z1 1 + ((k001)1) ! Z 0Z Z
k
The Tayl or seri es descri bi ng t he posi t i on of a poi nt x i s t heref ore gi ven by
1 ( 0 1)
k Z0 Z k
fX fX
x=
=
1
+
( 21)
Z
Z
k ( k 0 1) ! Z
k t hee by 1
Not i ce t hat t he zero t ermcont ai ns t he ort hographi c approxi mat i on f or x . Denot
k t h t ermof t he seri es:
k
k
( 22)
1 k = fZX ((k001)1) ! Z 0Z Z
Arecursi ve deni t i on of t he above seri es i s gi ven bel ow.
= Z1 +
0
"
1
X
0
+1
0
=1
X
0
#
0
0
=1
"
0
#
X
0
0
=1
( )
0
( )
0
0
10
Initialization:
x( 0)
=1
( 0)
fX
Z0
=
Iterative step:
Z k0
1 k = 0 ( Zk 00 1)
1
Z
x k = x k0 +1 k
where x k represent s t he k t h order approxi mat i on f or x ,k and
represent
1 s t he hi ghest order
k
t ermi n x .
Accordi ng t o t he ort hographi c approxi mat i on bot h X and Z can be expressed as l i near com
bi nat i ons of t he model vi ews ( Eq. ( 4) ) . We t heref ore appl y t he above procedure, approxi mat
X and Z at every st ep usi ng t he l i near combi nat i on t hat best al i gns t he model poi nt s wi t h
i mage poi nt s. The general i dea i s t heref ore t he f ol l owi ng. Fi rst , weand
est 1i matby
ex
sol vi ng t he ort hographi c case. Then at each st ep of t he i t erat i on we i mprove t he est i mat
seeki ng t he l i near combi nat i on t hat best est i mat es t he f act or
0
( )
(
1)
0
( )
(
1)
( )
( )
( )
( )
( 0)
( 0)
Z
x 0 xk0
0 ( Zk 00 1)
Z 1 k0
(
0
(
0
1)
( 23)
1)
Denot e by x 2 Rn t he vect or of i mage poi nt coordi nat es, and denot e by
P =[ x ; y ; x ; 1]
( 24)
an n 2 4 mat ri x cont ai ni ng t he posi t i on of t he poi nt s i n t he t wo model i mages. Denot e
P = ( P T P )0 P T t he pseudo- i nverse of P ( we assume P i s overdet ermi ned) . Al so denote
by a k t he coeci ent s comput ed f or t he k t h st ep.k represent
Pa
s t he l i near combi nat i on
comput ed at t hat st ep t o approxi mat e t he X or t he Z val ues. Si nce at every
, fst, ep
andZ
k and
k are const ant t hey can be merged i nt o t he l i near combi nat i on. Denot
e by1xk t he
vect ors of comput ed val ues of x and 1at t he k t h st ep. An i t erat i ve procedure t o al i gn a mo
t o t he i mage i s descri bed bel ow.
1
+
1
2
1
( )
( )
( )
Initialization:
Sol ve t he ort hographi c approxi mat i on, namel y
a =P x
x = 1 = Pa
( 0)
( 0)
Iterative step:
+
( 0)
( 0)
q k = ( x 0 x k0 ) 4 1 k0
ak = P qk
1 k = ( P a k ) 1 k0
x k = x k0 +1 k
( )
( )
(
+
( )
( )
(
( )
( )
(
1)
1)
11
(
( )
1)
1)
0
( )
where t he vect or operat i ons and 4 are dened as
u v = ( u v ; : : : n;vnu)
u4 v = ( uv ; : : uv: n;)
1 1
1
n
1
5 Projecti on M
odel { Error Anal ysi s
I n t hi s sect i on we est i mat e t he error obt ai ned by usi ng t he l i near combi nat i on met hod.
met hod assumes a weak perspect i ve proj ect i on model . We compare t hi s assumpt i on wi t h t he
more accurat e perspect i ve proj ect i on model .
Apoi nt ( X; Y; Z) i s proj ect ed under t he perspect i ve model t o ( x; y ) =( f X=Z; f Y =Z) i n
i mage, where f denot es t he f ocal l engt h. Under our weak perspect i ve model t he same poi
i s approxi mat ed by ( x^ ; ^y ) =( s X; s Y ) where s i s a scal i ng f act or. The best est i mat e f or
scal i ng f act or, i s gi ven by s =, fwhere
=Z Z i s t he average dept h of t he observed envi ronment .
Denot e t he error by
E = j ^x 0 xj
( 25)
The error i s expressed by
1 1
E = f X( 0 )
( 26)
Z Z
Changi ng t o i mage coordi nat es
1 1
( 27)
E = xZ ( 0 )
Z Z
or
Z
E = jxj
0 1
( 28)
Z
The error i s smal l when t he measured f eat ure i s cl ose t he opt i cal axi s, or when our est i
f or t he dept h, ,Z i s cl ose t o t he real dept h, Z. Thi s support s t he basi c i nt ui t i on t hat
i mages wi t h l owdept h vari ance and f or xat ed regi ons ( regi ons near t he cent er of t he i ma
t he obt ai ned perspect i ve di st ort i ons are rel at i vel y smal l , and t he syst emcan t heref ore
t he scene wi t h hi gh accuracy. Fi gures 1 and 2 showt he dept h ratasi oaZ=Z
f unct i on of x f or
=10 and 20 pi xel s, and Tabl e 5 shows a number of exampl es f or t hi s f unct i on. The al l owe
dept h vari ance, Z=Z
, i s comput ed as a f unct i on of x and t he t ol erat ed error, . For exampl
a 10 pi xel error t ol erat ed i n a el d of vi ew of up t o 650 pi xel s i s equi val ent t o al l owi ng
vari at i ons of 20%. Fromt hi s di scussi on i t i s apparent t hat when a model i s al i gned t o t he i
t he resul t s of t hi s al i gnment shoul d be j udged di erent l y at di erent poi nt s of t he i mage
f art her away a poi nt i s f romt he cent er t he more di screpancy shoul d be t ol erat ed between
predi ct i on and t he act ual i mage. Ave pi xel error at posi t i on x =50 i s equi val ent t o a 10
error at posi t i on x =100.
So f ar we have consi dered t he di screpanci es bet ween t he weak perspect i ve and t he persp
t i ve proj ect i ons of poi nt s. The accuracy of t he LCscheme depends on t he val i di ty of t he w
0
0
0
0
0
0
0
0
12
4.5
10/x + 1
4
3.5
3
2.5
2
1.5
1
0
50
100
150
200
250
300
Fi gure 1:ZZ0 as a f unct i on of x f or =10 pi xel s.
8
20/x + 1
7
6
5
4
3
2
1
0
50
100
150
200
250
300
Fi gure 2:ZZ0 as a f unct i on of x f or =20 pi xel s.
13
5
1. 2
1. 1
1. 07
1. 05
xn
25
50
75
100
10 15 20
1. 4 1. 6 1. 8
1. 2 1. 3 1. 4
1. 13 1. 2 1. 27
1. 1 1. 15 1. 2
, as a f unct i on of x ( hal f t he wi dt h of t he el d consi dered)
Tabl e 1: Al l owed dept h rat iZZos,
and t he error al l owed ( , i n 0pi xel s) .
perspect i ve proj ect i on bot h i n t he model vi ews and f or t he i ncomi ng i mage. I n t he rest of
sect i on we devel op an error t ermf or t he LC scheme assumi ng t hat bot h t he model vi ews and
t he i ncomi ng i mage are obt ai ned by perspect i ve proj ect i on.
The error obt ai ned by usi ng t he LCscheme i s gi ven by
E = jx 0 ax 0 b y0 c x0 dj
( 29)
Si nce t he scheme accurat el y predi ct s t he appearances of poi nt s under weak perspect i ve pr
t i on, i t sat i ses
^x = a ^x 0 b ^y0 c ^x0 d
( 30)
where accent ed l et t ers represent ort hographi c approxi mat i ons. Assume t hat i n t he t wo mo
pi ct ures t he dept h rat i os are roughl y equal :
1
1
Z0M
ZM
1
1
2
2
= ZZ ZZ
01
02
1
2
( 31)
( Thi s condi t i on i s sat i sed, f or exampl e, when bet ween t he two model i mages t he camera on
t ransl at es al ong t he i mage pl ane. ) Usi ng t he f act t hat
x=
we obt ai n
E
f X f X Z0
=Z Z
Z
0
=^x ZZ
( 32)
0
= jx 0 ax0 b y0 c x0 dj
M
M
M
^xZZ 0 a ^xZZ M 0 b ^yZZ M 0 c ^xZZ M 0
=
=
1
0
1
1
2
0
1
0
M
^xZZ0 0 ( a1^x0 b 1^y0 c 2^x) ZZ0M
Z0M
Z0
^x Z 0 ( ^x 0 Zd M) 0 d
14
2
0
d
0 d
( 33)
=
M
M
^x (ZZ0 0 ZZ0M ) 0 dZZ( 0M 0 1)
M
Z0M Z0
Z0
j ^xj Z 0 Z M + jdj Z M 0 1
The error t heref ore depends on t wo t erms. The rst get s smal l er as t he i mage poi nt s get cl
t o t he cent er of t he f rame and as t he di erence bet ween t he dept h rat i os of t he model and t
i mage get s smal l er. The second get s smal l er as t he t ransl at i on component get s smal l er an
t he model get s cl ose t o ort hographi c.
Fol l owi ng t hi s anal ysi s, weak perspct i ve can be used as a proj ect i on model when t he de
vari at i ons i n t he scene are rel at i vel y l owand when t he syst emconcent rat es on t he cent er
of t he i mage. We concl ude t hat , by xat i ng on di st i ngui shed part s of t he envi ronment ,
l i near combi nat i ons scheme can be used f or l ocal i zat i on and posi t i oni ng.
6 Imposi ng Constrai nts
Local i zat i on and posi t i oni ng requi re a l arge memory and a great deal of on- l i ne comput at
A l arge number of model s must be st ored t o enabl e t he robot t o navi gat e and mani pul at e
i n rel at i vel y l arge and compl i cat ed envi ronment s. The comput at i onal cost of model - i m
compari son i s hi gh, and i f cont ext ( such as pat h hi st ory) i s not avai l abl e t he number of req
compari sons may get very l arge. To reduce t hi s comput at i onal cost a number of const rai nt s m
be empl oyed. These const rai nt s t ake advant age of t he st ruct ure of t he robot , t he propert i
i ndoor envi ronment s, and t he nat ural propert i es of t he navi gat i on t ask. Thi s sect i on exa
some of t hese const rai nt s.
One t hi ng a syst emmay at t empt t o do i s t o bui l d t he set of model s so as t o reduce t he
eect of perspect i ve di st ort i ons i n order t o avoi d perf ormi ng i t erat i ve comput at i ons.
of t he envi ronment obt ai ned when t he syst em l ooks rel at i vel y deep i nt o t he scene usua
sat i sf y t hi s condi t i on. When perspect i ve di st ort i ons are l arge t he syst emmay consi der mo
subset s of vi ews rel at ed by a t ransl at i on paral l el t o t he i mage pl ane ( perpendi cul ar t o t
of si ght ) . I n t hi s case t he dept h val ues of t he poi nt s are roughl y equal across al l
consi dered, and i t can be shown t hat novel vi ews can be expressed by l i near combi nat i ons
two model vi ews even i n t he presence of l arge perspect i ve di st ort i ons. Thi s becomes appa
f romt he f ol l owi ng deri vat i on. Let
i; Yi;( ZXi) ; 1 i n be a poi nt proj ect ed i n t he i mage
t o ( xi; yi) = ( f Xi =Zi; f Yi=Zi) , and l et 0i(; xyi0 ) be t he proj ect ed poi nt af t er appl yi ng a ri gi d
t ransf ormat i on. Assumi ng t hat
i0 = Z i we obt ai n
Zi x0i = r Xi +r Yi +r Zi +t x
Zi yi0 = r Xi +r Yi +r Zi +t y
( 34)
Di vi di ng by iZwe obt ai n
1
x0i = r xi +r yi +r +t x
Z
11
12
13
21
22
23
11
12
15
13
i
( 35)
= r xi +r yi +r +t y Z1
i
Rewri t i ng t hi s i n vect or equat i on f ormgi ves
x0 = r x +r y +r 1 +t xz0
y0 = r x +r y +r 1 +t y z0
( 36)
where x, y, x0 , and y0 are t he vect ors ofi , xyi, x0i, and yi0 val ues respect i vel y, 1 i s a vect or
of al l 1s, and0 zi s a vect or of 1i =Z
val ues. Consequent l y, as i n t he weak perspect i ve case,
novel vi ews obt ai ned by a t ransl at i on paral l el t o t he i mage pl ane can be expressed by l i
combi nat i ons of f our vect ors.
An i ndoor envi ronment usual l y provi des t he robot wi t h a at , hori zont al support . Con
quent l y, t he mot i on of t he camera i s of t en const rai ned t o rot at i on about t he vert i cal ( Y
and t o t ransl at i on i n t he XZ- pl ane. Such mot i on has onl y t hree degrees of f reedomi nst ea
t he si x degrees of f reedomi n t he general case. Under t hi s const rai nt f ewer correspondenc
requi red t o al i gn t he model wi t h t he i mage. For exampl e, i n Eq. ( 4) ( above) t he coeci e
a = b = b = b =0. Three poi nt s rat her t han f our are requi red t o det ermi ne t he coeci ent
by sol vi ng a l i near syst em. Two, rat her t han t hree, are requi red i f t he quadrat i c const rai
al so consi dered. Anot her advant age t o consi deri ng onl y hori zont al mot i on i s t he f act t ha
mot i on const rai ns t he possi bl e epi pol ar l i nes between i mages. Thi s f act can be used t o
t he t ask of correspondence seeki ng.
Obj ect s i n i ndoor envi ronment s somet i mes appear i n roughl y pl anar set t i ngs. I n part i c
t he rel at i vel y st at i c obj ect s t end t o be l ocat ed al ong wal l s. Such obj ect s i ncl ude wi
shel ves, pi ct ures, cl oset s and t abl es. When t he assumpt i on of ort hographi c proj ect i on i
( f or exampl e, when t he robot i s rel at i vel y di st ant f romt he wal l , or when t he l i ne of si
roughl y perpendi cul ar t o t he wal l ) t he t ransf ormat i on between any t wo vi ews can be descri
by a 2Dane t ransf ormat i on. The di mensi on of t he space of vi ews of t he scene i s t hen reduc
t o t hree ( rat her t han f our) , and Eq. ( 4) becomes
x0 = a x +a y +a 1
y0 = b x +b y +b 1
( 37)
( a =b =0. ) Onl y one vi ewi s t heref ore suci ent t o model t he scene.
Most oce- l i ke i ndoor envi ronment s are composed of rooms connect ed by corri dors. Navi
gat i ng i n such an envi ronment i nvol ves maneuveri ng t hrough t he corri dors, ent eri ng and exi
t he rooms. Not al l poi nt s i n such an envi ronment are equal l y i mport ant . Junct i ons, pl aces w
t he robot f aces a number of opt i ons f or changi ng i t s di rect i on, are more i mport ant t han o
pl aces f or navi gat i on. I n an i ndoor envi ronment t hese pl aces i ncl ude t he t hreshol ds of
and t he begi nni ngs and ends of corri dors. A navi gat i on syst emwoul d t heref ore t end t o st
more model s f or t hese poi nt s t han f or ot hers.
One i mport ant propert y shared by many j unct i ons i s t hat t hey are conned t o rel at i ve
smal l areas. Consi der f or exampl e t he t hreshol d of a room. I t i s a rel at i vel y narrow
yi0
21
22
23
11
12
13
21
22
23
1
1
1
2
3
1
3
4
1
1
1
1
3
16
2
2
1
1
4
4
t hat separat es t he roomf romt he adj acent corri dor. When a robot i s about t o ent er a roo
a common behavi or i ncl udes st eppi ng t hrough t he door, l ooki ng i nt o t he room, and i dent i f y
i t bef ore a deci si on i s made t o ent er t he roomor t o avoi d i t . The set of i nt erest i ng i mag
t hi s t ask i ncl udes t he set of vi ews of t he roomf romi t s ent rance. Provi ded t hat t hreshol
narrowt hese vi ews are rel at ed t o each ot her al most excl usi vel y by rot at i on around t he ver
axi s. Under perspect i ve proj ect i on, such a rot at i on i s rel at i vel y easy t o recover. The
of poi nt s i n novel vi ews can be recovered f romone model vi ew onl y. Thi s i s apparent f r
t he f ol l owi ng deri vat i on. Consi der a poi nt p =( X; Y; Z) . I t s posi t i on i n a model vi ew i s
by ( x; y ) =( f X=Z; f Y =Z) . Now, consi der anot her vi ew obt ai ned by a rot at i on R around t
camera. The l ocat i on of p i n t he newvi ewi s gi ven by ( assumi ng f =1)
( 38)
( x0 ; y0) = ( rr XX ++rr YY ++rr ZZ ; rr XX ++rr YY ++rr ZZ )
i mpl yi ng t hat
( x0 ; y0 ) = ( rr xx ++rr yy ++rr ; rr xx ++rr yy ++rr )
( 39)
Dept h i s t heref ore not a f act or i n det ermi ni ng t he rel at i on bet ween t he vi ews. Eq. ( 39) bec
even si mpl er i f onl y rot at i ons about t he Y - axi s are consi dered:
+si n y
( x0; y0 ) =( 0x cos
;
)
( 40)
x si n +cos 0 x si n +cos where i s t he angl e of rot at i on. I n t hi s case can be recovered merel y f roma si ngl e c
spondence.
11
12
13
21
22
23
31
32
33
31
32
33
11
12
13
21
22
23
31
32
33
31
32
33
7 Experi ments
The LC met hod was i mpl ement ed and appl i ed t o i mages t aken i n an i ndoor envi ronment .
I mages of t wo oces, Aand B, t hat have si mi l ar st ruct ures were t aken usi ng a Panasoni c camer
wi t h a f ocal l engt h of 700 pi xel s. Semi - st at i c obj ect s, such as heavy f urni t ure and pi ct ur
used t o di st i ngui sh bet ween t he oces. Fi gure 3 shows t wo model vi ews of oce A. The vi ews
were t aken at a di st ance of about 4mf romt he wal l . Correspondences were pi cked manual l
The resul t s of al i gni ng t he model vi ews t o i mages of t he t wo oces are present ed i n Fi gur
The l ef t i mage cont ai ns an overl ay of a predi ct ed i mage ( t he t hi ck whi t e l i nes) , const ruc
l i nearl y combi ni ng t he t wo vi ews, and an act ual i mage of oce A. Agood mat ch between t he
two was achi eved. The ri ght i mage cont ai ns an overl ay of a predi ct ed i mage const ruct ed f r
a model of oce B and an i mage of oce A. Because t he oces share a si mi l ar st ruct ure t he
st at i c cues ( t he wal l corners) were perf ect l y al i gned. The semi - st at i c cues, however,
mat ch any f eat ures i n t he i mage.
Fi gure 5 shows t he mat chi ng of t he model of oce Awi t h an i mage of t he same oce obt ai ned by a rel at i vel y l arge mot i on f orward ( about 2m) and t o t he si de ( about 1. 5m) . Al t ho
17
Fi gure 3: Two model vi ews of oce A.
Fi gure 4: Mat chi ng a model of oce At o an i mage of oce A( l ef t ) , and mat chi ng a model of
oce B t o t he same i mage ( ri ght ) .
Fi gure 5: Mat chi ng a model of oce At o an i mage of t he same oce obt ai ned by a rel at i vel y
l arge mot i on f orward and t o t he ri ght .
18
Fi gure 6: Two model vi ews of a corri dor.
Fi gure 7: Mat chi ng t he corri dor model wi t h t wo i mages of t he corri dor. The ri ght i mage w
obt ai ned by a rel at i vel y l arge mot i on f orward ( about hal f of t he corri dor l engt h) and t
ri ght .
t he di st ances are rel at i vel y short most perspect i ve di st ort i ons are negl i gi bl e, and a goo
bet ween t he model and t he i mage i s obt ai ned.
Anot her set of i mages was t aken i n a corri dor. Here, because of t he deep st ruct ure
t he corri dor, perspect i ve di st ort i ons are not i ceabl e. Nevert hel ess, t he al i gnment res
demonst rat e an accurat e mat ch i n l arge port i ons of t he i mage. Fi gure 6 shows t wo model vi e
of t he corri dor. Fi gure 7 ( l ef t ) shows an overl ay of a l i near combi nat i on of t he model
wi t h an i mage of t he corri dor. I t can be seen t hat t he part s t hat are rel at i vel y di st ant
perf ect l y. Fi gure 7 ( ri ght ) shows t he mat chi ng of t he corri dor model wi t h an i mage obt ai ne
a rel at i vel y l arge mot i on ( about hal f of t he corri dor l engt h) . Because of perspect i ve di s
t he rel at i vel y near f eat ures no l onger al i gn ( e. g. , t he near door edges) . The rel at i vel y f
however, st i l l mat ch.
The next experi ment shows t he appl i cat i on of t he i t erat i ve process present ed i n Sect i
19
i n cases where l arge perspect i ve di st ort i on were not i ceabl e. Fi gure 8 shows two model v
and Fi gure 9 shows t he resul t s of mat chi ng a l i near combi nat i on of t he model vi ews t o
i mage of t he same oce. I n t hi s case, because t he i mage was t aken f roma rel at i vel y cl
di st ance, perspect i ve di st ort i ons cannot be negl ect ed. The eect s of perspect i ve di st ort
be not i ced on t he ri ght corner of t he board, and on t he edges of t he hanger on t he t op ri g
Perspect i ve eect s were reduced by usi ng t he i t erat i ve process. The resul t s of appl yi n
procedure af t er one and t hree i t erat i ons are shown i n Fi gure 10.
The experi ment al resul t s demonst rat e t hat t he LCmet hod achi eves accurat e l ocal i zat i o
many cases, and t hat when t he met hod f ai l s because of l arge perspect i ve di st ort i ons an i t e
comput at i on can be used t o i mprove t he qual i t y of t he mat ch.
8 Concl usi ons
Amet hod of l ocal i zat i on and posi t i oni ng i n an i ndoor envi ronment was present ed. The met h
i s based on represent i ng t he scene as a set of 2Dvi ews and predi ct i ng t he appearance of n
vi ews by l i near combi nat i ons of t he model vi ews. The met hod accurat el y approxi mat es t h
appearances of scenes under weak perspect i ve proj ect i on. Anal ysi s of t hi s proj ect i on a
as experi ment al resul t s demonst rat e t hat i n many cases t hi s approxi mat i on i s suci ent
accurat el y descri be t he scene. When t he weak perspect i ve approxi mat i on i s i nval i d, ei t
l arger number of model s can be acqui red or an i t erat i ve sol ut i on can be empl oyed t o acco
f or t he perspect i ve di st ort i ons.
The met hod present ed i n t hi s paper has several advant ages over exi st i ng met hods. I t u
rel at i vel y ri ch represent at i ons; t he represent at i ons are 2Drat her t han 3D, and l ocal i za
be done f roma si ngl e 2D vi ew onl y. The same basi c met hod i s used i n bot h t he l ocal i zat i
and posi t i oni ng probl ems, and a si mpl e al gori t hmf or reposi t i oni ng i s deri ved f romt hi s me
Fut ure work i ncl udes handl i ng t he probl emof acqui si t i on and mai nt enance of model s, devel
i ng eci ent and robust al gori t hms f or sol vi ng t he correspondence probl em, and bui l di ng m
usi ng vi sual i nput .
Appendi x
I n t hi s appendi x we deri ve t he expl i ci t val ues of t he coeci ent s of t he l i near combi nat i ons
case of hori zont al mot i on. Consi der a poi nt p =( x; y ; z ) t hat i s proj ect ed by weak persp
t o t hree i mages, , PP , and P0; P i s obt ai ned f rom Pby a rot at i on about t he Y - axi s by an
angl e , t ransl at imon, and
t scal e f act or
m, sand P0 i s obt ai ned f rom Pa rot at i on about t he
Y - axi s by an angl e , t ransl at
i on scal
t e ps. The posi t i on of p i n t he t hree i mages i s gi ven
p and
by
( x ; y ) = ( x; y )
1
2
2
1
1
1
1
20
Fi gure 8: Two model vi ews of oce C.
Fi gure 9: Mat chi ng t he model t o an i mage obt ai ned by a rel at i vel y l arge mot i on. Perspect
di st ort i ons can be seen i n t he t abl e, t he board, and t he hanger at t he upper ri ght .
Fi gure 10: The resul t s of appl yi ng t he i t erat i ve process t o reduce perspect i ve di st ort i o
one ( l ef t ) and t hree ( ri ght ) i t erat i ons.
21
( x ; y ) = ( smx cos +smz si n +mt ; sm y )
( x0 ; y0 ) = ( spx cos +spz si n +pt; spy )
The poi nt ( 0x; y0 ) can be expressed by a l i near combi nat i on of t he rst t wo poi nt s:
x0 = a x +a x +a
y0 = b y
Rewri t i ng t hese equat i ons we get
spx cos +sp z si n +pt = a x +a ( sm x cos +sm z si n +m
t ) +a
sp y = b y
Equat i ng t he val ues f or t he coeci ent s i n bot h si des of t hese equat i ons we obt ai n
sp cos = a +a sm cos sp si n = a sm si n tp = a tm +a
sp = b
and t he coeci ent s are t heref ore gi ven by
s si n( 0 )
a = p
si n sp si n a =
sm si n t s si n a = tp 0 m p
sm si n b = sp
2
2
1
1
2
2
3
1
1
2
1
3
2
2
2
3
1
3
4
References
[ 1] N. Ayache and O. D. Faugeras. Mai nt ai ni ng represent at i ons of t he envi ronment of a mobi
robot ,
Vol . 5, pp. 804{819, 1989.
[ 2] R. Basri and S. Ul l man. The al i gnment of obj ect s wi t h smoot h surf aces.
, Tarpon Spri ngs, FL, pp. 482{488, 1988.
[ 3] D. J. Braunegg. Marvel |Asyst emf or recogni zi ng worl d l ocat i ons wi t h st ereo vi si on
, 1990.
[ 4] D. F. DeMent hon and L. S. Davi s. Model - based obj ect pose i n 25 l i nes of code.
, Genova, I t al y, 1992.
IEEE Trans . on Robot i cs and Aut om
at i on,
Pr oc.
2nd I
Conf. on Com
put er Vi s i on
TR- 1229, MI T
Pr oc. 2
Eur opean Conf . on Comput er Vi s i on
22
[ 5] S. P. Engel son and D. V. McDermot t . I mage si gnat ures f or pl ace recogni t i on and ma
const ruct i on.
, Bost on, MA, 1991.
[ 6] C. Fennema, A. Hanson, E. Ri seman, R. J. Beveri dge, and R. Kumar. Model - di rect ed
mobi l e robot navi gat i on.
, Vol . 20, pp. 1352{
1369, 1990.
[ 7] M. A. Fi schl er and R. C. Bol l es. Randomsampl e consensus: a paradi gmf or model t t i n
wi t h appl i cat i on t o i mage anal ysi s and aut omat ed cart ography.
Vol . 24, pp. 381{395, 1981.
[ 8] D. P. Hut t enl ocher and S. Ul l man. Obj ect recogni t i on usi ng al i gnment .
, London, UK, pp. 102{111, 1987.
[ 9] D. G. Lowe. Three- di mensi onal obj ect recogni t i on f rom si ngl e t wo- di mensi onal i ma
, 1985.
[ 10] M. J. Mat ari c. Envi ronment l earni ng usi ng a di st ri but ed represent at i on.
, Ci nci nnat i , OH, 1990.
[ 11] R. N. Nel son. Vi sual homi ng usi ng an associ at i ve memory.
, pp. 245- 262, 1989.
[ 12] K. Onoguchi , M. Wat anabe, Y. Okamot o, Y. Kuno, and H. Asada. A vi sual navi gat i on
syst emusi ng a mul t i i nf ormat i on l ocal map.
,
Ci nci nnat i , OH, pp. 767{774, 1990.
[ 13] T. Poggi o. 3D obj ect recogni t i on: on a resul t by Basri and Ul l man.
, Povo, I t al y, 1990.
[ 14] K. B. Sarachi k. Vi sual navi gat i on: const ruct i ng and ut i l i zi ng si mpl e maps of an i
envi ronment .
, 1989.
[ 15] D. W. Thompson and J. L. Mundy. Three di mensi onal model mat chi ng f roman unconst rai ned vi ewpoi nt .
, Ral ei gh, NC, pp. 208{
220, 1987.
[ 16] S. Ul l man. Al i gni ng pi ct ori al descri pt i ons: an approach t o obj ect recogni t i on.
Vol . 32, pp. 193{254, 1989.
[ 17] S. Ul l man and R. Basri . Recogni t i on by l i near combi nat i ons of model s.
Vol . 13, pp. 992{1006, 1991.
Pr oc. SPI E Sym
pos i umon I nt el l i gent Robot i c Sys t em
s
I EEE Trans . on Sys t em
s, M
an and Cyber net i cs
Comm
uni cat i ons of
t he
ACM
,
Proc. 1s t I nt . C
on Comput er Vi s i on
Robot i cs Res earch Techni cal
Repor t 202,
Cour ant I ns t i t ut e of
M
at h.
Sci ences ,
New Yor k
Uni ver s i t y
Pr oc. I nt .
on Robot i cs and Aut om
at i on
DARPAI m
age Under s t andi ng
Wor ks hop
Pr oc. I nt . Conf . on Robot i cs and Aut om
at i on
Techni cal
Repo
9005- 03, I RST
AI - TR- 1113, M
IT
Pr oc. I nt . Conf . on Robot i cs and Aut omat i on
Cog
I EEE Trans . o
Pat t er n Anal ys i s and M
achi ne I nt el l i gence,
23