Nonlinearity 8 (1995) 461-476. Printed in the UK
A toral diffeomorphism with a nonpolygonal rotation set
Jaroslaw Kwapiszt
Mathematics Depmmenr, SUNY ai Stony Brook,
NY 11794-3651, USA
Received 24 October 1994
Recommended by H Hofer
We consimct a diffeomorphism of the two-dimensional torus which is isotopic to
the identity and whose rotation set is not a polygon.
Abstract
AMS classification scheme numbers: 54H20, 58F08
Introduction
The rotation set is an important invariant associated with a homeomorphism of the twodimensional torus. In the literature there are a number of different definitions of the rotation
set; however, all of these sets can only differ on the boundary. Moreover, all of the secs
have the same convex hull which is equal to the Misiurewicz-Ziemian rotation set. This
rotation set consists of all asymptotic average winding vectors exhibited by the system and
is always a compact convex svbset of the plane 171. ,
A natural question is whether every compact convex subset of the plane is a rotation
set for some IONS homeomorphism. This question is still open. Even deciding which line
segments are rotation sets has not yet been resolved. Given any interval with rational slope
passing through a point with rational coordinates it is easy to give a homeomorphism that
hai the interval as its rotation set. The construction in [5] implies that intervals with at least
one rational endpoint and irrational slope can be realized as well. Based on their results
for time-one maps of toral flows Franks and Misiurewicz conjecture that these two are the
only possibilities [5].
In the case of a nonempty interior, it is known that all polygons with vertices having
rational coordinates can be realized as rotation sets [3]. When the rotation set has an
interior, it depends continuously on the map [8]. This clearly implies that some sets which
are not rational polygons must occur, however it is still possible that only polygons occur.
In this p p e r we show that this is not the,case. We construct a C'-diffeomophism whose
rotation set is not a polygon. The regularity of the example is determined by the use of a
circle homeomorphism exhibiting a wandering arc. By Denjoy's theorem wandering arcs
do not happen in the Cz-category [Z, 61. .One is then prompted to ask whether there is a
Cm-diffeomorphism whose rotation set has infinitely many extrema1 points?
The construction of a C'-diffeomorphism, G, whose rotation set is shown in figure 1,
goes roughly as follows. Choose an in;\tional number p . The one-dimensional skeleton of
the two-dimensional torus is a bouquet of two circles: Call one of them the vertical circle
and the other the horizontal circle. The definition of G begins by defining a map F on this
t Supported in pan by Polish Academy of Sciences grant 210469101 Yieracje i F ~ k r a l e ' .
0951-7715/95/@40461+16$19.50 @ IS95 IOP Publishing Ltd and LMS Publishing Ltd
46 1
462
J Kwapisz
Figure
1.
bouquet. The map F is the composition of two transformations. The first transformation
is obtained by mapping the horizontal circle into itself by a circle homeomorphism which
has a rotation number p and exhibits a wondering arc containing thc iommon point of the
two circles. Since we want the transformation to be continuous we must fold a piece of the
vertical circle into the horizontal one, and we keep this folding as simple as possible. The
second transformation is defined analogously with the roles of the horizontal and vertical
circles interchanged.
The dynamics of the composition of the two transformations. F , will contain trajectories
which remain on a fixed circle. These orbits all have a common average rotation rate p
and give rotation vectors (0, p ) and ( p . 0). The trajectories which visit the vertical and
horizontal circles repeatedly account for all other extrema1 points except (0.0). Our ability
to effectively calculate the rotation vectors of these trajectories stems from the fact that the
essential part of the dynamics is controlled by an infinite Markov partition. Indeed, for any
given natural number n, the set of points on, say, the horizontal circle that jump to the
vertical one after exactly n iterates is an arc. The collection of thus defined arcs has the
Markov property. Section 2 contains the details of the construction of the map F on the
bouquet of circles.
The next step is to ‘perturb’ F to a C’-smooth embedding of a neighbourhood of the
skeleton into itself. The perturbation is carefully chosen so that its dynamics are as close
as possible to those of the inverse limit of the map on the skeleton. An obvious difficulty
arises at the trajectoly of the common point of the vertical and horizontal circles. This is
overcome by locking this point into a wandering domain. This is where the construction
depends crucially on the use of a Denjoy example. It is a routine to conclude that the
rotation set of the perturbed map is the same as that of F . To complete the construction,
we extend the dynamics to G on the whole toms by putting a source in the complement of
the neighbourhood of the skeleton. This places the vector (0,0) in the rotation set of G .
The construction of the ‘perturbed’ map with appropriate proofs can he found in section 3.
For a more general approach yielding only continuous embedding see section 4.
1. Preliminaries a n d statements of results
Let TZ be the two-dimensional torus R2/Z2 and K : R2 + TZ the projection. For any
compact subset X c T2, let ?
, := n - ’ ( X ) and denote by & ( X ) the set of all continuous
A toral diffeomorphism with a nonpolygonal rotation set
+
463
+
F : .f + 2 satisfying F(.? U) = F ( i ) U for all 2 E 2, U E 2'. Denote by X ( X ) all
mappings F : X + X that are projections of those in 7?(X). Notice that Z(T') is the
space of all continuous maps of T' homotopic to the identity. For any F E f i ( X ) we have
the displacementfunction 4.z : X --f Rz defined by $ ~ ( x :=
) F ( i ) - X where X IS a llft
of x . We borrow the following definition from (71.
Definition 1. For F E f i ( X ) and a point x E X , its rotation set p ( F , x ) is the set of all
limit points of a sequence (:(p(i)
- ?)".N
where 2 is a lift o f x . This set is independent
on the choice of lift 2. The rotation set of the map is
For F E X ( X ) we set p ( F ) := p ( F ) where
translation by a vector in Z2.
F is a lift of F.
This defines p ( F ) only up to
It is a useful general fact that the convex hull of the rotation set for any map is generated
by the rotation sets of generic points of ergodic invariant measures. In particular, we have
the following proposition. (Here we write Conv(A) for the convex hull of A c Rz.)
Proposition 1 [A.For any
E
f i ( X ) the following sets coincide:
fi) Conv(p(6);
(ii)( j 4 d~p : p is an F-invariant probnbility measure on X I ;
(iii)Conv(Up(k',x) : x E X such that p ( F , x ) i s a point).
Although we defined the rotation set-for maps on subsets of the torus our primary
interest is in the case when X = T2 and F E fi(Tz) is a homeomorphism. In this setting
p ( F ) is a convex compact in R2 [7]. Our main result is existence of a C'-diffeomorphism
6 E fi(Tz) with p ( G ) which is not a polygon. To be more precise let us adopt the following
definitions. (We use [xl for the smallest integer greater than or equal to x and LxJ for the
largest integer less than or equal to x.)
Definition 2. For any irrational p E ( 0 , l ) we define a sequence of numbers ( a n hby the
formula
an := rnpl
- np
and vectors pm.,, in R2 by
(rmpi, r p l )
m+n+l
Also we define the convex set Ap by
Pm.n :=
A, := Conv(0, P,,.~ : m , n E N, a,,'< p , or, < p )
Figure 1 gives an indication of the shape of A,. The next proposition asserts a crucial
property of this set, namely, that it is not a polygon.
Proposition 2. For any irrational p E (0, I), the set Ap has infinitely mnny extrema1points.
There are exactly two accumulation points of the extrema1 points of A,, namely, (0, p ) and
( P , 0).
Proof of proposition 2. Recall that pm." := ( x ~ ,ymJ
~ , where
xm," := rmpl/(m
+ n + 1) = (mp + a,)/(m + n + 1)
464
J Kwapisz
ym.,, := rn.oll(m
+ n + 1))
= (np
+ a,)/(m+ n + 1).
Suppose mk, nk E N, k = 1.2.3.. . are such that xm,."* -+ 0. Then nk/mb + CO, so Y , , , ~ . ~ ~
is asymptotic to Tnkpl/nr and thus converges to p . On the other hand, sequences ( m i )
and (nk) with
E A,, for which xmr.& + 0 exist due to irrationality o f p . In this way
the vectors pm.* in A,, which approach the y-axis accumulate to (0, p). Also, since A,, is
contained in the first quadrant { ( x , p) : x , y
0) and contains the origin, it follows that
(0, p ) is an extremal point.
Denote by y,,,," the slope of the line through pm." and (p.0). We claim the following:
if me, nk with p,,.,, E A,, for k = 1,2,3.. ., are such that
>
lim
K+m
Ymk.nk
= SUP(Ym,n : Pm.n E Ap)
then ymr.nr.
is not eventually constant and limt,,
To prove this claim, note that we have
Ymr.nr
=
nkp
P,,,~,,,~ = (0, p).
:
p(mk + nk f 1)
mkP f a m i
Since p is irrational sot's can be arbitrarily close to p so it is clear that the supremum is
z -1 and is not attained. Thus either mk + CO or nk -+ CO. The first possibility leads to
ynJB,nt
-+ -1 which is definitely not the supremum. It follows that mk's are bounded and
nk + CO which gives P ~ , . ~+
, (0, p).
This last fact guarantees that (0, p ) is a condensation point of extremal points. The same
is true of ( p , 0). In this way we have an infinite number of extremal points. Moreover,
using the formula for the slope ym." one can see that points P , , , ~ accumulate towards the
antidiagonal ( ( x , y ) : x y = p}. Clearly there are no other extremal points on it except
for (0, p ) and ( p , 0). This proves the second p m of proposition 2.
0
+
Now we are ready to formulate our theorem.
Theorem 1 (main theorem). For any irrational p E (0,l) there is a C'-diffeomorphism
G E 'H(T2) whose rotation ser p(G) equals A p (modZ2).
An outline of the proof of theorem 1 can be found in the introduction. Sections 2 and
3 give more formal arguments.
2. A map on a one-dimensional skeleton of the torus
Denote the x and y-axis in Rz by R") and R("),respectively. These lines project to circles
on T2 which we call S(*) and S("). We put X := S(*) U S(") and .f := d ( X ) . We
have obvious injections R/Z = S' -+ S") for U E ( h , U). For x E S' we denote the
corresponding point in S") by x(") E SC"), U E ( h , U]. We also have a 'projection' X + S'
sending x H that is defined by the relation .
"= x , x E X.
Define the set a,, c R2 by
x
a,, := Conv(p,,,,
: a,,,,
a. < p }
where pm.n.am and ct, are as defined in section 1.
We construct F : X + X , F E R(X) with a rotation set
p ( F ) = Q,(modZZ)
A toral diffeomorphism with a nonpolygonal rotation set
465
This map is a composition of two analogous transformations of X. Roughly speaking,
the first one is what one gets trying to rotate S") by a positive angle keeping the map
continuous and the unavoidable folding of S(") into Sch) as simply as possible. The other
does the same to S("). The detailed definition^ preceded by some necessary preliminaries
follows.
We think of the one-dimensional circle SI as R (mod 2).Let I be a small symmetric
closed arc around zero and i its lift to R containing zero. For q5 : S' + SI a degree-one
map that is 1-1 everywhere except I where it has a plateau, let : R + R.be the lift
satisfying r := $(f) E [0,1] (see figure 2). As long as we are not concerned with the
issue of smoothness, to make our construction work we need to choose any such q5 with
the rotation number p ( 4 ) equal to p.
4
thegraphof (p
Figure 2.
However, since we ultimately want a C1-example we have to choose q5 more carefully.
We may start with a Denjoy C'-diffeomorphism of S' with rotation number equal to p. We
may further require that it has a wandering domain slightly larger than I, say equal to the
dilation of I by a factor of
about the origin that we denote by g l . Now since % I is
wandering, any modification of the diffeomoiphism on this arc without altering its image
does not affect the rotation number. In particular, we can redefine the map on $1 so that 1
is sent to a point and C'-smoothness is preserved. This is the map we are ultimately going
to take for 6. Let us stress at this point that we will not use wandering properties of q5 until
the considerations of section 2.
There is a unique homeomorphism : S' + SI such that q5 = I/! o p where p collapses
I to zero and is Affine on the complement of I. Indeed, @(x) := q5 o p-'(x) for x # 0
extends through 0 because q5 o p-'(O) = @(I) = 7 is a point. For our q5 derived from
Denjoy's example the map is C'-smooth.
Let q : [-1. I] + [0, 11be given by the formula q ( x ) := l - x z . Take for qr : I + [O,r]
the map whose lift is a linear rescaling of q to a transformation from f to [O,r]. The
following definition gives for each U E ( h , U ) a continuous F(") : X + X . (Here by
definition I; := U and ij := h.)
'E
e
e@("'
ifx
E
S")
qr@(")
if x
E
I(:)
p@@'
if
E
,
\ j(t),
Remark 1. The definition is valid for any degree-one mapping @J which is 1-1 everywhere
J Kwapisz
466
except a symmetric plateau around zero. Moreover, F(") depends conrinuously on r$ in the
co-ropology.
The map F is defined as a composition of F'") and F"),
F := F(U) F(").
It is easy to homotope F(") and F(")to the identity. In view of remark 1 one can
do this by coming up with an appropriate homotopy connecting r$ to the identity. Say
4, := o pr where @, := id t ( $ - id) and p, collapses t i to zero being Affine on
the complement of t l . By lifting homotopies connecting F(" and F(") to the identity to
homotopies also terminating at the identity, we get lifts of our mappings that we denote by
F(') and F('), respectively. Both F(")and F(') belong to f i ( X ) . So does their composition
+
+,
F := p(") F : ( k ) ,
Proposition 3. For
F defined as above we have
p ( F ) = Q,.
It is convenient to notice that in view of part (iii) of proposition 1, proposition 3 reduces
to the following.
Proposition 4.
(i) rffor z E Z the limit p ( F : , i )=limn-,- t ( P ( i )- i )exists, then p ( F , i )E Q~
(ii) For any m, n E N with a,, a,,< p. there is 2 E .f with p ( F , Z ) = pm.n.
The remainder of this section is devoted to the proof of proposition 4.
Proof of proposition 4. Since F = F(")o F(') we will find it convenient to regard as the
(forward) orbit of x E X the sequence XO, XI/Z, XI, ~ 3 1 ~ .2 , ..., with xo := x , XI/* :=
Fth'(x), XI := F ( x ) , 2312 := F") o F(x). . ..
Looking at the definition of the map F(') we see that it preserves S''). The circle S(")
is not forward invariant, however the only part of S(")that is folded into S(') consists of
I(") which (by definition) is a symmetric neighbourhood of the origin. For F(") we have
an analogous situation (with U and h switched).
There are two types of (forward) orbits:
e
eventually free, i.e. staying in the complement of I(') U I(") for all sufficiently high
indices;
interacting, i.e. having returns to I ( ' ) U I(") for arbitrarily large indices. (In fact, returns
to I(') and I(") must alternate.)
To understand what happens in each of these two cases and develop a useful notation
we draw the following diagrams associated with the orbit of xo E X .
Case offree xo E S('):
F(")
*
X p 4
F(U)
x$
F(k)
+ XI"' + ;;X
P
@
-
L
4
_
F(u)
F(k)
i
Xz(h)
P
_
F(J1)
+ x g +X
P
*
_
Q
F('J)
y 4
@
v
4
....
(1)
467
A fora1 diffeomorphism with a nonpolygonal rotation set
Case of free xo E S");
F(h)
xo("I
-
F("'
+ xi;; +
*
P
F(")
x y
xi;;
VJ
P
-*
F(")
F("'
F(h)
+ x2(U) + xi;;
.. ..
P
L_r__L___-
d
0
d
Case of interacting x g E 1'")for which x n t + p := Fch) o F"(x0) sirs in I") and x,+,+I
F"+"+'(xo) is thejirst retim to I("):
F(h)
@U'
F(h)
F(h)
F'")
F(h)
F(h1
0')
("1
$1
---f x;;;
----f
+ . . ..:
Xm+l/Z
%+I
x23,*.
-Xy)
vr
P
*
.
*'
d
8
F'"'
+
P
VJ
P
%
c___i
d
--*
d
=
+
F(h)
.;;,":,
-.+ . . . .... . .
(2)
(U)
Xniin+,
+'.
., .
(3)
. .
4
The diagrams indicate which of FCh) or F(") acts and what the corresponding transformation
of circle coordinate is. For example,
F(h)
x
~2'
*
+
(h)
xm+l/?
F ' h ) ( ~ , )and x,+1/2 = $(x,).
- It also indicates that both x,,, and
x , + ~ , ~belong to So". (The ambiguity arising for 0 = O(h) = O(")will cause no confusion
in our further discussion so it is ignored.)
We see that if x~ stays on one of S'"), U E ( h , U ) . f o r k = 0,
1,
2, . . ., then the
projected coordinate XK on SI evolves under the iteraies of @ =
o p (i.e. XX+I = @ (xK)).
When x x hits I ( - ) , a E { h , U } . it makes a transition to the other circle S @ ) through the
folding action of qr. Given m, n E N , all points x g E S'"' behaving according to diagram (3)
form a set (perhaps empty) that will be denoted K t L . That is, for any m, n E (0,1,2.. .},
we set
tells us that
~ , , , + ~ /=
2
*4,
2,
KZL := {XO E I("' : x,+,+I is the first return to 1'") and x,,,+1/2 E I(")).
Denote by h : SI + SI the unique semiconjugacy of @ to rigid rotation by the angle
p such that h(0) = 0. We define H : X + X by H ( x ( u ) ) := h(x)("),U E ( h , U ) . Notice
that H ( l c h ) U I(")) = (0, O), so in particular, H is a well defined continuous map. For
# : --f t ~ R2
c we take the lift of H preserving (0,O).
x
Claim 1. r f X E 2 is as in (i) ofproposition 4 and x = n(X),then
(i) forx evemuz1lyfre-e. p ( F , X) E ((0, p ) , ( p , 01);
(ii)f o r x interacting, p ( p , i )belongs to
Proof of claim 1. We start with a standard observation that for any fi E & ( X ) the limits
of sequences k(p(i)
- i) and
o *(i) - H ( i ) )coincide. Indeed, since I?(?) - 7 is
a Z2-periodic function of 9, we have IIE o p ( i )- *(.?)[I
< sup,,,> Ill?(?) - ,I'1 < +ca.
i(fi
468
J Kwapisz
If x is eventually free, after skipping a finite number of iterates, we %e in the
situation of one of the, first two diagrams. In the case of diagram (1) ,we write
fio P ( 9 - fi(i) = fi(i+
2(io)+X;:; (fi(i,+,,,)- fi(2k-l/z))+fi(i")
- H(inn-l/;?).
Since f i ( & + l p ) - f i ( i k - l , d = ( p , 0) we get p ( p , x ) = ( p , 0). Similarly in the case of
diagram (2) we have fi c p(2) fi(i) = x:II=,(f?(L)
- fi(?k-J) =
p ) , so
P ( F , x ) = (0,PI.
For an interacting orbit we may assume that it stam in I("). Then we split it into
segments between consecutive returns to I("). Each such a segment (after obvious shift
of indices) looks like that in diagram (3). In particular, it has an associated (m,n ) E N2
such that x,+]/z E ICh) and m n 1 is the total length of the segment. Now if we
take N E N such that XO, ... ,X N is a certain concatenation of segments as above, then
i ( f i o F N ( i )- fi(i))is a convex combination of average displacements in each segment.
0
x:lt=l(O,
-
+ +
Let us now calculate the average displacement on the sets
K2.i.
Claim 2. rfn(2) E K$', for some m , n E N, then
fi
p+"+I(i)
m+n+l
Proof of claim 2.
-~fi(f)
= Pm.n.
For io with a(20) = xo E K::,
we have
AGO) = (fi(fl:1,2) - f i G 0 ) ) + C L (fi(2k+1/2) - A ( i k - I / Z ) ) + (fi(f",+I)
E;=](A(&+,+d
-A(%",)).
fi(fm+,+l)
-
- ri(im+,/2)+
Observe that H ( x 0 ) = H ( x , + l p ) = H ( x ~ + ~ +=~ (
) 0.0). Also H ( @ - r ( I ) ( u ) ) =
(-rp (modZ))(*) for r E N, while xP+lp E (q5-mck(I))(h) for k = 0, _ _m_ and
,
Lemma 1. The set KZ', is nonempty if and only if or,,a, e p. Moreover, if K$L
is nonempry, then it is a disjoint union of foirr subarcs of I("), each mapped by F"'+"+'
homeomorphically onto I(").
Proof of lemma 1. The arcs { q 5 - i ( l ) ( ' ) ] j e ~ are pairwise disjoint and ordered on S")
in the same manner as the corresponding backward orbit of rigid rotation by p. Also
q r ( I ) = [0,@ ( I ) ] .In this way the set of all points in I which get folded by qr into @-'"(I)
is either empty or consists of two arcs, each mapped by qr homeomorphically onto @+"I).
The set is nonempty exactly when -mp E (0,p) (modZ), i.e. -mp - L-mpJ < p or
equivalently a, = rmpl - mp e p .
Now look at diagram (3). For a point xo E I(") to have x,+l/z as the first iterate hitting
I'h) means that q r ( s ) E q5- ( I ) . 1.n view ofthe preceding remarks such points xg exist only
if a, e p . and then they form two subarcs of I('), call them K , and Kz. Note that under
Fm+l/' = F m o F(') (which is essentially @"' c qr), each of them maps homeomorphically
onto I('').
Analogously, points xm+l/z in ICh)that have x , ~ + " + ~as the first iterate hitting I(") exist
if and only if a. e p. As before, if an p, then those points form two subarcs of I ( h ) ,
call them L I and L2. Under F" o F(") (which is essentially q5" o q<), each of them maps
homeomorphically onto I(") (see diagram (3)).
469
A toral diffeomorphism with a nonpolygonal rotation set
K$i
Now
is clearly equal to F-(m+'/2)(L1,U L2)n ( K , U K2). This set, if it is
nonempty, consists of four subarcs of IC") each mapped by Fm+"+' = Fm+'/* o F(") o F"
homeomorphically onto I@). The lemma is proved.
ci
.
.
We are ready to conclude the proof of proposition 4. To prove (ii), note that from
lemma 1 we see that for m , n with cm, a, < p the sets KZL, F-(m+n+L'(KZ)J,
F - 2 ( m + " + 1 ) m.n
( K ( U.).)., are nested, so the intersection
F-k(m+"+')(KZ',) is nonempty.
Take any x in this set.
nEo
To prove (i) we have to consider two possibilities. For eventually free'x we.use part (i)
of claim 1 together with the fact that both (0, p ) and ( p , 0) belong to Q p . For interacting
x we have to combine part (ii) of claim 1 with claim 2.
0
3. A diffeomorphism of the torus from the map on the one-skeleton
Recall that $1 is a wandering arc under 6 and set
W :=@(GI).
Shrinking $1 we can get a symmetric arc I" containing I such that Jo := p(I") is a
nondegenerated arc and q,(Jo) is contained in W (see figure 3). Note that @ ( J o ) c W . Put
I' := q;'([O, r ] \ Jo)
and
AI := cl(['' \ 1').
(See figure 3.)
'ql,(AInJ)
*---
-6
,I I,
1
$
I
1
I
!
*
\,
I
LJo--2
r
, __-_-I"
,
I
L
I
I
VJJJ
3
-_j
(I I
<
,
I
_-_______
!I
AI
II 1L
WfJJ
vm
j j,
I
L---_-_---_-----~
W
c _ _ _ - ~ - ~ ~ ~ ~ ~ ~ ~ ~ _ _ _ _ _ _ _ _ ~ ~ ~ ~ ~ !
18/17 I
Figure 3.
For c E [h, U } put
U(') := { x E Tz : dist(x, S'"') < $length(Jo)}
.
(I := U'h'
U U(").
Also set
C := { ( x , y ) E U : 1x1, IyI < $ength(Jo)).
There is a retraction r : U \ C + X \ C given by
r ( x , y ) :=
I
(x,0)
for ( x , y) F U(")
(0, y )
for (x,y ) F U ( " ) .
470
J Kwapisz
Theorem 1 is a consequence of the following proposition which is the main result of
this section.
Proposition 5. There exists a C'-embedding G : cl(U) -+ 'L such that p ( G ) = R,.
Let us first see how theorem 1 follows from this proposition.
Proof of theorem 1. It is simple to extend the map G acting on U c to a diffeomorphism
on the whole T2 by putting a single source outside U repelling all other points towards
G"(U) (i.e. we require that T2\n,,, G l ( U ) is a basin of attraction under G-I). For
this extended G we have p(G) = Conv[(O, 0), R,) = A,.
0
n,,,
The rest of this section is devoted to the proof of proposition 5.
Proof of proposition 5. The construction of the C'-embedding G : cl(U) -+ U is done in
steps.
Step 1. As suggested by figure 4, we modify F(") on (I")'") to obtain F:)
C'-smooth embedding satisfying the conditions listed below.
:X
+Ua
(A) (Ff'))-'(cl(C)) = AI(') U @-'(Jo)"';
(B)F ~ " ( c ~ ( cc) r) - ' ( W * ) ) :
(c)if x 6 c U ( F ~ ) ) - ' ( c ) then
,
r o F?'(X) = F ( ~ ) ( x ) .
Step 2. We extend F f " to a C'-smooth embedding G'") : c f ( U )-+ U with the Following
analogues of conditions (A), (B)and (C) satisfied:
(A') (G(*))-'(cl(C))= r - ' ( A f ( " )U @ - ' ( J , ) ( * ) ) ;
(B') G'h)(cl(C))c r-'(W(h));
(C') if x 6 C U ( G h ) ) - ' ( C ) ,then r o cYh)(x) = Fh)
o r(x).
See figure 5 for the construction of G('). In an analogous way we obtain G(") from
F(").
Step 3. Finally we define G : cl(U) + U by
G :=
0
Let us now concentrate on these aspects of the dynamics of G that are critical for the
calculation of the rotation set p ( G ) . First let us see how alternating applications of G(")and
I
Figure 4.
47 1
A toral diyeomorphism with a nonpolygonal rotation set
Follow e'on X sending marked
fibers on one picture 10 these marked
on the other.
Figure 5.
G(') act on r-'(W(')) and r-'(W(")). We expect that these domains are wandering under
G and move 'freely' governed by the action of .$ on the appropriate circle. We formalize
this in the following lemma. (We deal with the case of r-'(W(')). For r-'(W(")) there is
an analogous statement.)
Lemma 2. For any M = 0 , 1 , 2 . . .
2N mmpodfions
r
.
.
(a)if M is even and M = 2N, then the set G(h)o G(") o . .. o dh)
o G ( " ) ( T - ] ( W ( ~ is
)))
contained in r-'(@N(W)(h))
which is disjointfrom C U (G("))-'(C):
2N+1 compositions
(b) i f M i s o d d a n d M = 2 N f l , t h e n G ( " ) 0 G ( ~ )...
o ~G(~)oG(~).(r-'(W(~)))iScontained
in r-'(p o @N(W)(h))
which is disjointfram C U (G(*))-'(C),
Proof of lemma 2. We proceed by induction on M.
For M = 0 our claim follows from (A') for G(") and the fact that W n (Jo U AI) = 0.
--
Now we will describe the induction step.
First we deal with part (b). By the inductive hypothesis and (C') for G(")we have
2N+1
r o G'") o . .. o G(")(r-l(w('))) c r c
o r-'(@N(W)(h))
=r
or-' o F ( " ) ( $ ~ ( W ) (=
~))
2N+I
po@'"(W)('). Thus G ( Y ) ~ . . . ~ G ( " ) ( r - ' ( W ( *~)r) )- ' ( p o @ ~ ( W ) ( ' ) ) .
Now r - ' ( p o @ N ( W ) ( h ) ) r l C= 0 follows from p o @ N ( W ) f l J o= 0 which is equivalent
to ~ o p c @ N ( W ) n @ ( J=o @Nt'(W)n@(Jo)
)
= 0. Since $(Jo) c W and W is wandering
this is true.
On the other hand, r - ] ( p o @N(W(*)))
n (G('))-'(C) = 0 in view of (A') follows from
p o @ N ( W ) n @ - ' ( J o=
) 0 or equivalently @ N + ' ( W ) n J o= 0, once again by the wandering
property of W .
J Kwapisz
412
Let us now consider part (a). By the inductive hypothesis and (C') for
2N
,
7
we have
.
r o GCh)o .. . o G(")(r-'(W(k)) = r o G(h)o r - l ( p o @N-'(W)(h))
= r o r-I o F ( * ) ( p o
@N-l(w)(*))
= ( $ ( p o @ N - I ( W ) ) ( h )=@N(W)(h).
Now r ~ l ( @ ~ ( W n
) (C~ =
) )0 follows from C N ( W )n JO =~0.On the other hand, using
the version of (A') for.G(") we see that r - l ( @ N ( W ) ( hn
) )(G("))-'(C) = 0 follows from
4N(W)nAI=0.
0
~
Now we are going to verify that the region C under iterations of G stays in the wandering
domains of lemma 2. This assures that all the rotationally nontrivial dynamics are carried
on a certain forward invariant set C on which r is a well defined semiconjugacy between
G and F . The lemma below gives precise formulations.
C-,,p = U,",
Lemma 3. IfCrit :=
G-"(C U (G(h))-'(C))and C := U \ Crit, then
(i) G(X) c 2 ;
(ii) For any x E U there is N E (0,1,2.. .} such that XN := ' G N ( x )E C. Moreover, $
x 6 C then there exists N E N such that either X N + ~E r-' ( p o cpk(W)"") for all k E N
or X N + ~E r-'(@k(W)c"))forali
k EN;
(iii) The following diagram commutes:
C
G
+
E
r.l
.IT
r(c) ' 4 r ( E ) .
Note that the nonwandering set of G is contained in E. Also the nonwandering set of
F sits in r ( Z ) .
Proof of lemma 3.
(ii) Suppose that xo 6 C. Then for some N E { 1 , 2 , .. .) we have X N - ~ E C U (G(h))-'(C).
If xN-l E C,then by (B') we have X N - ] / 2 = G(*'(XN-I)E r-'(W(h))and from lemma 2
we conclude that for every k 0
>
=+I
com~osiliom
~
XN+K
= G(') o G(h)o ... o G"'
o
G@"(xN-I/~)
is contained in r - ' ( p o @k(W)(h)).
This set is disjoint from C U (G(h))-'(C)so X N ZCrit.
If XN-' E (GCh))-'(C),then x p ~ - ~=p ( G ( * ) ) ( X N - ~E)C and, by (B') for G("), we have
XN = G(")(xN-IJ
E~r-I(W(")).
)
Applying lemma 2 with the role of U and h interchanged
we see that for every k 2 0
2k cnmposirionr
X N + ~= G(') o
G(h)o ... o G(")o G ( h ' ( ~ ~ )
is contained in r-'(@'(W)(")). This set is disjoint from C U (C(b))-'(C)and so X N ZCrit.
(iii) Suppose that xo 6 Crit, then xo 6 C U (G(h))-'(C)and x l p = Gch)(x0) @
CU(G("))-'(C). Indeed, otherwise xo E CU(Gc"))-'(C)U(G("oG(h))-'(C) c Crit. Thus
we ma9 apply (C') with x = x g and (C') for G(")with x = x l I 2 to get r o G(")o G@)(x)=
F(") o P h ) ( r ( x ) ) .
0
To conclude the proof of proposition 5 we need to show that p ( F ) = p ( G ) = Sp.
Observe that lemma 3, part (ii) tells us that points x
C contribute only ( 0 , p ) or
( p , 0) to the rotation set of G. The analogous statement i s also true for F, that is, the
A toral diffeomorphism with a nonpolygonal rotation set
473
contribution of points which are not in r ( E ) to the rotation set of F is either ( p . 0) or
(0,p ) . This contribution is due to free orbits on S(h) or S'"), respectively. Moreover, using
the diagram in (iii) of lemma 3 lifted to the universal cover we see that for any x E E,
p(6.x) = p ( F , ~ ( x ) ) Indeed,
.
we have I/&'(.?)
- @ o ?(:)I/ = IIC"(:) - i o &'(,?)I] 6
0
suPyE$ IliW) - 711 < w.
4. Deriving invertible dynamics on the torus from noninvertible dynamics on a
skeleton
In this section we want to indicate that, as far as one is ready to give up on smoothness
requirements, there is a general, method essentially replacing considerations of section 3.
The construction is fairly robust and intuitive, so we assume a very informal style to avoid
blurring the essence with a cloud of details.
Our approach is a variation of the method used by Barge and Martin [4] to prove that
inverse limits of interval transformations can be realized as attractors for homeomorphisms
of the plane. The key fact is the following theorem which is an easy corollruy of Morton
Brown's results in [I]. Let us recall that a continuous map f of a compact metric space
Y is a nearhomeomorphism if and only if there exists a sequence of homeomorphisms
f. : Y + Y converging uniformly to f.
Theorem 2. If Y is a compact metric space, then any nearhomeomorphism f : Y + Y is
Y such
a factor of a homeomorphism f of Y , i.e. there is a continuous onto map h : Y
t h a r h o f = f oh.
The paper of Brown is clear and puts the above theorem in^ a perspective of general
facts about inverse limits. However, we feel it will be beneficial to the reader if we present
a self-contained proof of the result.
Proof of theorem 2. Let d be the mehic on Y . Given any two maps of Y , say f and
g, use d( f,g) to denote their uniform distance equal by definition to sup(d(f(x), g ( x ) ) :
x E YJ. Suppose that a sequence of homeomorphisms (f n ) n E N converges uniformly to f.
Compositions f i o ... o fn and f;' o ... o f;' are denoted by fl." and f,,,;:respectively.
For any n E N set
an :=~sup(maxld(fi,~-i(x),~ L - I ( Y ) ) , d(&n-i
: d(x, Y ) < d(f-3
B.
:= sup(maxId(fi,,-~(x),
:
:= 3(%
f (x). fi,,-i 0 f ( y ) ) )
0
f ( x ) , &-I
f)J
.L-I(Y)),
d(fi,n-i
0
f ()'))I
Y) < d(f"+l,f)J
yn := W d ( f " - ' ( x ) , f"-'(y))
E"
0
: d(x, y) c d ( f , fd}
+ 0" + y d .
Notice that skipping certain elements of the sequence (fn&
one can make the sequence
(6,Jz1
converge to zero as fast as we wish. Indeed, one can proceed inductively as
follows. Suppose that we have already chosen f i , ... ,f. so that a i , . .. ,a,, pi,. .. ,
and y i , ..., yn 'are as small as we wish. By the uniform continuity of the functions f].,,,
f1..-1,
fi., o f, ~ L - I 0 f and P , by picking fn+1 sufficiently close to f we get an+],
B.
and yn+l as small as we wish. This ends the induction step. In this way we may assume
that E , c w.
J Kwapisz
414
We claim that the sequences consisting of the following maps of X are uniformly
convergent:
f^. :=
fi.n+l 0
f2
1. := fl..
0
f21.i
h, := f" of:;.
in
This claim already implies the theorem. Indeed, we have ino = fn o iX= idx and
h,
o
s f o h.. Consequently, the limits, denoted by f,2 and h, respectively, satisfy
f o g = g o f = i d x and h o f =~
f oh.
Convergence of the sequences is an immediate consequence of the assumption about the
convergence of the series of c,'s and the following estimates valid for any x E X. Setting
y := f,T; (,r) we see that
A
^
d ( f , ( x ) ,f^.-i(x)) =d(fi.n+i
0
fi..
f;;f,T:o,
0 f,?i.iCx))
+d(fl.n-l
f 0 fn+1Cv). f1.n-I 0 f n 0 f(Y))
f 0 f n + l ( U ) , h.n-I0 f 0 fCv))
0 f 0 MY). fl..-I
f 0 f(Y))
0 fn 0 f"+l(Y)? f1.n-I 0 f 0 fn+l(Y))
+d(fi.n-l
o f o f n ( ~ ) f1.n-i
,
ofofn(~))
=q
L - 1 0
< d(fi.n-10
fd(f1.n-l
0
< 6" +a, +an +a.
Similarly taking fn&(x)
44.
for y we get
d(i%(xL &-I@)) = d ( f i . n 0 f , I , l , f1.n-I 0 f;;)
=d(fi,n-, 0 fn(y), f i . n - I 0 ~ + I ( Y )4
) d(f1,n-l 0 f n ( ~ ) , fin-I
+d(fi,n-l 0 fn+~(y), fi.n-1 0 f(y)) 4 6n +an Q en.
Finally using y :=~
f;; ( x ) we obtain
d(h,(x),h,t-i(x)) = 4f"
0 f[;(x),
K 6" .
< <
f"-' 0 f,,.,(x))
=d(f"-'
0
0
f (Y))
f ( ~ Ti
), 0fd~))
0
We will now explain how theorem 2 enables us to find a torus homeomorphism with
rotation set equal to A,. Sections 1 and 2 are prerequisites for our considerations. In
particular, one should look there for definitions. For the space Y in theorem 2 we take the
torus T2. For the map f we need a homotopic to the identity near homeomorphism which is
an extension of the map F : X + X to'the whole torus and has X = S(') U S(")as a global
attractor. Such a map f would have the desired rotation set. Construction of the map f
is not difficult. Once again it is convenient to obtain it as a composition of two 'maps f")
and f'"). We will roughly sketch the constntction of fCh) now. To get f'") interchange the
role of horizontal and wartical directions.
First we take an embedding g(k) of X into the torus such that its postcomposition with
the map collapsing the marked annulus (see figure 6 ) onto S") along vertical fibres is equal
to F('). Next we extend g@) to a homeomorphism of Tz which we are going to denote
with the same letter. We also extend the collapsing map to a mapping p(h) of T2. These
extensions need to be reasonably chosen since we want X to be a global attractor. For f (4
we take the composition p(h)o g(". This is a nearhomeomorphism because p(h) is one:
the maps pi') contracting the annulus along the vertical direction by a factor say 1f n are
homeomorphisms that approximate it.
From theorem 2 we get a homeomorphism f of which f is a factor. This map has the
same rotation set as f-it is equal to A, as required.
A toral diffeomorphism with a nonpolygonal mtaiion sei
----------
475
----------
Figure 6.
The last implication hinges on the fact that the factor map is homotopic to the identity
and the following simple observation.
Fact 1. If f,f,r : TZ + TZ are continuous, homotopic to the identity and satisfy
f c r = r c f,and furthermore F , p , R : Rz+ RZ are their lifts such that F o R = R c i,
then p ( F ) = p ( ~ ) .
Proof. The map R is surjective and, for any x E R?, we have IIP(x) - F" o R(x)II =
I I P ( x ) - R o @(x)[l < supJER[ [ R ( y ) y [ [< 03. The fact follows now trivially from the
defmition of the rotation set.
U
If r is not homotopic to the identity, it induces a linear map r, on the first real homology
which we identify with the universal cover RZ. Then p ( F ) = r * ( p ( p ) ) ,provided r , is
nonsingular (or at least r is surjective). To see this one can modify slightly the above proof,
or just compose r with the linear toms automorphism covered by r;' and use fact 1.
Let us end with a remark on the continuity of the dependence of
f on f.
Remark 2. rf f' : Y + Y , t E [O,11 is a homoropy of nearhomeomorphisms and there
exists a sequence ofisotopies fi : Y + Y, t E [0,1] converging uniformly in r to f ' , then
one can assure that the family of maps f^'from theorem 2 is also a homotopy.
'To see why the remark is true, observe that the dependence of maps f^' on t is
continuous if only all estimates in the proof of the theorem are uniform in I . This
uniformity would follow if we choose the approximating family f; in the proof so that
E, := SUP(E: : r E [0, I]] has as before a finite sum
E, < m. One achieves this by an
analogous inductive procedure to that from the beginning of the argument.
Acknowledgments
I would like to thank John Milnor for asking the question that prompted this work. I am
also very much indebted to Phil Boyland whose support and countless suggestions made
this paper possible.
References
[I] Brown M 1960 Some applications of an approximalion theorem for inverse limits Pmc. Am. Math. Suc. 11
478-83
[Z] Denjoy A 1932 Sur les courbes definies par les equations differentiales a la surface de tore 1.Math. Purer
Appliquees 11 33S15
416
J Kwapisz
[3] Kwapisz J 1992 Every convex rational polygon is a rotation set Erpd. Theor. Dynom. Sysf. 12 333-9
141 Barge M and Martin J 1990 The construction of global anmctors Proc. Am. Moth. Soc. 110 523-5
[SI Franks I and Misiurewicz M 1990 Romtion se& of Coral flows Pmc. Am. Moth. Soc. 109 243-9
[6] Nitecki Z 1971 Diferentiable Dynamics~(Cambridge,MA: MIT Press)
171 Misiurewicz M and Ziemian K 1989 Rotation sets for maps of tori 1.London Math. Soc. 40 490-506
181 Misiurewicz M and Ziemian K 1991 Rotation sets and ergodic measures for torus homeomorphisms Fund.
Md h . 137 45-52