TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 00, Number 0, Pages 000–000 S 0002-9947(XX)0000-0 CROSSINGS AND NESTINGS OF MATCHINGS AND PARTITIONS WILLIAM Y.C. CHEN, EVA Y.P. DENG, ROSENA R.X. DU, RICHARD P. STANLEY, AND CATHERINE H. YAN Abstract. We present results on the enumeration of crossings and nestings for matchings and set partitions. Using a bijection between partitions and vacillating tableaux, we show that if we fix the sets of minimal block elements and maximal block elements, the crossing number and the nesting number of partitions have a symmetric joint distribution. It follows that the crossing numbers and the nesting numbers are distributed symmetrically over all partitions of [n], as well as over all matchings on [2n]. As a corollary, the number of knoncrossing partitions is equal to the number of k-nonnesting partitions. The same is also true for matchings. An application is given to the enumeration of matchings with no k-crossing (or with no k-nesting). 1. Introduction A (complete) matching on [2n] = {1, 2, . . . , 2n} is a partition of [2n] of type (2, 2, . . . , 2). It can be represented by listing its n blocks, as {(i1 , j1 ), (i2 , j2 ), . . . , (in , jn )}, where ir < jr for 1 ≤ r ≤ n. Two blocks (also called arcs) (ir , jr ) and (is , js ) form a crossing if ir < is < jr < js ; they form a nesting if ir < is < js < jr . It is well-known that the number of matchings on [2n] with no crossings (or with no nestings) is given by the n-th Catalan number 2n 1 . Cn = n+1 n See [25, Exercise 6.19] for many combinatorial interpretations of Catalan numbers, where item (o) is for noncrossing matchings, and item (ww) can be viewed as nonnesting matchings, in which the blocks of the matching are the columns of the standard Young tableaux of shape (n, n). Nonnesting matchings are also one of the items of [26]. Let k ≥ 2 be an integer. A k-crossing of a matching M is a set of k arcs (ir1 , jr1 ), (ir2 , jr2 ), . . . , (irk , jrk ) of M such that ir1 < ir2 < · · · < irk < jr1 < jr2 < · · · < jrk . A matching without any k-crossing is a k-noncrossing matching. Similarly, a knesting is a set of k arcs (ir1 , jr1 ), (ir2 , jr2 ), . . . , (irk , jrk ) of M such that ir1 < ir2 Received by the editors April 19, 2005 and, in revised form, November 9, 2005. 2000 Mathematics Subject Classification. Primary 05A18; Secondary 05E10, 05A15. Key words and phrases. Crossing, nesting, partition, vacillating tableau. The first author was supported by the 973 Project on Mathematical Mechanization, the National Science Foundation, the Ministry of Education and the Ministry of Science and Technology of China. The fourth author was supported in part by NSF grant #DMS-9988459. The fifth author was supported in part by NSF grant #DMS-0245526 and a Sloan Fellowship. c 1997 American Mathematical Society 1 2 W. CHEN, E. DENG, R. DU, R. STANLEY, AND C. YAN < · · · < irk < jrk < · · · < jr2 < jr1 . A matching without any k-nesting is a k-nonnesting matching. Enumeration on crossings/nestings of matchings has been studied for the cases k = 2 and k = 3. For k = 2, in addition to the above results on Catalan numbers, the distribution of the number of 2-crossings has been studied by Touchard [29], and later more explicitly by Riordan [21], who gave a generating function. M. de SainteCatherine [8] proved that 2-crossings and 2-nestings are identically distributed over all matchings of [2n], i.e., the number of matchings with r 2-crossings is equal to the number of matchings with r 2-nestings. The enumeration of 3-nonnesting matchings was first studied in [11] by Gouyou -Beauschamps, where he gave a bijection between involutions with no decreasing sequence of length 6 and pairs of noncrossing Dyck left factors by a recursive construction. His bijection is essentially a correspondence between 3-nonnesting matchings and pairs of noncrossing Dyck paths, where a matching can also be considered as a fixed-point-free involution. We observed that the number of 3-noncrossing matchings also equals the number of pairs of noncrossing Dyck paths, and a one-to-one correspondence between 3-noncrossing matchings and pairs of noncrossing Dyck paths can be built recursively. In this paper, we extend the above results. Let cr(M ) be maximal i such that M has an i-crossing, and ne(M ) the maximal j such that M has a j-nesting. Denoted by fn (i, j) the number of matchings M on [2n] with cr(M ) = i and ne(M ) = j. We shall prove that fn (i, j) = fn (j, i). As a corollary, the number of matchings on [2n] with cr(M ) = k equals the number of matchings M on [2n] with ne(M ) = k. Our construction applies to a more general structure, viz., partitions of a set. Given a partition P of [n], denoted by P ∈ Πn , we represent P by a graph on the vertex set [n] whose edge set consists of arcs connecting the elements of each block in numerical order. Such an edge set is called the standard representation of the partition P . For example, the standard representation of 1457-26-3 is {(1, 4), (4, 5), (5, 7), (2, 6)}. Here we always write an arc e as a pair (i, j) with i < j, and say that i is the lefthand endpoint of e and j is the righthand endpoint of e. Let k ≥ 2 and P ∈ Πn . Define a k-crossing of P as a k-subset (i1 , j1 ), (i2 , j2 ), . . . , (ik , jk ) of the arcs in the standard representation of P such that i1 < i2 < · · · < ik < j1 < j2 < · · · < jk . Let cr(P ) be the maximal k such that P has a k-crossing. Similarly, define a k-nesting of P as a k-subset (i1 , j1 ), (i2 , j2 ), . . . , (ik , jk ) of the set of arcs in the standard representation of P such that i1 < i2 < · · · < ik < jk < · · · < j2 < j1 , and ne(P ) the maximal j such that P has a j-nesting. Note that when restricted to complete matchings, these definitions agree with the ones given before. Let gn (i, j) be the number of partitions P of [n] with cr(P ) = i and ne(P ) = j. We shall prove that gn (i, j) = gn (j, i), for all i, j and n. In fact, our result is much stronger. We present a generalization which implies the symmetric distribution of cr(P ) and ne(P ) over all partitions in Πn , as well over all complete matchings on [2n]. To state the main result, we need some notation. Given P ∈ Πn , define min(P ) = {minimal block elements of P }, max(P ) = {maximal block elements of P }. CROSSINGS AND NESTINGS 3 For example, for P = 135-26-4, min(P ) = {1, 2, 4} and max(P ) = {4, 5, 6}. The pair (min(P ), max(P )) encodes some useful information about the partition P . For example, the number of blocks of P is | min(P )| = | max(P )|; number of singleton blocks is | min(P ) ∩ max(P )|; P is a (partial) matching if and only if min(P ) ∪ max(P ) = [n], and P is a complete matching if in addition, min(P ) ∩ max(P ) = ∅. Fix S, T ⊆ [n] with |S| = |T |. Let Pn (S, T ) be the set {P ∈ Πn : min(P ) = S, max(P ) = T }, and fn,S,T (i, j) be the cardinality of the set {P ∈ Pn (S, T ) : cr(P ) = i, ne(P ) = j}. Theorem 1.1. (1.1) fn,S,T (i, j) = fn,S,T (j, i). In other words, (1.2) X P ∈Pn (S,T ) X xcr(P ) y ne(P ) = xne(P ) y cr(P ) . P ∈Pn (S,T ) That is, the statistics cr(P ) and ne(P ) have a symmetric joint distribution over each set Pn (S, T ). Summing over all pairs (S, T ) in (1.1), we get (1.3) gn (i, j) = gn (j, i). We say that a partition P is k-noncrossing if cr(P ) < k. It is k-nonnesting if ne(P ) < k. Let NCNk,l (n) be the number of partitions of [n] that are k-noncrossing and l-nonnesting. Summing over 1 ≤ i < k and 1 ≤ j < l in (1.3), we get the following corollary. Corollary 1.2. NCNk,l (n) = NCNl,k (n). Letting l > n, Corollary 1.2 becomes the following result. Corollary 1.3. NCk (n) = NNk (n), where NCk (n) is the number of k-noncrossing partitions of [n], and NNk (n) is the number of k-nonnesting partitions of [n]. Theorem 1.1 also applies to complete matchings. A partition P of [2n] is a complete matching if and only if | min(P )| = | max(P )| = n and min(P )∩max(P ) = ∅. (It follows that min(P ) ∪ max(P ) = [2n].) Restricting Theorem 1.1 to disjoint pairs (S, T ) of [2n] with |S| = |T | = n, we get the following result on the crossing and nesting number of complete matchings. Corollary 1.4. Let M be a matching on [2n]. 1. The statistics cr(M ) and ne(M ) have a symmetric joint distribution over P2n (S, T ), where |S| = |T | = n, and S, T are disjoint. 2. fn (i, j) = fn (j, i) where fn (i, j) is the number of matchings on [2n] with cr(M ) = i and ne(M ) = j. 3. The number of matchings on [2n] that are k-noncrossing and l-nonnesting is equal to the number of matchings on [2n] that are l-noncrossing and k-nonnesting. 4. The number of k-noncrossing matchings on [2n] is equal to the number of knonnesting matchings on [2n]. The paper is arranged as follows. In Section 2 we introduce the concept of vacillating tableau of general shape, and give a bijective proof for the number of vacillating tableaux of shape λ and length 2n. In Section 3 we apply the bijection 4 W. CHEN, E. DENG, R. DU, R. STANLEY, AND C. YAN of Section 2 to vacillating tableaux of empty shape, and characterize crossings and nestings of a partition by the corresponding vacillating tableau. The involution on the set of vacillating tableaux defined by taking the conjugate to each shape leads to an involution on partitions which exchanges the statistics cr(P ) and ne(P ) while preserves min(P ) and max(P ), thus proving Theorem 1.1. Then we modify the bijection between partitions and vacillating tableaux by taking isolated points into consideration, and give an analogous result on the enhanced crossing number and nesting number. This is the content of Section 4. Finally in Section 5 we restrict our bijection to the set of complete matchings and oscillating tableaux, and study the enumeration of k-noncrossing matchings. In particular, we construct bijections from k-noncrossing matchings for k = 2 or 3 to Dyck paths and pairs of noncrossing Dyck paths, respectively, and present the generating function for the number of k-noncrossing matchings. 2. A Bijection between Set Partitions and Vacillating Tableaux Let Y be Young’s lattice, that is, the set of all partitions of all integers n ∈ N ordered component-wise, i.e., P (µ1 , µ2 , . . . ) ≤ (λ1 , λ2 , . . . , ) if µi ≤ λi for all i. We write λ ` k or |λ| = k if λi = k. A vacillating tableau is a walk on the Hasse diagram of Young’s lattice subject to certain conditions. The main tool in our proof of Theorem 1.1 is a bijection between the set of set partitions and the set of vacillating tableaux of empty shape ∅. Definition 2.1. A vacillating tableau Vλ2n of shape λ and length 2n is a sequence λ0 , λ1 , . . . , λ2n of integer partitions such that (i) λ0 = ∅, and λ2n = λ, (ii) λ2i+1 is obtained from λ2i by doing nothing (i.e., λ2i+1 = λ2i ) or deleting a square, and (iii) λ2i is obtained from λ2i−1 by doing nothing or adding a square. In other words, a vacillating tableau of shape λ is a walk on the Hasse diagram of Young’s lattice from ∅ to λ where each step consists of either (i) doing nothing twice, (ii) do nothing then adding a square, (iii) removing a square then doing nothing, or (iv) removing a square and then adding a square. Note that if the length is larger than 0, λ1 = ∅. If the vacillating tableau is of empty shape, then λ2n−1 = ∅ as well. Example 2.2. Abbreviate λ = (λ1 , λ2 , . . . ) by λ1 λ2 · · · . There are 5 vacillating tableaux of shape ∅ and length 6. They are (2.1) λ0 ∅ ∅ ∅ ∅ ∅ λ1 ∅ ∅ ∅ ∅ ∅ λ2 ∅ ∅ 1 1 1 λ3 ∅ ∅ ∅ ∅ 1 λ4 ∅ 1 1 ∅ 1 λ5 ∅ ∅ ∅ ∅ ∅ λ6 ∅ ∅ ∅ ∅ ∅ Example 2.3. An example of a vacillating tableau of shape 11 and length 10 is given by ∅, ∅, 1, 1, 1, 1, 2, 2, 21, 11, 11. Theorem 2.4. (i) Let gλ (n) be the number of vacillating tableaux of shape λ ` k and length 2n. By a standard Young tableau (SYT) of shape λ, we mean an array T of shape λ whose entries are distinct positive integers that increase in every row CROSSINGS AND NESTINGS 5 and column. The content of T is the set of positive integers that appear in it. (We don’t require that content(λ) = [k], where λ ` k.) We then have gλ (n) = B(n, k)f λ , where f λ is the number of SYT’s of shape λ and content [k], and B(n, k) is the number of partitions of [n] with k blocks distinguished. (ii) The exponential generating function of B(n, k) is given by (2.2) X n≥0 B(n, k) xn 1 = (ex − 1)k exp(ex − 1). n! k! To prove Theorem 2.4, we construct a bijection between the set Vλ2n of vacillating tableaux of shape λ and length 2n, and pairs (P, T ), where P is a partition of [n], and T is an SYT of shape λ such that content(T ) ⊆ max(P ). In the next section we apply this bijection to vacillating tableaux of empty shape, and related it to the enumeration of crossing and nesting numbers of a partition. In the following we shall assume familiarity with the RSK algorithm, and use row-insertion P ←− k as the basic operation of the RSK algorithm. For the notation, as well as some basic properties of the RSK algorithm, see e.g. [25, Chapter 7]. In general we shall apply the RSK algorithm to a sequence w of distinct integers, denoted by RSK w 7−→ (A(w), B(w)), where A(w) is the (row)-insertion tableau and B(w) the recording tableau. The shape of the SYT’s A(w) and B(w) is also called the shape of the sequence w. The Bijection ψ from vacillating tableaux to pairs (P, T ). Given a vacillating tableau V = (∅ = λ0 , λ1 , . . . , λ2n = λ), we will recursively define a sequence (P0 , T0 ), (P1 , T1 ),. . . , (P2n , T2n ) where Pi is a set of ordered pairs of integers in [n], and Ti is an SYT of shape λi . Let P0 be the empty set, and let T0 be the empty SYT (on the empty alphabet). (1) If λi = λi−1 , then (Pi , Ti ) = (Pi−1 , Ti−1 ). (2) If λi ⊃ λi−1 , then i = 2k for some integer k ∈ [n]. In this case let Pi = Pi−1 and Ti is obtained from Ti−1 by adding the entry k in the square λi \ λi−1 . (3) If λi ⊂ λi−1 , then i = 2k − 1 for some integer k ∈ [n]. In this case let Ti be the unique SYT (on a suitable alphabet) of shape λi such that Ti−1 is obtained from Ti by row-inserting some number j. Note that j must be less than k. Let Pi be obtained from Pi−1 by adding the ordered pair (j, k). It is clear from the above construction that (i) P0 ⊆ P1 ⊆ · · · ⊆ P2n , (ii) for each integer i, it appears at most once as the first component of an ordered pair in P2n , and appears at most once as the second component of an ordered pair in P2n . Let ψ(V ) = (P, T2n ), where P is the partition on [n] whose standard representation is P2n . Note that if an integer i appears in T2n , then P2n can not contain any ordered pair (i, j) with i < j. It follows that i is the maximal element in the block containing it. Hence the content of T2n is a subset of max(P ). Example 2.5. As an example of the map ψ, let the vacillating tableau be ∅, ∅, 1, 1, 2, 2, 2, 2, 21, 21, 211, 21, 21, 11, 21. 6 W. CHEN, E. DENG, R. DU, R. STANLEY, AND C. YAN Then the pairs (Bi , Ti ) (where Bi is the pair added to Pi−1 to obtain Pi ) are given by 0 ∅ i Ti Bi 1 ∅ 2 3 4 5 6 7 8 9 10 11 12 13 1 1 12 12 12 12 12 12 12 14 14 1 4 4 4 5 5 5 5 (2, 6) (4, 7) 14 17 5 Hence 7 T = 1 5 , P = 1-26-3-47-5. The map ψ is bijective since the above construction can be reversed. Given a pair (P, T ), where P is a partition of [n] and T is an SYT whose content consists of maximal elements of some blocks of P , let E(P ) be the standard representation of P , and T2n = T . We work our way backwards from T2n , reconstructing the preceding tableaux and hence the sequence of shapes. If we have the SYT T2k for some k ≤ n, we can get the tableaux T2k−1 , T2k−2 by the following rules. (1) T2k−1 = T2k if the integer k does not appear in T2k . Otherwise T2k−1 is obtained from T2k by deleting the square containing k. (2) T2k−2 = T2k−1 if E(P ) does not have an edge of the form (i, k). Otherwise there is a unique i < k such that (i, k) ∈ E(P ). In that case let T2k−2 be obtained from T2k−1 by row-inserting i, or equivalently, T2k−2 = (T2k−1 ←− i). Proof of Theorem 2.4. Part (i) follows from the bijection ψ, where a block of P is distinguished if its maximal element belongs to content(T ). For part (ii), simply note that to get a structure counted by B(n, k), we can partition [n] into two subsets, S and T , and then partition S into k blocks and put a mark on each block, and partition T arbitrarily. The generating function of B(n, k) then follows from the well-known generating functions for S(n, k), the Stirling number of the second kind, and for the Bell number B(n), X X xn 1 xn S(n, k) = (ex − 1)k , B(n) = exp(ex − 1). n! k! n! n≥0 n≥k Remark 2.6. (1). Restricting to vacillating tableaux of empty shape, the map ψ provides a bijection between the set V∅2n of vacillating tableaux of empty shape and length 2n and the set of partitions of [n]. In particular, g∅ (n), the cardinality of V∅2n , is equal to the nth Bell number B(n). (2). Note that there is a symmetry between the four types of movements in the definition of vacillating tableaux. Thus any walk from ∅ to ∅ in m + n steps can be viewed as a walk from ∅ to some shape λ in n steps, then followed by the reverse of a walk from ∅ to λ in m steps. It follows that X (2.3) gλ (n)gλ (m) = g∅ (m + n) = B(m + n). λ For the case m = n = k, the identity (2.3) is proved by Halverson and Lowandowski [16], who gave a bijective proof using similar procedures as those in ψ. (3). The partition algebra Pn is a certain semisimple algebra, say over C, whose dimension is the Bell number B(n) (the number of partitions of [n]). (The algebra Pn depends on a parameter x which is irrelevant here.) See [15, 16] for a survey CROSSINGS AND NESTINGS 7 of this topic. Vacillating tableaux are related to irreducible representations of Pn in the same way that SYT of content [n] are related to irreducible representations of the symmetric group Sn . In particular, the irreducible representations Iλ of Pn are indexed by partitions λ for which there exists a vacillating tableau of shape λ and length 2n, and dim In is the number of such vacillating tableaux. This result is equivalent to [15, Thm. 2.24(b)], but that paper does not explicitly define the notion of vacillating tableau. Combinatorial identities arising from partition algebra and its subalgebras are discussed in [16], where the authors used the notion of vacillating tableau after the distribution of a preliminary version of this paper. 3. Crossings and Nestings of Partitions In this section we restrict the map ψ to vacillating tableaux of empty shape, for which ψ provides a bijection between the set of vacillating tableaux of empty shape and length 2n and the set of partitions of [n]. To make the bijection clear, we restate the inverse map from the set of partitions to vacillating tableaux. The Map φ from partitions to vacillating tableaux. Given a partition P ∈ Πn with the standard representation, we construct the sequence of SYT’s, hence the vacillating tableau φ(P ) as follows: Start from the empty SYT by letting T2n = ∅, read the number j ∈ [n] one by one from n to 1, and define T2j−1 , T2j−2 for each j. There are four cases. 1. If j is the righthand endpoint of an arc (i, j), but not a lefthand endpoint, first do nothing, then insert i (by the RSK algorithm) into the tableau. 2. If j is the lefthand endpoint of an arc (j, k), but not a righthand endpoint, first remove j, then do nothing. 3. If j is an isolated point, do nothing twice. 4. If j is the righthand endpoint of an arc (i, j), and the lefthand endpoint of another arc (j, k), then delete j first, and then insert i. The vacillating tableau φ(P ) is the sequences of shapes of the above SYT’s. Example 3.1. Let P be the partition 1457-26-3 of [7]. t t t t t t t 1 2 3 4 5 6 7 Figure 1. The standard representation of the partition 1457-26-3. Starting from ∅ on the right, go from 7 to 1, the seven steps are (1) do nothing, then insert 5, (2) do nothing, then insert 2, (3) delete 5 and insert 4, (4) delete 4 and insert 1, (5) do nothing twice, (6) remove 2 then do nothing, (7) remove 1 then do nothing. Hence the corresponding SYT’s, constructed from right to left, are ∅ ∅ 1 1 1 1 1 2 2 2 2 24 2 2 5 5 5 ∅ ∅ The vacillating tableau is ∅, ∅, 1, 1, 11, 11, 11, 1, 2, 1, 11, 1, 1, ∅, ∅. The relation between cr(P ), ne(P ) and the vacillating tableau is given in the next theorem. 8 W. CHEN, E. DENG, R. DU, R. STANLEY, AND C. YAN Theorem 3.2. Let P ∈ Πn and φ(P ) = (∅ = λ0 , λ1 , . . . , λ2n = ∅). Then cr(P ) is the most number of rows in any λi , and ne(P ) is the most number of columns in any λi . Proof. We prove Theorem 3.2 in four steps. First, we interpret a k-crossing/knesting of P in terms of entries of SYT’s Ti in φ(P ). Then, we associate to each SYT Ti a sequence σi whose terms are entries of Ti . We prove that Ti is the insertion tableau of σi under the RSK algorithm, and apply Schensted’s theorem to conclude the proof. Step 1. Let T (P ) = (T0 , T1 , . . . , T2n ) be the sequence of SYT’s associated to the vacillating tableau φ(P ). By the construction of ψ and φ, a pair (i, j) is an arc in the standard representation of P if and only if i is an entry in the SYT’s T2i , T2i+1 , . . . , T2j−2 . We say that the the integer i is added to T (P ) at step i and leaves at step j. First we prove that the arcs (i1 , j1 ), . . . , (ik , jk ) form a k-crossing of P if and only if there exists a tableau Ti in T (P ) such that the integers i1 , i2 , . . . , ik ∈ content(Ti ), and i1 , i2 , . . . , ik leave T (P ) in increasing order according to their numerical values. Given a k-crossing ((i1 , j1 ), . . . , (ik , jk )) of P , where ir < jr for 1 ≤ r ≤ k and i1 < i2 < · · · < ik < j1 < j2 < · · · < jk , the integer ir is added to T (P ) at step ir and leaves at step jr . Hence all ir are in T2j1 −2 , and they leave T (P ) in increasing order. The converse is also true: if there are k integers i1 < i2 < · · · < ik all appearing in the same tableau at some steps, and then leave in increasing order, say at steps j1 < j2 < · · · < jk , then ik < j1 and the pairs ((i1 , j1 ), . . . , (ik , jk )) ∈ P form a k-crossing. By a similar argument arcs (i1 , j1 ), . . . , (ik , jk ) form a knesting of P if and only if there exists a tableaux Ti in T (P ) such that the integers i1 , i2 , . . . , ik ∈ content(Ti ), and i1 , i2 , . . . , ik leave T (P ) in decreasing order. Step 2. For each Ti ∈ T (P ), we define a permutation σi of content(Ti ) (backward) recursively as follows. Let σ2n be the empty sequence. (1) If Ti = Ti−1 , then σi−1 = σi . (2) If Ti−1 is obtained from Ti by row-inserting some number j, then σi−1 = σi j, the juxtaposition of σi and j. (3) If Ti is obtained from Ti−1 by adding the entry i/2, (where i must be even), then σi−1 is obtained from σi by deleting the number i/2. Note that in the last case, i/2 must be the largest entry in σi . Clearly σi is a permutation of the entries in content(Ti ). If σi = w1 w2 · · · wr , then the entries of content(Ti ) leave T (P ) in the order wr , . . . , w2 , w1 . RSK Step 3. Claim: If σi 7−→ (Ai , Bi ), then Ai = Ti . We prove the claim by backward induction. The case i = 2n is trivial as both A2n and T2n are the empty SYT. Assume the claim is true for some i, 1 ≤ i ≤ 2n. We prove that the claim holds for i − 1. If Ti−1 = Ti , the claim holds by the inductive hypothesis. If Ti−1 is obtained from Ti by inserting the number j, then the claim holds by the definition of the RSK algorithm. It is only left to consider the case that Ti−1 is obtained from Ti by removing the entry j = i/2. Let us write σi as u1 u2 · · · us jv1 · · · vt , and σi−1 as u1 u2 · · · us v1 · · · vt , where j > u1 , . . . , us , v1 , . . . , vt . We need to show that the insertion tableau of σi−1 is the same as the insertion tableau of σi deleting the entry j, i.e., Ai−1 = Ai \{j}. Proof by induction on t. If t = 0 then it is true by the RSK algorithm that Ai is obtained from Ai−1 by adding j at the end of the first row. Assume it is true for t − 1, i.e., A(u1 · · · us v1 · · · vt−1 ) = A(u1 · · · us jv1 · · · vt−1 ) \ {j}. Note that in A(u1 · · · us jv1 · · · vt−1 ), if j is in position (x, y), then there is no element in CROSSINGS AND NESTINGS 9 positions (x, y + 1) or (x + 1, y). Now we insert the entry vt by the RSK algorithm. Consider the insertion path I = I(A(u1 · · · us jv1 · · · vt−1 ) ←− vt ). If j does not appear on this path, then we would have the exact same insertion path when inserting vt into A(u1 · · · us v1 · · · vt−1 ). This insertion path results in the same change to A(u1 · · · us jv1 · · · vt−1 ) and A(u1 · · · us v1 · · · vt−1 ), which does not touch the position (x, y) of j. So A(u1 · · · us v1 · · · vt−1 vt ) = A(u1 · · · us jv1 · · · vt−1 vt )\{j}. On the other hand, if j appears in the insertion path I, i.e., (x, y) ∈ I, then since j is the largest element, it must be bumped into the (x + 1)-th row, and become the last entry in the (x + 1)-th row without bumping any number further. Then the insertion path of vt into A(u1 · · · us v1 · · · vt−1 ) is I minus the last position {(x + 1, ∗)}, and again we have A(u1 · · · us v1 · · · vt−1 vt ) = A(u1 · · · us jv1 · · · vt−1 vt ) \ {j}. This finishes the proof of the claim. Step 4. We shall need the following theorem of Schensted [22][25, Thms. 7.23.13, 7.23.17], which gives the basic connection between the RSK algorithm and the increasing and decreasing subsequences. Schensted’s Theorem Let σ be a sequence of integers whose RSK terms are distinct. Assume σ 7−→ (A, B), where A and B are SYT’s of the shape λ. Then the length of the longest increasing subsequence of σ is λ1 (the number of columns of λ), and the length of the longest decreasing subsequence is λ01 (the number of rows of λ). Now we are ready to prove Theorem 3.2. By Steps 1 and 2, a partition P has a k-crossing if and only if there exists i such that σi has a decreasing subsequence of length k. The claim in Step 3 implies that the shape of the sequence σi is exactly the diagram of the i-th partition λi in the vacillating tableau φ(P ). By Schensted’s Theorem, σi has a decreasing subsequence of length k if and only if the partition λi in φ(P ) has at least k rows. This proves the statement for cr(P ) in Theorem 3.2. The statement for ne(P ) is proved similarly. The symmetric joint distribution of statistics cr(P ) and ne(P ) over Pn (S, T ) follows immediately from Theorem 3.2. Proof of Theorem 1. From Theorem 3.2, a partition P ∈ Πn has cr(P ) = k and ne(P ) = j if and only if for the partitions {λi }2n i=0 of the vacillating tableau φ(P ), the maximal number of rows of the diagram of any λi is k, and the maximal number of columns of the diagram of any λi is j. Let τ be the involution defined on the set V∅2n by taking the conjugate to each partition λi . For i ∈ [n], i ∈ min(P ) (resp. max(P )) if and only if λ2i−1 = λ2i−2 and λ2i \ λ2i−1 = , (reps. λ2i−2 \ λ2i−1 = and λ2i = λ2i−1 ). Since τ preserves min(P ) and max(P ), it induces an involution on Pn (S, T ) which exchanges the statistics cr(P ) and ne(P ). This proves Theorem 1.1. Let λ = (λ1 , λ2 , . . . ) be the shape of a sequence w of distinct integers. Schensted’s Theorem provides a combinatorial interpretation of the terms λ1 and λ01 : they are the length of the longest increasing and decreasing subsequences of w. In [14] C. Greene extended Schensted’s Theorem by giving an interpretation of the rest of the diagram of λ = (λ1 , λ2 , . . . ). Assume w is a sequence of length n. For each k ≤ n, let dk (w) denote the length of the longest subsequence of w which has no increasing subsequences of length k + 1. It can be shown easily that any such sequence is obtained by taking the 10 W. CHEN, E. DENG, R. DU, R. STANLEY, AND C. YAN union of k decreasing subsequences. Similarly, define ak (w) to be the length of the longest subsequence consisting of k ascending subsequences. Theorem 3.3 (Greene). For each k ≤ n, ak (w) = λ1 + λ2 + · · · + λk , dk (w) = λ01 + λ02 + · · · + λ0k , where λ0 = (λ01 , λ02 , . . . ) is the conjugate of λ. We may consider the analogue of Greene’s Theorem for set partitions. Let P ∈ Πn with the standard representation {(i1 , j1 ), (i2 , j2 ), . . . , (it , jt )}, where ir < jr for 1 ≤ r ≤ t. Let er = (ir , jr ). We define the crossing graph Cr(P ) of P as follows. The vertex set of Cr(P ) is {e1 , e2 , . . . , et }. Two arcs er and es are adjacent if and only if the edges er and es are crossing, that is, ir < is < jr < js . Clearly a k-crossing of P corresponds to a k-clique of Cr(P ). Let crr (P ) be the maximal number of vertices in a union of r cliques of Cr(P ). In other words, crr (P ) is the maximal number of arcs in a union of r crossings of P . Similarly, let Ne(P ) be the graph defined on the vertex set {e1 , . . . , et } where two arcs er and es are adjacent if and only if ir < is < js < jr . Let ner (P ) be the maximal number of vertices in a union of r cliques of Ne(P ). In other words, ner (P ) is the maximal number of arcs in a union of r nestings of P . Proposition 3.4. Let P = ((i1 , j1 ), (i2 , j2 ), . . . , (it , jt )) be the standard representation of a partition of [n], where ir < jr for all 1 ≤ r ≤ t and j1 < j2 < · · · < jt . Let α(P ) be the sequence i1 i2 · · · it . Then there is a one-to-one correspondence between the set of nestings of P and the set of decreasing subsequences of α(P ). Proof. Let φ(P ) be the vacillating tableau corresponding to P , and T (P ) the sequence of SYT’s constructed in the bijection. Then α(P ) records the order in which the entries of Ti ’s leave T (P ). Let σ = it · · · i2 i1 be the reverse of α(P ), and {σi : 1 ≤ i ≤ 2n} the permutation of content(Ti ) defined in Step 2 of the proof of Theorem 3.2. Then the σi ’s are subsequences of σ. From Steps 1 and 2 of the proof of Theorem 3.2, nestings of P are represented by the increasing subsequences of σi , 1 ≤ i ≤ 2n, and hence by the increasing subsequences of σ. Conversely, let ir1 < ir2 < · · · < irt be an increasing subsequence of σ. Being a subsequence of σ means that its terms leave T (P ) in reverse order, so irt leaves first in step jrt . Thus all the entries ir1 , . . . , irt appear in the SYT’s Ti with 2irt ≤ i ≤ 2jrt − 2. Therefore ir1 ir2 · · · irt is also an increasing subsequence of σi , for 2irt ≤ i ≤ 2jrt − 2. Combining Proposition 3.4 and Greene’s Theorem, we have the following corollary describing net (P ). Corollary 3.5. Let P and α(P ) be as in Proposition 3.4. Then ner (P ) = λ01 + λ02 + · · · + λ0r , where λ is the shape of α(P ), and λ0 is the conjugate of λ. The situation for crr (P ) is more complicated. We don’t have a result similar to Proposition 3.4. Any crossing of P uniquely corresponds to an increasing subsequence of α(P ). But the converse is not true. An increasing subsequence ir1 ir2 · · · irt corresponds to a t-crossing of P only if we have the additional condition irt < jr1 . It would be interesting to get a result for crr (P ) analogous to Corollary 3.5. CROSSINGS AND NESTINGS 11 To conclude this section we discuss the enumeration of noncrossing partitions. The following theorem is a direct corollary of the bijection between vacillating tableaux and partitions. Theorem 3.6. Let i denote the ith unit coordinate vector in Rk−1 . The number of k-noncrossing partitions of [n] equals the number of closed lattice walks in the region Vk = {(a1 , a2 , . . . , ak−1 ) : a1 ≥ a2 ≥ · · · ≥ ak−1 ≥ 0, ai ∈ Z} from the origin to itself of length 2n with steps ±i or (0, 0, . . . , 0), with the property that the walk goes backwards (i.e., with step −i ) or stands still (i.e., with step (0, 0, . . . , 0)) after an even number of steps, and goes forwards (i.e., with step +i ) or stands still after an odd number of steps. Recall that a partition P ∈ Πn is k-noncrossing if cr(P ) < k, and is k-nonnesting if ne(P ) < k. A partition P has no k-crossings and no j-nestings if and only if for all the partitions λi of φ(P ), the diagram fits into a (k − 1) × (j − 1) rectangle. Taking the conjugate of each partition, we get bijective proofs of Corollaries 1.2 and 1.3. Theorem 1.1 asserts the symmetric distribution of cr(P ) and ne(P ) over Pn (S, T ), for all S, T ⊆ [n] with |S| = |T |. Not every Pn (S, T ) is nonempty. A set Pn (S, T ) is nonempty if and only if for all i ∈ [n], |S ∩ [i]| ≥ |T ∩ [i]|. Another way to describe the nonempty Pn (S, T ) is to use lattice paths. Associate to each pair (S, T ) a lattice path L(S, T ) with steps (1, 1), (1, −1) and (1, 0): start from (0, 0), read the integers i from 1 to n one by one, and move two steps for each i. 1. If i ∈ S ∩ T , move (1, 0) twice. 2. If i ∈ S \ T , move (1, 0) then (1, 1). 3. If i ∈ T \ S, move (1, −1) then (1, 0). 4. If i ∈ / S ∪ T , move (1, −1) then (1, 1). This defines a lattice path L(S, T ) from (0, 0) to (2n, 0), Conversely, the path uniquely determines (S, T ). Then Pn (S, T ) is nonempty if and only if the lattice path L(S, T ) is a Motzkin path, i.e., never goes below the x-axis. There are existing notions of noncrossing partitions and nonnesting partitions, e.g., [25, Ex.6.19]. A noncrossing partition of [n] is a partition of [n] in which no two blocks “cross” each other, i.e., if a < b < c < d and a, c belong to a block B and b, d to another block B 0 , then B = B 0 . A nonnesting partition of [n] is a partition of [n] such that if a, e appear in a block B and b, d appear in a different block B 0 where a < b < d < e, then there is a c ∈ B satisfying b < c < d. It is easy to see that P is a noncrossing partition if and only if the standard representation of P has no 2-crossing, and P is a nonnesting partition if and only if the standard representation of P has no 2-nesting. Hence the vacillating tableau correspondence, in the case of 1-row/column tableaux, gives bijections between noncrossing partitions of [n], nonnesting partitions of [n], (both are counted by Catalan numbers) and sequences 0 = a0 , a1 , ..., a2n = 0 of nonnegative integers such that a2i+1 = a2i or a2i − 1, and a2i = a2i−1 or a2i−1 + 1. These sequences a0 , ..., a2n give a new combinatorial interpretation of Catalan numbers. Replacing a term ai+1 = ai + 1 with a step (1, 1), a term ai+1 = ai − 1 with a step (1, −1), and a term ai+1 = ai with a step (1, 0), we get a Motzkin path, so we also have a bijection between noncrossing/nonnesting partitions and certain Motzkin paths. The Motzkin paths are exactly the ones defined as L(S, T ), where 12 W. CHEN, E. DENG, R. DU, R. STANLEY, AND C. YAN S = min(P ) and T = max(P ). Conversely, given a Motzkin path of the form L(S, T ), we can recover uniquely a noncrossing partition and a nonnesting partition. Write the path as {(i, ai ) : 0 ≤ i ≤ 2n}. Let A = [n] \ T and B = [n] \ S. Clearly |A| = |B|. Assume A = {i1 , i2 , . . . , it }< , and B = {j1 , j2 , . . . , jt }< where elements are listed in increasing order. Then to get the standard representation of the noncrossing partition, pair each jr with max{is ∈ A : is < jr , a2is = a2jr −2 }. To get the standard representation of the nonnesting partition, pair each jr with ir , for 1 ≤ r ≤ t. Remark 3.7. In our definition, a k-crossing is defined as a set of k mutually crossing arcs in the standard representation of the partition. There exist some other definitions. For example, in [17] M. Klazar defined 3-noncrossing partition as a partition P which does not have 3 mutually crossing blocks. It can be seen that P is 3-noncrossing in Klazar’s sense if and only if there do not exist 6 elements a1 < b1 < c1 < a2 < b2 < c2 in [n] such that a1 , a2 ∈ A, b1 , b2 ∈ B, c1 , c2 ∈ C, and A, B, C are three distinct blocks of P . Klazar’s definition of 3-noncrossing partitions is different from ours. For example, let P be the partition 15-246-37 of [7], with standard representation as follows: r r r r r r r 1 2 3 4 5 6 7 According to Klazar’s definition, P has a 3-crossing, since we have 1 < 2 < 3 < 5 < 6 < 7 and {1, 5}, {2, 6} and {3, 7} belong to three different blocks, respectively. On the other hand, P has no 3-crossing on our sense. For general k, these three notions of k-noncrossing partitions, i.e., (1) no kcrossing in the standard representation of P , (2) no k mutually crossing arcs in distinct blocks of P , and (3) no k mutually crossing blocks, are all different, with the first being the weakest, and the third the strongest. 4. A Variant: Partitions and Hesitating Tableaux We may also consider the enhanced crossing/nesting of a partition, by taking isolated points into consideration. For a partition P of [n], let the enhanced representation of P be the union of the standard representation of P and the loops {(i, i) : i is an isolated point of P }. An enhanced k-crossing of P is a set of k edges (i1 , j1 ), (i2 , j2 ), . . . , (ik , jk ) of the enhanced representation of P such that i1 < i2 < · · · < ik ≤ j1 < j2 < · · · < jk . In particular, two arcs of the form (i, j) and (j, l) with i < j < l are viewed as crossing. Similarly, an enhanced k-nesting of P is a set of k edges (i1 , j1 ), (i2 , j2 ), . . . , (ik , jk ) of the enhanced representation of P such that i1 < i2 < · · · < ik ≤ jk < · · · < j2 < j1 . In particular, an edge (i, k) and an isolated point j with i < j < k form an enhanced 2-nesting. Let cr(P ) be the size of the largest enhanced crossing, and ne(P ) the size of the largest enhanced nesting. Using a variant form of vacillating tableau, we again obtain a symmetric joint distribution of the statistics cr(P ) and ne(P ). The variant tableau is a hesitating tableau of shape ∅ and length 2n, which is a path on the Hasse diagram of Young’s lattice from ∅ to ∅ where each step consists of a pair of moves, where the pair is either (i) doing nothing then adding a square, (ii) removing a square then doing nothing, or (iii) adding a square and then removing a square. CROSSINGS AND NESTINGS 13 Example 4.1. There are 5 hesitating tableaux of shape ∅ and length 6. They are (4.1) ∅ 1 ∅ 1 ∅ ∅ ∅ ∅ ∅ ∅ ∅ 1 ∅ ∅ 1 11 1 ∅ 1 2 ∅ 1 1 ∅ 1 1 ∅ ∅ ∅ ∅ ∅ 1 ∅ ∅ ∅ To see the equivalence with vacillating tableaux, let U be the operator that takes a shape to the sum of all shapes that cover it in Young’s lattice (i.e., by adding a square), and similarly D takes a shape to the sum of all shapes that it covers in Young’s lattice (i.e., by deleting a square). Then, as is well-known, DU − U D = I (the identity operator). See, e.g. [23][25, Exer. 7.24]. It follows that (4.2) (U + I)(D + I) = DU + ID + U I. Iterating the left-hand side generates vacillating tableaux, and iterating the righthand side gives the hesitating tableaux defined above. A bijective map between partitions of [n] and hesitating tableaux of empty shape has been given by Korn [18], based on growth diagrams. Here by modifying the map φ defined in Section 3 we get a more direct bijection φ̄ between partitions and hesitating tableaux of empty shape, which leads to the symmetric joint distribution of cr(P ) and ne(P ). The construction and proofs are very similar to the ones given in Sections 2 and 3, and hence are omitted here. We will only state the definition of the map φ̄ from partitions to hesitating tableaux, to be compared with the map φ in Section 3. The Bijection φ̄ from Partitions to Hesitating Tableaux. Given a partition P ∈ Πn with the enhanced representation, we construct the sequence of SYT’s, and hence the hesitating tableau φ̄(P ), as follows: Start from the empty SYT by letting T2n = ∅, read the numbers j ∈ [n] one by one from n to 1, and define two SYT’s T2j−1 , T2j−2 for each j. When j is a lefthand endpoint only, or a righthand endpoint only, the construction is identical to that of the map φ. Otherwise, 1. If j is an isolated point, first insert j, then delete j. 2. If j is the righthand endpoint of an arc (i, j), and the lefthand endpoint of another arc (j, k), then insert i first, and then delete j. Example 4.2. For the partition 1457-26-3 of [7] in Figure 2, the corresponding SYT’s are ∅ ∅ 1 1 1 13 1 14 24 24 2 5 5 ∅ ∅ 2 2 2 2 5 5 The hesitating tableau φ̄(P ) is ∅, ∅, 1, 1, 11, 21, 11, 21, 2, 21, 11, 1, 1, ∅, ∅. j t t t t t t t 1 2 3 4 5 6 7 Figure 2. The enhanced representation of the partition 1457-26-3. 14 W. CHEN, E. DENG, R. DU, R. STANLEY, AND C. YAN The conjugation of shapes does not preserve min(P ) or max(P ). Instead, it preserves min(P )\max(P ), and max(P )\min(P ). Let S, T be disjoint subsets of [n] with the same cardinality, P̄n (S, T ) = {P ∈ Πn : min(P )\max(P ) = S, max(P )\ min(P ) = T }, and f¯n,S,T (i, j) = #{P ∈ P̄n (S, T ) : cr(P ) = i, ne(P ) = j}. Theorem 4.3. We have f¯n,S,T (i, j) = f¯n,S,T (j, i). As a consequence, Corollaries 1.2 and 1.3 remain valid if we define k-noncrossing (or k-nonnesting) by cr(P ¯ ) < k (or ne(P ¯ ) < k). Remark 4.4. As done for vacillating tableaux, one can extend the definition of hesitating tableaux by considering moves from ∅ to λ, and denote by fλ (n) the number of such hesitating tableaux of length 2n. Identity (4.2) implies that fλ (n) = gλ (n), the number of vacillating tableaux from ∅ to λ. It follows that f∅ (n) = B(n), and X fλ (n)fλ (m) = B(m + n), λ where B(n) is the nth Bell number. For further discussion of the number fλ (n), see [26, Problem 33] (version of 17 August 2004). 5. Enumeration of k-Noncrossing Matchings Restricting Theorem 1.1 to disjoint subsets (S, T ) of [n], where n = 2m and |S| = |T | = m, we get the symmetric joint distribution of the crossing number and nesting number for matchings, as stated in Corollary 1.4 in Section 1. In a complete matching, an integer is either a left endpoint or a right endpoint in the standard representation. In applying the map φ to complete matchings on [2m], if we remove all steps which do nothing, we obtain a sequence ∅ = λ0 , λ1 , . . . , λ2m = ∅ of partitions such that for all 1 ≤ i ≤ 2m, the diagram of λi is obtained from that of λi−1 by either adding one square or removing one square. Such a sequence is called an oscillating tableau (or up-down tableau) of empty shape and length 2m. Thus we get a bijection between complete matchings on [2m] and oscillating tableaux of empty shape and length 2m. This bijection was originally constructed by the fourth author, and then extended by Sundaram [27] to arbitrary shapes to give a combinatorial proof of the Cauchy identity for the symplectic group Sp(2m). The explicit description of the bijection has appeared in [27] and was included in [25, Exercise 7.24]. Oscillating tableaux first appeared (though not with that name) in [5]. Recall that the ordinary RSK algorithm gives a bijection between the symmetric group Sm and pairs (P, Q) of SYTs of the same shape λ ` m. This result and the Schensted’s theorem can be viewed as a special case of what we do. Explicitly, identify an SYT T of shape λ and content [m] with a sequence ∅ = λ0 , λ1 , . . . , λm = λ of integer partitions where λi is the shape of the SYT T i obtained from T by deleting all entries {j : j > i}. Let w be a permutation of [m], and form the matching Mw on [2m] with arcs between w(i) and 2m − i + 1. We get an oscillating tableau Ow that increases to the shape λ ` m and then decreases to the empty RSK shape. Assume w 7−→ (A(w), B(w)), where A(w) is the (row)-insertion tableau and B(w) the recording tableau. Then A(w) is given by the first m steps of Ow , and B(w) the reverse of the last m steps. The size of the largest crossing (resp. CROSSINGS AND NESTINGS 15 nesting) of Mw is exactly the length of the longest decreasing (resp. increasing) subsequence of w. Example 5.1. Let w = 231. Then A(w) = 1 2 3 , B(w) = 1 3 2 . The matching Mw and the corresponding oscillating tableau are given below. r 1 ∅ r 2 1 r 3 11 r 4 21 r 5 2 r 6 1 ∅ Remark 5.2. The Brauer algebra Bm is a certain semisimple algebra, say over C, whose dimension is the number 1·3·· · · (2m−1) of matchings on [2m]. (The algebra Bm depends on a parameter x which is irrelevant here.) Oscillating tableaux of length 2m are related to irreducible representations of Bm in the same way that SYT of content [m] are related to irreducible representations of the symmetric group Sm and that vacillating tableaux of length 2m are related to irreducible representations of the partition algebra Pm . In particular, the irreducible representations Jλ of Bm are indexed by partitions λ for which there exists an oscillating tableau of shape λ and length 2m, and dim Jλ is the number of such oscillating tableaux. See e.g. [4, Appendix B6] for further information. Next we use the bijection between complete matchings and oscillating tableaux to study the enumeration of k-noncrossing matchings. All the results in the following hold for k-nonnesting matchings as well. For complete matchings, Theorem 3.6 becomes the following. Corollary 5.3. The number of k-noncrossing matchings of [2m] is equal to the number of closed lattice walks of length 2m in the set Vk = {(a1 , a2 , . . . , ak−1 ) : a1 ≥ a2 ≥ · · · ≥ ak−1 ≥ 0, ai ∈ Z} from the origin to itself with unit steps in any coordinate direction or its negative. Restricted to the cases k = 2, 3, Corollary 5.3 leads to some nice combinatorial correspondences. Recall that a Dyck path of length 2m is a lattice path in the plane from the origin (0, 0) to (2m, 0) with steps (1, 1) and (1, −1), that never passes below the x-axis. A pair (P, Q) of Dyck paths is noncrossing if they have the same origin and the same destination, and P never goes below Q. Corollary 5.4. (1) The set of 2-noncrossing matchings is in one-to-one correspondence with the set of Dyck paths. (2) The set of 3-noncrossing matchings is in one-to-one correspondence with the set of pairs of noncrossing Dyck paths. Proof. By Corollary 5.3, 2-noncrossing matchings are in one-to-one correspondence with closed lattice paths {~vi = (xi )}2m i=0 with x0 = x2m = 0, xi ≥ 0 and xi+1 − xi = ±1. Given such a 1-dimensional lattice path, define a lattice path in the plane by letting P = {(i, xi ) | i = 0, 1, . . . , 2m}. Then P is a Dyck path, and this gives the desired correspondence. For k = 3, 3-noncrossing matchings are in one-to-one correspondence with 2dimensional lattice paths {~vi = (xi , yi )}2m i=0 with (x0 , y0 ) = (x2m , y2m ) = (0, 0), 16 W. CHEN, E. DENG, R. DU, R. STANLEY, AND C. YAN xi ≥ yi ≥ 0 and (xi+1 , yi+1 )−(xi , yi ) = (±1, 0) or (0, ±1). Given such a lattice path, define two lattice paths in the plane by setting P = {(i, xi + yi ) | i = 0, 1, . . . , 2m} and Q = {(i, xi − yi ) | i = 0, 1, . . . , 2m}. Then (P, Q) is a pair of noncrossing Dyck paths. It is easy to see that this is a bijection. Example 5.5. We illustrate the bijections between 3-noncrossing matchings, oscillating tableaux, and pairs of noncrossing Dyck paths. The oscillating tableau is ∅, 1, 2, 21, 31, 21, 11, 21, 2, 1, ∅. The sequence of SYT’s as defined in the bijection is 2 1 2 4 , 1 2 , 1 , 1 7 , 37, 3, ∅. ∅, 1, 12, 1 3 , 3 3 3 3 The corresponding matching and the pair of noncrossing Dyck paths are given in Figure 3. r r r r r @ @r r @ @r @ @r r @ @r @ @r r @ @r r@ @r @r@ r @ @r @ @r r r r r r r r r r r r r 1 2 3 4 5 6 7 8 9 10 Figure 3. The matching and the pair of noncrossing Dyck paths. Let fk (m) be the number of k-noncrossing matchings of [2m]. By Corollary 5.3 it is also the number of lattice paths of length 2m in the region Vk from the origin to itself with step set {±1 , ±2 , . . . , ±k−1 }. Set Fk (x) = X fk (m) m x2m . (2m)! It turns out that a determinantal expression for Fk (x) has been given by Grabiner and Magyar [12]. It is simply the case λ = η = (m, m − 1, ..., 1) of equation (38) in [12], giving (5.1) k−1 Fk (x) = det [Ii−j (2x) − Ii+j (2x)]i,j=1 , where Im (2x) = X j≥0 xm+2j , j!(m + j)! the hyperbolic Bessel function of the first kind of order m [30]. One can easily check that when k = 2, the generating function of 2-noncrossing matchings equals F2 (x) = I0 (2x) − I2 (2x) = X j≥0 Cj x2j , (2j)! CROSSINGS AND NESTINGS 17 where Cj is the j-th Catalan number. When k = 3, we have f3 (m) 3!(2m + 2)! m!(m + 1)!(m + 2)!(m + 3)! = 2 = Cm Cm+2 − Cm+1 . This result agrees with the formula on the number of pairs of noncrossing Dyck paths due to Gouyou-Beauchamps in [11]. Remark 5.6. The determinant formula (5.1) has been studied by Baik and Rains in [2, Eqs. (2.25)]. One simply puts i − 1 for j and j − 1 for k in (2.25) of [2] to get our formula. The same formula was also obtained by Goulden [10] as the generating function for fixed point free permutations with no decreasing subsequence of length greater than 2k. See Theorem 1.1 and 2.3 of [10] and specialize hi to be xi /i!, so gl becomes the hyperbolic Bessel function. The asymptotic distribution of cr(M ) follows from another result of Baik and Rains. In Theorem 3.1 of [3] they obtained the limit distribution for the length of the longest decreasing subsequence of fixed point free involutions w. But representing w as a matching M , the condition that w has no decreasing subsequence of length 2k + 1 is equivalent to the condition that M has no k + 1-nesting, and we already know that cr(M ) and ne(M ) have the same distribution. Combining the above results, one has ! √ cr(M ) − 2m x lim Pr = F1 (x), ≤ m→∞ 2 (2m)1/6 where Z ∞ 1 u(s)ds , 2 x where F (x) is the Tracy-Widom distribution and u(x) the Painlevé II function. F1 (x) = p F (x) exp Similarly one can try to enumerate complete matchings of [2m] with no (k + 1)crossing and no (j + 1)-nesting. By the oscillating tableau bijection this is just the number of walks of length 2m from 0̂ to 0̂ in the Hasse diagram of the poset L(k, j), where 0̂ denotes the unique bottom element (the empty partition) of L(k, j), the lattice of integer partitions whose shape fits in a k × P j rectangle, ordered by 2m inclusion. Let gk,j (m) be this number, and Gk,j (x) = be the m gk,j (m)x generating function. For j = 1, the number gk,1 (m) counts lattice paths from (0, 0) to (2m, 0) with steps (1, 1) or (1, −1) that stay between the lines y = 0 and y = k. The evaluation of gk,1 (m) was first considered by Takács in [28] by a probabilistic argument. Explicit formula and generating function for this case are well-known. For example, in [19] one obtains the explicit formula by applying the reflection principle repeatedly, viz., X 2m 2m gk,1 (m) = − . m − i(k + 2) m + i(k + 2) + k + 1 i The generating function Gk,1 (x) is a special case of the one for the duration of the game in the classical ruin problem, that is, restricted random walks with absorbing barriers at 0 and a, and initial position z. See, for example, Equation (4.11) of Chapter 14 of [9]: Let ∞ X Uz (x) = uz,m xm , m=0 18 W. CHEN, E. DENG, R. DU, R. STANLEY, AND C. YAN where uz,n is the probability that the process ends with the n-th step at the barrier 0. Then z a−z q λ1 (x) − λa−z (x) 2 Uz (x) = , a a p λ1 (x) − λ2 (x) where p p 1 + 1 − 4pqx2 1 − 1 − 4pqx2 λ1 (x) = , λ2 (x) = . 2px 2px The generating function Gk,1 (x) is just U1 (2x)/x with a = k + 2, z = 1, and p = q = 1/2. In general, by the transfer matrix method [24, §4.7] (5.2) Gk,j (x) = det(I − xAk,j (0)) det(I − xAk,j ) is a rational function, where Ak,j is the adjacency matrix of the Hasse diagram of L(k, j), and Ak,j (0) is obtained from Ak,j by deleting the row and the column corresponding to 0̂. Ak,j (0) is also the adjacency matrix of the Hasse diagram of L(k, j) with its bottom element (the empty partition) removed. Note that det(I − xAk,j ) is a polynomial in x2 since L(k, j) is bipartite [6, Thm. 3.11]. Let det(I − xAk,j ) = pk,j (x2 ). The following is a table of pk,j (x) for the values of 1 ≤ k ≤ j ≤ 4. (We only need to list those with k ≤ j since pk,j (x) = pj,k (x).) √ (k, j) pk,j (x) = det(I − xAk,j ) (1, 1) 1 − x (1, 2) 1 − 2x (1, 3) 1 − 3x + x2 (1, 4) (1 − x)(1 − 3x) (2, 2) (1 − x)(1 − 5x) (2, 3) (1 − x)(1 − 3x)(1 − 8x + 4x2 )) (2, 4) (1 − 14x + 49x2 − 49x3 )(1 − 6x + 5x2 − x3 ) (3, 3) (1 − x)(1 − 19x + 83x2 − x3 )(1 − 5x + 6x2 − x3 )2 (3, 4) (1 − 2x)2 (1 − 8x + 8x2 )(1 − 4x + 2x2 )2 (1 − 16x + 60x2 − 32x3 + 4x4 ) ·(1 − 24x + 136x2 − 160x3 + 16x4 ) (4, 4) (1 − x)2 (1 − 18x + 81x2 − 81x3 )2 (1 − 27x + 99x2 − 9x3 )(1 − 9x + 18x2 − 9x3 )2 ·(1 − 27x + 195x2 − 361x3 )(1 − 6x + 9x2 − x3 )2 (1 − 9x + 6x2 − x3 )2 The polynomial pk,j (x) seems to have a lot of factors. We are grateful to Christian Krattenthaler for explaining equation (5.4) below, from which we can explain the factorization of pk,j (x). By an observation [13, §5] of Grabiner, gk,j (n) is equal to the number of walks with n steps ±ei from (j, j − 1, . . . , 2, 1) to itself in the chamber j + k + 1 > x1 > x2 > · · · > xj > 0 of the affine Weyl group C̃n . Write m = j + k + 1. By [13, (23)] there follows " 2m−1 #j X 1 X x2n = det sin(πra/m) sin(πrb/m) · exp(2x cos(πr/m)) . gk,j (n) (2n)! m r=0 n a,b=1 When this determinant is expanded, we obtain a linear combination of terms of the form (5.3) X xn exp(2x(cos(πr1 /m)+· · ·+cos(πrj /m))) = 2n (cos(πr1 /m)+· · ·+cos(πrj /m))n , n! n≥0 CROSSINGS AND NESTINGS 19 where 0 ≤ ri ≤ 2m − 1 for 1 ≤ i ≤ j. In fact, the case η = λ of Grabiner’s formula [13, (23)] shows that the number of walks of length n in the Weyl chamber from any integral point to itself is again a linear combination of terms 2n (cos(πr1 /m) + · · · + cos(πrj /m))n . It follows that every eigenvalue of Ak,j has the form (5.4) θ = 2(cos(πr1 /m) + · · · + cos(πrj /m)). (In particular, the Galois group over Q of every irreducible factor of pk,j (x) is abelian.) Note that a priori not every such θ may be an eigenvalue, since it may appear with coefficient 0 after the linear combinations are taken. The algebraic integer z = 2(cos(πr1 /m) + · · · + cos(πrj /m)) lies in the field Q(cos(π/m)), an extension of Q of degree φ(2m)/2, where φ is the Euler phi-function. To see this, let z be a primitive 2m-th root of unity. Then z is a root of x + 1/x = 2 cos(π/m). Hence the field L = Q(z) is quadratic or linear over K = Q(cos(π/m). Since K is real and L is not for m > 1, we cannot have K = L. Hence [L : K] = 2. Since [L : Q] = φ(2m), we have [K : Q] = φ(2m)/2. It follows that the minimal polynomial over Q of z has degree dividing φ(2m)/2. Thus every irreducible factor of det(I − Ax) has degree dividing φ(2m)/2, explaining why pk,j (x) has many factors. A more careful analysis should yield more precise information about the factors of pk,j (x), but we will not attempt such an analysis here. An interesting special case of determining pk,j (x) is determining its degree, since the number of eigenvalues of Ak,j equal to 0 is given by j+k − 2 · deg pk,j (x). j Equivalently, since Ak,j is a symmetric matrix, 2 · deg pk,j (x) = rank(Ak,j ). We have observed the following. (1) For k + j ≤ 12 and 1 ≤ k ≤ j, Ak,j is invertible exactly for (k, j) =(1, 1), (1, 3), (1, 5), (1, 7), (1, 9), (1, 11), (3, 3), (3, 7), (3, 9), (5, 5) and (5, 7). (2) A1,j is invertible if and only if j is odd. This is true because L(1, j) is a path of length j, whose determinant satisfies the recurrence det(A1,j ) = − det(A1,j−2 ). The statement follows from the initial conditions det(A1,1 ) = −1 and det(A1,2 ) = 0. (3) If Ak,j is invertible, then kj is odd. To see this, let X0 (X1 ) be the set of integer partitions of even (odd) n whose shape fits in a k × j rectangle. Since L(k, j) is bipartite graph with vertex partition (X0 , X1 ), a necessary condition for Ak,j to be invertible is |X0 | = |X1 |. That is, the generat P must have a root at q = −1, where ing function n p(k, j, n)q n = k+j k p(k, j, n) is the number of integer partitions on n whose shape fits into a k × j rectangle. But the multiplicity of 1 + q in the Gaussian polynomial j k+j k is b k+j k 2 c − b 2 c − b 2 c, which is 0 unless both j and k are odd. Item 3 is also proved independently by Jason Burns, who also found a counterexample for the converse: For k = 3 and j = 11, A3,11 is not invertible, in fact its corank is 6. The invertibility of Ak,j for kj being odd is currently under investigation. Acknowledgments The authors would like to thank Professor Donald Knuth for carefully reading the manuscript and providing many helpful comments. References 1. J. Baik, P. Deift and K. Johansson, On the distribution of the length of the longest increasing subsequence of random permutations. J. Amer. Math. Soc., 12 (1999), No. 4, 1119–1178. 20 W. CHEN, E. DENG, R. DU, R. STANLEY, AND C. YAN 2. J. Baik and E. Rains, Algebraic aspects of increasing subsequences, Duke Math. J., 109 (2001), No. 1, 1–65. 3. J. Baik and E. Rains, The asymptotics of monotone subsequences of involutions, Duke Math. J., 109 (2001), 205–282. 4. H. Barcelo and A. Ram, Combinatorial representation theory, in New Perspectives in Algebraic Combinatorics (Berkeley, CA, 1996–97), MSRI Publ. 38, Cambridge University Press, Cambridge, 1999, pp. 23–90. 5. A. Berele, A Schensted-type correspondence for the symplectic group, J. Combinatorial Theory (A), 43 (1986), 320–328. 6. D. M. Cvetković, M. Doob, and H. Sachs, Spectra of Graphs. 3rd ed., Johann Ambrosius Barth, Heidelberg, 1995. 7. H. Davenport and A. Schinzel, A combinatorial problem connected with differential equations, Amer. J. Math., 87 (1965), 684–694. 8. M. de Sainte-Catherine, Couplages et Pfaffiens en combinatoire, physique et informatique, Ph.D. Thesis, University of Bordeaux I, 1983. 9. W. Feller, An Introduction to Probability Theory and Its Applications, 3rd edition, John Wiley & Sons, Inc., New York, 1968. 10. I. P. Goulden, A linear operator for symmetric functions and tableaux in a strip with given trace, Discrete Math., 99 (1992), 69–77. 11. D. Gouyou-Beauschamps, Standard Young tableaux of height 4 and 5, Europ. J. Combin., 10 (1989), 69–82. 12. D. Grabiner and P. Magyar, Random walks in Weyl chambers and the decomposition of tensor powers, J. Alg. Combinatorics, 2 (1993), 239–260. 13. D. Grabiner, Random walk in an alcove of an affine Weyl group, and non-colliding random walks on an interval, J. Combin. Theory Ser. A, 97 (2002), 285-306. 14. C. Greene, An extension of Schensted’s theorem, Adv. Math., 14 (1974), 254–265. 15. T. Halverson and A. Ram, Partition algebras, European J. of Combinatorics, 26 (2005), no. 6, 869–921. 16. T. Halverson and T. Lewandowski, RSK insertion for set partitions and diagram algebras, Electron. J. Combin. 11(2004/06), No.2, Research paper 24. Preprint available at math.CO/0507026. 17. M. Klazar, Bell numbers, their relatives, and algebraic differential equations, J. Combin. Theory Ser. A, 102 (2003), 63–87. 18. M. Korn, Personal communication. 19. S. G. Mohanty, Lattice Path Counting and Applications, Academic Press, New York, 1979. 20. R. C. Mullin and R. C. Stanton, A map-theoretic approach to Davenport-Schinzel sequences, Pacific J. Math., 40 (1972), 167–172. 21. J. Riordan, The distribution of crossings chords joining pairs of 2n points on a circle, Math. Computation, 29 (1975), 215-222. 22. C. E. Schensted, Longest increasing and decreasing subsequences, Canad. J. Math., 13 (1961), 179–191. 23. R. Stanley, Differential posets, J. Amer. Math. Soc., 1 (1988), 919–961. 24. R. Stanley, Enumerative Combinatorics, vol. 1, Wadsworth and Brooks/Cole, Pacific Grove, CA, 1986; second printing, Cambridge University Press, Cambridge, 1996. 25. R. Stanley, Enumerative Combinatorics, vol. 2, Cambridge University Press, Cambridge, 1999. 26. R. Stanley, Supplementary Exercises for Chapter 7 of Enumerative Combinatorics, available at http://www-math.mit.edu/∼rstan/ec. 27. S. Sundaram, The Cauchy identity for Sp(2n), J. Combin. Theory Ser. A, 53 (1990), 209–238. 28. L. Takács, Ballot problems, Z. Wahrsch. Verw. Gebiete, 1 (1962), 154–158. 29. J. Touchard, Sur un problème de configuration et sur les fractions continues, Canad. J. Math., 14 (1952), 2-25. 30. E. T. Whittaker and G. N. Watson, A Course of Modern Analysis, Cambridge University Press, Cambridge, 1927. CROSSINGS AND NESTINGS 21 Center for Combinatorics, LPMC, Nankai University, Tianjin 300071, P.R. China E-mail address: chen@nankai.edu.cn Center for Combinatorics, LPMC, Nankai University, Tianjin 300071, P.R. China Current address: Department of Applied Mathematics, Dalian University of Technology, Dalian, Liaoning 116024, P.R.China E-mail address: dengyp@eyou.com Center for Combinatorics, LPMC, Nankai University, Tianjin 300071, P.R. China Current address: Department of Mathematics, East China Normal University, Shanghai 200062, P.R. China E-mail address: rxdu@math.ecnu.edu.cn Department of Mathematics, Massachusetts Institute of Technology, Cambridge, MA 02139 E-mail address: rstan@math.mit.edu Department of Mathematics, Texas A&M University, College Station, TX 77843-3368 E-mail address: cyan@math.tamu.edu