Appendix S1: Description of Algorithms A.1 and A.2.
Algorithm A.1. An E-M type algorithm for constrained estimation in linear mixed models
As noted earlier in this article, our testing procedure is flexible as one could any
constrained estimator (e.g. RMLE). Along the lines of [1] we propose a simple E-M type
algorithm to obtain a constrained estimator as described below. If the unconstrained estimator of
 is either exactly or asymptotically multivariate normally distributed then following arguments
in [2], under some suitable conditions, the resulting constrained estimator of   A would
converge to the restricted maximum likelihood estimator. Thus, in the following we do not
assume that the underlying data are normally distributed. However, since we are using the
MINQUE for the variance components, the unconstrained estimator of 
(under suitable
regularity conditions) is asymptotically normally distributed.
Let ˆ1( m) , ˆ2( m) , ˆ 2( m ) , ˆ 2( m ) denote the mth iterate estimates of 1 ,  2 ,  2 , and  2 ,
respectively.
1
Step 0 (m = 0). Let X   X1 : X2  ,   1,  2  . Compute  (0)   XX  XY , the ordinary least
squares estimator for  . Compute  i2(0) 
1
Yi  Xi (0)
ni
2
for i  1,
, k . For ˆi2(0) we use
MINQUE [3].
Step 1. Set m = m + 1. Fix 1 ,  2 , and  2 at ˆ1( m1) , ˆ2( m1) , ˆ 2( m1) respectively, and iteratively
estimate  2 (for m = 1, 2, …):
1 4( m 1)
ˆ i

ni
ˆ i2( m )   i2( m 1) 
 ˆ ( m 1)
tr  Ψ



ˆ ( m 1)
 Ψ

 Y  X ˆ
1
1
( m 1)
1 1
 , i  1,
 ii

 X 2ˆ2( m 1) Y  X1ˆ1( m 1)  X 2ˆ2( m 1)
  Ψˆ 
( m 1)
1
(A1)
, k,

ˆ ( m1)  UT
ˆ ( m1) U  Σ
ˆ , Τˆ ( m1)  diag ˆ2( m1) I :
where Ψ
1
c1

: ˆq2( m1) Icq , ˆ 2(0)   2( m1) , and Aii
indicates the (i, i)th block of A.
Step 2.
(a) Fix  2 at ˆ 2( m1) and iteratively estimate 1 ,  2 , and  2 using the following equations (for m
= 1, 2, …):




ˆ1( m)  ˆ1( m1)  X1 Σˆ 1X1
ˆ2( m)  ˆ2( m 1)  X2 Σˆ 1X2
ˆi2( m )  ˆi2( m1) 
ˆi4( m1)
ci


 Y  X1ˆ1( m 1)  X2ˆ2( m 1)
where ˆ(0)   ( m1) ,

diag 12( m1)Ic1 :

ˆ ( m 1)
tr U i  Ψ


1
1
 Y  X ˆ
 X2ˆ2( m1) ,

(A2)

 Y  X ˆ
 X2ˆ2( m 1) ,

(A3)
ˆ ( m 1)
X2 Ψ
1
1
( m 1)
1 1
   Ψˆ 
1
( m 1)
( m 1)
1 1
1
 Y  X ˆ
 ˆ ( m 1)
Ψ
ˆ 2(0)   2( m1) ,

ˆ ( m1)
X1 Ψ
1
( m 1)
1 1

 X2ˆ2( m 1) 

 U , i  1,
 i
ˆ ( m1)  UT
ˆ ( m1) U  Σ
ˆ,
Ψ
(A4)
, q,
and
ˆ ( m1)
Τ
is

:  q2( m1)Icq . Note that although we could have combined equations (A2) and



(A3) and use ( XΣ1X) XΣ1Y to obtain a combined updated estimate for ˆ( m )  ˆ1( m) : ˆ2( m) ,
our proposed approach is computationally more stable and simpler.
(b) Using the estimates obtained in (a), we apply the PAVA type methodology along the lines of
[4] with weights proportional to the inverse of the sample sizes to obtain ˆ1( m) under the desired
inequality constraints. Alternatively, one could perform the following constrained optimization
to obtain ˆ1( m) :
ˆ -1(i ) (Y - X  i  X ˆi ) .
min(Y - X11i  X2ˆ2i ) ' Ψ
1 1
2 2
A 0
(A5)
Steps 1 and 2 are iterated until convergence. The resulting constrained estimators are denoted by



ˆ  ˆ1,ˆ2 . Since it is well-known that the RMLE can perform poorly, even for simple order
restriction when the data are correlated (see [4]), therefore as an alternative to (A5), we consider
PAVA type methodology of [4]. Thus the point estimators derived here are very generally
applicable for any order restriction. In Step 2(b), if one uses (A5) for obtaining a constrained
2



estimator then applying the general theory established in [5], we note that ˆ  ˆ1,ˆ2 is
consistent.
To describe the MINQUE methodology for estimating   12 , 22 ,..., q2 ,  12 ,  22 ,...,  k2 
we rewrite the linear model (1) as Y  X   , where   1,  2  , X   X1 : X2  with
qk
E ( )  0 , Var ( )  G  i Fi , Fi  Ui Ui , i  1, 2, ..., q, and Fi  Diag[0 : 0 :...: I : 0...0] ,
i 1
i  q  1, 2, ..., q  k with the identity matrix of order ni  q  ni  q located at the ith location. Each
Fi is N  N . Let F  F1 : F2 :...: Fq  k  , W  G  XX , R  W  W X(X ' W X) X ' W ,
z  (Y R F1R Y , Y R F2 R Y ,..., Y R Fq  k R Y ) , and S  Tr (R Fi R F j ), i, j  1, 2,..., q  k  .
In the above expression, A  denotes a generalized inverse (or g-inverse) of A . The
above expressions are invariant to the choice of g-inverse. Hence without loss of generality one
may use the Moore-Penrose inverse A  . From [3], the MINQUE of  is then obtained by
solving the system of linear equations S ( 0)   z ( 0) , where  (0) denotes an initial estimate of  .
Since the MINQUE depends upon the initial estimate, Rao and Kleffe [6] recommend iterating
as above until convergence. The resulting estimator is known as the iterated MINQUE (or IMINQUE). Denote the I-MINQUE of  by ˆ . As discussed in [7], estimated parameters can be
negative. As is commonly done, in such cases we replace them with 0.01. Since   A is an
estimable linear function of  , its weighted least squares estimator is given by
ˆ  A( XWˆ X) XWˆY .
Let
R1: 0  min 1 , 2 ,
R2: E 
R3:
4
, q  k   min 1 , 2 ,
, q  k    .
.

 d
Tr  W F W F 
Tr W Fi W Fj



i

ij
, where the matrix D   dij , i, j  1, 2,
i
3
, q  k  is non-singular.
R4:
max  W Fi  max  W Fj 

Tr W Fi W Fi

 0 , where max  H  is the largest eigenvalue of a matrix H .
R5: ( X' W X)   M ( ) , where M ( ) is a positive definite matrix.
R6: max (XXWG W XX)  0 .
Theorem A1: For any estimable linear function A in a linear mixed model (1) satisfying the
regularity conditions R1 to R6,
A(θˆ - θ )
asymptotically
~
N (0, AM( ) -1 A' )
Proof: Under the regularity conditions R1 to R4, from Theorem 10.2.3 in [6] we deduce that the
MINQUE ˆ is consistent for  . Appealing to Noether’s conditions (R5 and R6) we deduce the
asymptotic normality of Aˆ from the discussion in Chapter 10.7 in [6].
Algorithm A.2. The EBLUP based bootstrap methodology
To derive the p-values of the above test statistics, we now describe the non-parametric EBLUP
bootstrap methodology for deriving the null distribution of the test statistic (9). We begin by
constructing bootstrap sample Y * as follows.
Step 1: Obtain the point estimator of   1,  2  under the null hypothesis. Denote it by


 0  10 , 20 .



Step 2: Let θˆ  ˆ1,ˆ2 , T̂ , and Σ̂ denote the unconstrained point estimators of   1,  2  , T
and Σ . Compute
ˆ  UT
ˆ  UTU
ˆ 1X) 1 XΨ
ˆ 1 ,
ˆ, Ψ
ˆ  Σ
ˆ , Pˆ  X( XΨ
C


 ˆ ˆ 1
Ψ (I  Pˆ )Y .
ˆ  (I  Pˆ )Y , ˆ  ˆ1 : ˆ2 :...: ˆq  C
Step 3: Let ˆi 
ˆi
ˆi
, i  1, 2,..., k where sd (.) represents the
, i  1, 2,..., q and let ˆi 
ˆ
sd (ˆi )
sd (i )
4
usual sample standard deviation of the elements in the vector.
Step 4: Let ˆi* , i  1, 2,..., q , denote a random vector obtained by taking a random sample (with
replacement) of size ci 1, i  1, 2,
i  1, 2,
, q from the components of ˆi . Similarly, letˆi* ,
, k denote a random vector obtained by taking a random sample (with replacement) of
size ni  1 from the components of ˆi . Finally, let ˆ*  ˆiˆi* , i  1, 2,
i  1, 2,
, q and ˆi*  ˆ iˆi* ,
, k , then the EBLUP bootstrap sample Y * is constructed using the following equation
Y *  X 0  Uˆ*  ˆ* .
The above model honors the null hypothesis regarding the parameter   1,  2  as well
as honors the underlying variance components structure.
REFERENCES
1 Hoferkamp CL, Peddada SD (2002). Parameter estimation in linear models with
heteroscedastic variances subject to order restrictions. Journal of Multivariate Analysis 82: 6587.
2 Davidov O, Rosen S (2011). Constrained inference in mixed-effects models for longitudinal
data with application to hearing loss. Biostatistics 12: 327-340.
3 Rao CR (1972). Estimation of variance and covariance components in linear models. Journal of
the American Statistical Association 67: 112-115.
4 Hwang JTG and Peddada SD (1994). Confidence Interval Estimation Subject to Order
Restrictions. Annals of Statistics, 22: 67-93.
5 Nettleton, D (1999). Convergence properties of the em algorithm in constrained parameter
spaces. The Canadian Journal of Statistics / La Revue Canadienne de Statistique 27: 639-648.
6 Rao CR, Kleffe J (1988). Estimation of variance components and applications.
Amsterdam; New York; New York, N.Y., U.S.A.: North-Holland.
7 Rao JNK, Subrahmaniam K (1971). Combining independent estimators and
estimation in linear regression with unequal variances. Biometrics 27: 971-990.
5