1. Latin hypercube sampling
Latin hypercube (LH) sampling is a method (Santner et al. 2003) for generating a distribution of
parameter vectors in a multidimensional space of parameter values:

Let x represent a D dimensional parameter vector x = (x
1
, x
2
…, x
D
) T , where D is the number of parameters in our ODE model.

When sampling for M parameter vectors (i.e., sets of possible parameter values), the range of each parameter is divided into M > 1 mutually exclusive intervals.

One sample point is randomly selected from each interval for the first parameter x
1
. Hence,
M values are sampled: x
1,1
through x
1, M
. These values are paired with M values sampled for the second parameter x
2
randomly without replacement, which means that each x
2
value is paired with only one value of x
1
.

Then, these M pairs are combined with M values of x
3
, once again without replacement, thereby generating M triplets.

This procedure is continued until M D -tuples are generated.
We implement LHS using Matlab code available in
(http://www.mathworks.com/matlabcentral/fileexchange/4352-latin-hypercubesampling/content/lhsu.m).
2. Differential evolution
Differential Evolution (DE) is a method for improving a set of parameter values (i.e., a parameter vector x , as above) with respect to an objective function, Φ ( x ), by generating a sequence of trial
parameter vectors by processes of reproduction and selection (Price et al. 2005). Reproduction
generates an offspring parameter vector u from a parent parameter vector x by diversification. If the offspring performs better than the parent, i.e., if Φ ( u ) > Φ ( x ), then the parent vector x is replaced
1

by the offspring vector u in the next generation. In our case, the objective function,
Φ
( x ), is an integer-valued function that counts the number of constraints that are correctly predicted by the model given the parameter values in the vector x .
During DE, parameter vectors evolve from generation to generation. Each generation
(indexed by t = 0, 1, …) consists of N parameter vectors x j
, j = 1, …, N . Hence, the real number x i,j
( t ) is the value of the i th parameter in the j th parent in the t th generation. Let u j
( t ) be the parameter vector for the single offspring of the j th parent in the t th generation. The components of this vector, u i , j
( t ) for i = 1, …, D , are constructed in two steps (mutation and crossover). Then, given the two parameter vectors x j
( t ) and u j
( t ), a decision is made as to which one is propagated to generation t +1. The following three operations propagate parameter vectors from one generation to the next:
1.
Mutation: First, we create a “mutant” vector v j
( t ) by perturbing a parental vector x j
( t ). In our DE approach, we let the perturbation vector be the difference between the parameter vectors of two additional parents, j ′ and j ″, chosen at random from the t th generation of parents. All three parents must be different. The “mutant” vector is defined by: 𝐯 𝑗
(𝑡) = 𝐱 𝑗
(𝑡) + 𝐹 ∙ (𝐱
′ 𝑗
(𝑡) − 𝐱 𝑗
′′
(𝑡)) (2) where F is a scalar, 0 < F
< 1, that determines how “aggressive” the mutation is.
2.
Crossover: Next we allow for crossover between the parental parameter set x j
( t ) and the mutant parameter set v j
( t ). Component-wise, the offspring vector u j
( t ) receives a parameter value from the mutant vector with probability C (the crossover probability) or from the parent vector with probability 1−
C : 𝑢 𝑖,𝑗
(𝑡) = { 𝑣 𝑖,𝑗
(𝑡), if 𝑟𝑎𝑛𝑑(0,1) ≤ 𝐶 𝑥 𝑖,𝑗
(𝑡), otherwise
(3) 𝑖 = 1,2, … , 𝐷 and 𝑗 = 1,2, … , 𝑁.
We choose C =0.5 so that parental values and mutant values have equal chances to be in the offspring vector.
2

3.
Selection: Depending on their relative performances, x j
( t ) or u j
( t ) passes on to the next generation. There are two possibilities here. For “greedy” selection, the offspring replaces its parent if it is superior: 𝐱 𝑗
(𝑡 + 1) = { 𝐮 𝑗
(𝑡), if 𝛷(𝐮 𝑗
(𝑡)) > 𝛷(𝐱 𝑗
(𝑡)) 𝐱 𝑗
(𝑡), otherwise
(4)
For “non-greedy” selection, the condition for replacement is:
𝛷 (𝐮 𝑗
(𝑡)) ≥ 𝛷 (𝐱 𝑗
(𝑡)) (5)
We implement DE using the Matlab code available in
(http://www1.icsi.berkeley.edu/~storn/code.html).
3. Identification of feasible parameter vectors and robustness analysis
During the evolutionary search of DE, the parameter vectors that reproduced 18–22 constraints were recorded as feasible vectors; hence our feasibility criterion for each parameter vectors was the ability to reproduce at least 82% (18/22) of the constraints. In ~500 generations of the second round of DE, we identified a total of 312 vectors that met this criterion. While optimizing parameters, we only ran one realization per parameter vector during DE. After optimization was completed, in order to assess the reproducibility of the model’s success among the feasible set of vectors, we performed 15 realizations per vector (15×312 = 4680 total realizations) and found out that 20.35 ±
1.16 (mean ± standard deviation) constraints were reproduced among these realizations.
Furthermore, per parameter vector, the average number of constraints captured (among 15 realizations) was below 17 for only two of the 312 feasible vectors. From these results, we concluded that the vector space spanned by this population of parameter vectors was a good representation of the feasible region in the model’s parameter space.
3

In order to assess the model’s robustness, per feasible parameter vector, we simulated the entire set of experimental conditions upon perturbing each model parameter with six different perturbation magnitudes (±30%, ±60%, and ±90%). Each perturbation gave rise to a new parameter vector with which 22 simulations corresponding to conditions listed in Table 1were performed. Since the control parameters were already varied in the simulations to mimic different experimental conditions (per given parameter vector), we did not perturb those parameters to create new parameter vectors. Similarly, an additional group of five parameters (not listed in Table S3) that decide the unit strengths of the TCR and TGF-β signals were excluded since they were simply the scaling factors used to sample the signal strength space.
Next, we analyzed the simulation results with parameter vectors that were generated by the perturbations. Per parameter, we computed the fraction of simulations that capture the experimental constraints. Here, we performed a total of 41184 simulations (312 vectors × 6 perturbation levels
× 22 constraints) for each parameter. The fraction of successful simulations per parameter ( f p
) had a mean value of 0.83 and standard deviation of 0.11 (range of 0.49–0.93). The least robust model parameter was able to reproduce the experimental constraints about half of the time despite large perturbations (range of −90% to +90%). These results show that the model is highly robust.
Interestingly, the most sensitive ten model parameters ( f p
range of 0.49–0.75) included the basal activation states of all four master regulators in addition to the weights of activation by TCR for the same regulators. This result suggested that the basal states and the TCR induced activation strengths need to be finely tuned so that the system can faithfully reproduce the experimental constraints. The remaining two top ten sensitive parameters were the steepness values of the sigmoidal functions for RORγt and GATA3 activities, whereas the steepness parameters of Foxp3 and T-bet activities were also among the top half of the sensitivity based ranking among model parameters. This was not surprising since the steepness values of these sigmoidal functions are
4

expected to affect the temporal orders of the activation/inactivation events in the simulations, thereby affecting model’s ability to satisfy the experimental constraints.
The least sensitive ten parameters ( f p
range of 0.92–0.93) were either activation or inhibition weights. For eight of these ten interaction parameters, the affected protein (not the effector) was either T-bet or GATA3 suggesting higher robustness of these two regulators compared to Foxp3 and RORγt. Measuring these parameters experimentally can potentially help narrow down the parametric uncertainty of the model since with the current list of imposed experimental constraints, these parameters do not affect the model’s performance despite being significantly perturbed from their optimized values.
We next computed the fraction of successful simulations per constraint ( f c
) in order to rank the experimental constraints based on their robustness against perturbations. Here, we note that the total number of simulations to compute f c
is 87984 (312 parameter vectors × 47 parameters × 6 perturbation levels) per constraint. Forty-seven model parameters (excluding the control parameters) had f c
values ranging 0.38–0.99. Hence, the least robust constraint was satisfied by the model 38% of the time despite large perturbations in parameter values. The mean f c
value among all constraint was 0.83 ± 0.17 once again highlighting the model’s robustness. Six constraints that are based solely on naïve cell fractions were among the most robust nine constraints with f c
values ranging 0.94–0.99. This suggested that when a constraint depended on only one cell fraction (as in
7 of the 22 constraints) instead of two or more fractions, it was favored by the model. In addition, experimental conditions in which TCR signal was present tended to be less robust. TCR signal based constraints (16 of the 22) had an average f c value of 0.78 whereas the non-TCR constraints averaged an f c
value 0.96. This result was also consistent with the high sensitivity values of TCR induced activation rates of the master regulators reported earlier in this section.
Of course, the fact that the 47 model parameters are optimized with 22 experimental constraints raises some doubts about our ability to constrain these parameters. However, with the analysis
5

presented in this section, we provide some insights into potentially useful experiments while differentiating between robust and sensitive aspects of the model. Nevertheless, the high robustness of the model suggests that one may need a much larger group of experimental constraints to obtain parameter estimates with low uncertainties.
Finally, we computed the fraction of successful simulations per feasible parameter vector ( f v
) in order to rank the feasible vectors based on their robustness against perturbations. Among the 312 feasible vectors, f v
had a range of 0.69–0.89. We used the most robust parameter vector in order to generate the results in the manuscript and also to populate Table S3 with the optimized parameter values.
4. Bifurcation of a three-fold symmetrical differentiation system
Our symmetrical model with three master regulators is similar to the elastic cube model described
by Ball and Schaeffer (Ball and Schaeffer 1983) and Golubitsky et al. (Golubitsky et al. 1988). In
this later model, an incompressible elastic body in the shape of a unit cube is under uniform traction normal to each face. With the increase of the three-fold symmetrical traction, the elastic body deforms into rod-like shape (one side longer than the other two) or plate-like shape (one side shorter than the other two), yielding two types of steady states.
In our model, expression levels of three master regulators can be viewed as the sides of the elastic body. Therefore, the naïve cell and the triple-positive cell correspond to the cube-like state, the single-positive cell corresponds to the rod-like state, and the double-positive cells correspond to the plate-like state. Bifurcation among these types of states is illustrated concisely in Fig. S1(a), whereas Fig. S1(b) depicts each local bifurcation scenario with the standard bifurcation diagram.
With the increase of the signal (traction), the cube-like state first bifurcates into rod-like states (Fig.
S1(b) left inset), then each rod-like state bifurcates into two asymmetrical states, which soon
6

bifurcate into plate-like state with two-fold symmetry (Fig. S1(b) upper and right insets). With further still increase of signal, the plate-like states bifurcate into a cube-like state with three-fold symmetry.
We used variation of parameters in the ODE to simulate cell-to-cell variability in a cell population. This is because we considered the cell-to-cell variability as ‘population noise’ rather than ‘temporal noise’. Nonetheless, stochastic differential equation (SDE) based model with temporal fluctuations of state variables is another way to simulate cell-to-cell variability. To check if the two methods for modeling noise are equivalent, we have rewritten our equations for all four state variables in a form of SDE with additive white noise as follows: dx / dt
     where dx / dt
  
is ODE with the original basal parameters, and
  
is Gaussian white noise with mean 0 and variance 0.12). With this simple SDE form, we reproduced all of the 22 constraints, suggesting that our approach is robust to the choice of methods for modeling cell-to-cell variability.
7

References
Ball J, Schaeffer D Bifurcation and stability of homogeneous equilibrium configurations of an elastic body under dead-load tractions. In: Math. Proc. Camb. Phil. Soc, 1983. Cambridge
Univ Press, pp 315-339
Golubitsky M, Stewart I, Schaeffer DG (1988) Singularities and Groups in Bifurcation Theory
Volume II Applied Mathematical Sciences, http://www1.icsi.berkeley.edu/~storn/code.html
. Accessed August 2013 http://www.mathworks.com/matlabcentral/fileexchange/4352-latin-hypercubesampling/content/lhsu.m
. Accessed August 2013
Price KV, Storn RM, Lampinen JA (2005) Differential evolution : a practical approach to global optimization. Natural computing series,. Springer, Berlin
Santner TJ, Williams BJ, Notz W (2003) The Design and analysis of computer experiments.
Springer series in statistics. Springer, New York
8

Legend for Supplementary Figure S1
Illustrative bifurcation diagrams of three-fold symmetrical models. a) Abstract illustration of the bifurcations among rod-like, plate-like and cube-like states for an incompressible elastic cube under uniform traction normal to all six faces [5, 6]. b) Bifurcation diagram of our three-fold symmetrical model with respect to S1. Solid curves: stable steady states; dashed curves: unstable steady states.
Insets are zoomed-in plots to illustrate the local bifurcation scenarios. Radar plots are used to denote the steady states in each region of the bifurcation diagram.
9

Table SI. Parameter values for the generic model with three master regulators
Parameter Description Value
Strength of S1 signal
0 − 10
S1
 o
Y

Z o

X

X

Y

Y

Z

Z

Y

X

X

Y

Z

Y

Y

Z

X

Z

Z

X

S 1

X

S 1

Y

S 1

Z

X

Y

Z

X

Y

Z

X o
Relaxation rate of X
Relaxation rate of Y
Relaxation rate of Z
Steepness of sigmoidal function for X
Steepness of sigmoidal function for Y
Steepness of sigmoidal function for Z
Basal activation state of X
Basal activation state of Y
Basal activation state of Z
Weight of autoactivation of X
Weight of autoactivation of Y
Weight of autoactivation of Z
Weight of inhibition on X by Y
Weight of inhibition on Y by X
Weight of inhibition on Y by Z
Weight of inhibition on Z by Y
Weight of inhibition on Z by X
Weight of inhibition on X by Z
Weight of activation on X by S1
Weight of activation on Y by S1
Weight of activation on Z by S1
5
5
5
3
3
3
−2
−2
−2
0.2
0.2
0.2
−3
−3
−3
−3
−3
−3
1
1
1
10

Table SII. Parameter values for the generic model with four master regulators
Parameter Description Value
Strength of S1 signal
0 − 15
S1
 o
Y
 o
Z

W

W

X

X

Y

Y

Z

Z

X

W

Y

W

Z

W

W

X

Y

X

Z

X

W

Y

X

Y

Z

Y

W

Z

Y

Z

X

Z

S 1

W

S 1

X

S 1

Y

S 1

Z

W

X

Y

Z

W

X

Y

Z
 o
W
 o
X
Relaxation rate of W
Relaxation rate of X
Relaxation rate of Y
Relaxation rate of Z
Steepness of sigmoidal function for W
Steepness of sigmoidal function for X
Steepness of sigmoidal function for Y
Steepness of sigmoidal function for Z
Basal activation state of W
Basal activation state of X
Basal activation state of Y
Basal activation state of Z
Weight of autoactivation of W
Weight of autoactivation of X
Weight of autoactivation of Y
Weight of autoactivation of Z
Weight of inhibition on W by X
Weight of inhibition on W by Y
Weight of inhibition on W by Z
Weight of inhibition on X by W
Weight of inhibition on X by Y
Weight of inhibition on X by Z
Weight of inhibition on Y by W
Weight of inhibition on Y by X
Weight of inhibition on Y by Z
Weight of inhibition on Z by W
Weight of inhibition on Z by Y
Weight of inhibition on Z by X
Weight of activation on W by S1
Weight of activation on X by S1
Weight of activation on Y by S1
Weight of activation on Z by S1
5
5
5
5
3
3
3
3
−2
−2
−2
−2
0.2
0.2
0.2
0.2
−3.5
−3.5
−3.5
−3.5
−3.5
−3.5
−3.5
−3.5
−3.5
−3.5
−3.5
−3.5
1
1
1
1
11

Parameter name
TCR
TGF-β
IL-12
IL-4
IL-6
IL-23
IL-1
ATRA antiIFNγ
 antiIL4

T bet
GATA3

RORγ t

Foxp3


T bet
GATA3

RORγ t

Foxp3

T o
bet
 o
GATA3
 o
RORγ t
 o
Foxp3

T bet

T bet

GATA3

GATA3

RORγ t

RORγ t

Foxp3

Foxp3

GATA3

T bet

RORγ t

T bet

Foxp3

T bet

T bet

GATA3

RORγ t

GATA3

Foxp3

GATA3

T bet

RORγ t

GATA3

RORγ t
Table SIII. Parameter values for CD4 + T cell model
Description
Parameter range
Strength of TCR signal
Strength of TGF-β signal
Strength of IL-12 signal
Strength of IL-4 signal
Strength of IL-6 signal
Strength of IL-23 signal
Strength of IL-1 signal
Strength of ATRA signal
Strength of antiIFNγ signal
Strength of antiIL4 signal
Relaxation rate of T-bet
0 - 3
0 or 5
0 or 5
0 or 5
0 or 5
0 or 5
0 or 5
0 or 5
0 or 0.25
0 or 0.75
[2.5, 10]
Relaxation rate of GATA3
[2.5, 10]
Relaxation rate of RORγt
[2.5, 10]
[2.5, 10]
Relaxation rate of Foxp3
Steepness of sigmoidal function for T-bet
[1.25, 5]
Steepness of sigmoidal function for GATA3
[1.65, 6.6]
Steepness of sigmoidal function for RORγt
[1, 4]
[1, 4]
Steepness of sigmoidal function for Foxp3
[−4, −1]
Basal activation state of T-bet
[−4, −1]
Basal activation state of GATA3
Basal activation state of RORγt
[−4.4, −1.1]
[−4, −1]
Basal activation state of Foxp3
Weight of autoactivation of T-bet
[1, 4]
Weight of autoactivation of GATA3
[1, 4]
Weight of autoactivation of RORγt
[1.5, 6]
Weight of autoactivation of Foxp3
[1, 4]
[−7, −1.75]
Weight of inhibition on T-bet by GATA3
Weight of inhibition on T-bet by RORγt
[−3, −0.75]
Weight of inhibition on T-bet by Foxp3
[−7, −1.75]
[−5, −1.25]
Weight of inhibition on GATA3 by T-bet
Weight of inhibition on GATA3 by RORγt [−7, −1.75]
[−7, −1.75]
Weight of inhibition on GATA3 by Foxp3
Weight of inhibition on RORγt by T-bet
Weight of inhibition on RORγt by GATA3
[−0.4, −0.1]
[−7, −1.75]
12
Optimized value
0 – 3*
0 or 5*
0 or 5*
0 or 5*
0, 1.5 or 5*
0 or 5*
0 or 5*
0 or 5*
0 or 0.25*
0 or 0.75*
4.23
9.99
7.33
6.70
4.61
3.98
3.46
3.89
−2.76
−3.01
−2.82
-2.67
2.05
2.00
3.99
3.56
−6.81
−1.03
−1.92
−2.44
−5.58
−2.92
−0.29
−3.80


Foxp3

RORγ t

T bet

Foxp3

RORγ t

Foxp3

GATA3

Foxp3

TCR

T bet

TCR

GATA3

TCR

RORγ t

TCR

Foxp3

TGF -
β 
T bet

TGFβ

GATA3

TGFβ

RORγ t

TGF -
β 
Foxp3

IL 1

T bet

IL 1

RORγ t

IL 12

T bet

IL 12

GATA3

IL 4

T bet

IL 4

GATA3

IL 6

RORγ t

IL 6

Foxp3

ATRA

RORγ t

ATRA

Foxp3

IL 23

RORγ t
Weight of inhibition on RORγt by Foxp3
Weight of inhibition on Foxp3 by T-bet
Weight of inhibition on Foxp3 by RORγt
Weight of inhibition on Foxp3 by GATA3
Weight of activation on T-bet by TCR
Weight of activation on GATA3 by TCR
Weight of activation on RORγt by TCR
Weight of activation on Foxp3 by TCR
Weight of inhibition on T-bet by TGF-β
Weight of inhibition on GATA3 by TGF-β
Weight of activation on RORγt by TGF-β
Weight of activation on Foxp3 by TGF-β
Weight of activation on T-bet by IL-1
Weight of activation on RORγt by IL-1
Weight of activation on T-bet by IL-12
Weight of inhibition on GATA3 by IL-12
Weight of inhibition on T-bet by IL-4
Weight of activation on GATA3 by IL-4
Weight of activation on RORγt by IL-6
Weight of inhibition on Foxp3 by IL-6
Weight of inhibition on RORγt by ATRA
Weight of activation on Foxp3 by ATRA
Weight of activation on RORγt by IL-23
* Control parameters not subject to optimization.
[−7, −1.75]
[−7, −1.75]
[−7, −1.75]
[−7, −1.75]
[0.5, 2]
[0.49, 1.96]
[0.15, 0.6]
[0.15, 0.6]
[−1.6, −0.4]
[−1.6, −0.4]
[0.055, 0.22]
[0.06, 0.24]
[0.1, 0.4]
[0.04, 0.16]
[0.05, 0.2]
[−2, −0.5]
[−2, −0.5]
[0.05, 0.2]
[0.015, 0.06]
[−0.06, −0.02]
[−2, −0.5]
[0.01, 0.04]
[0.01, 0.04]
0.09
0.03
−0.06
−1.49
0.01
0.02
0.17
0.19
0.10
0.19
−0.92
−1.15
−1.77
−2.70
−1.96
−6.83
1.02
0.89
0.48
0.45
−1.53
−0.75
0.17
13