MS 305 Recitation 11
Output Analysis I
16.05.2013
Half Width

We enforce an upper bound on error h, satisfying the following
inequality:

When the above probability is equal to (1 - α), h is called as half-width.

In our analyses, we make use of the fact that the statistic below follows
a t-distribution:

The half width can be computed by plugging T into the equation below.
P(t / 2,n 1  T  t / 2,n 1 )  1   .
2
Half Width
Then, the formula for half-width is given by:

Half-width =
Keep in mind that the t-distribution assumption is only valid
when the outputs are independent, identically distributed, and
follows a normal distribution.



3
i.i.d. assumption is justified by ARENA since we are choosing
initialize statistics and initialize system.
Normality assumption is justified using the Central Limit
Theorem.
Number of Replications
4
Number of Replications

Once the approximate number of replications is obtained, you should
re-run the model in order to check whether the desired half-width is
obtained.

Example:





5
Suppose that the initial number of replication is 10 and it leads to an initial s²
value,
Using this we compute n,
We re-run the model,
If the calculated half-width is less than or equal to h (the specified value), we
are done,
Otherwise we take the recent s² as our initial guess and reuse the
approximation to recalculate the new value of n…
Number of Replications
A reasonable strategy to specify an upper bound on half width
(h) would be to use a proportion of the sample mean.


i.e.
where γ denotes the percentage of deviation from
sample mean

When the formula of half-width is given, you should know how
to approximate the number of replications.

In other words, the approximation formulas will not be given
in the exam since they are intuitive and can be obtained by
simply manipulating the half-width formula.
6
Exercise 6.03
In exercise 5.03, about how many replications would
be required to bring the half-width of a 95%
confidence interval for the expected average cycle
time for both trimmers down to one minute?


7
Start with 10 replications.
Exercise 6.03
We run the simulation with 10 replications and obtain the half-width
associated with the 95% confidence interval.


Important: α=0.05 is the default value used in the half-width
computation of Arena.

The half-widths for the expected average cycle time for primary and
secondary trimmer are 2.11, and 14.77, respectively.

To reduce the half-widths to 1, we work with worst (14.77) and
compute the approximate number of required replications (using the
second approximation for sample size) as (10)x(14.77/1)²= 2181.529.

Rounding up to the nearest integer gives 2182 replications.
8
Confidence Interval
Using the results on half width, we can construct a
100(1-α)% confidence interval on mean as below:

where
9
Statistical Comparison of Outputs

In output analysis, we are often concerned with the
comparison of outcomes obtained from different sources.

Examples for these sources could be:


Two simulation models that could represent the current system and a
modified version of the same system,
Data from the real life, and data from the simulation model.


To prove the validity of the model.
More specifically, we want to decide whether these two
outcomes are significantly different in terms of the specified
mean performance measure.
10
Statistical Comparison of Outputs

Such a comparison can be made by performing the following
hypothesis test:
where μ1 and μ2 denote the mean of the random outcome from
data source 1 and 2.

Note: t-test is a two-sided test; both positive and negative
deviations from the hypothesized value (0 in our case) matter.
11
Statistical Comparison of Outputs

In order to make such a comparison, one can utilize the
confidence intervals.

Let X and Y represent the random outcome obtained from
source 1 and source 2, we define the following random
variable:
.

We construct a confidence interval for d:

If 0 lies inside this interval, then we cannot claim that the
mean differences are significantly different from 0.
12
Example


Suppose we have gathered the following
(average total time) statistics from 10
replications of Model 1 and Model 2.
Test whether these two models are
significantly different from each other
with respect to the average total time
performance measure.



Carry out the hypothesis test
Construct the C.I. and observe that the same
conclusion is obtained due to the duality of
confidence intervals and hypotheses tests.
#
Model 1
Model 2
1
14.1443
13.3307
2
11.5068
10.2171
3
10.6117
10.6563
4
14.2636
10.2678
5
23.0772
10.3082
6
28.2713
12.1404
7
11.1804
11.3499
8
15.0567
9.0096
9
11.8765
9.0139
16.0089
10.1748
Check ttest.xlsx for calculations & result. 10
13
Output Analyzer

Main uses:




Comparing means / variances (hypothesis tests)
Computing confidence intervals
Plotting histograms, charts, moving averages
Consider Call Center Model.

Compute confidence intervals for



14
Average number in Process Product Type 1’s Queue
Average time in system
Let the level of significance be 0.05.
Call Center Example
15
Call Center Example

Suppose that we are allowed to hire 7 new personnels.



Scenario 1: Hire 3 Sales person, and 4 Tech All
Scenario 2: Hire 1 Sales person, and 6 Tech All
Since “0” is an element of the C.I, there is no statistical evidence
that one scenario is better than the other.
16