Olivero, Pavlik, Sierra, Tessier
November 4, 2009
ME 6105
HW4: Uncertainty Analysis
Task 1: Become Familiar with ModelCenter
Complete.
Task 2: Identify and Model the Uncertainty in your Design Analysis
Model
We identified ten variables that we believe contribute to the uncertainty of the measure of
effectiveness of our model. We believe that each variable may contribute to the overall model and we
would like to see how much each variable affects the overall uncertainty of the model. To determine this
we used the DOE tool in Model center and conducted a Case Study. The following variables were
considered:
Person height and weight were taken from actual population specifications. Both of these factors are
represented by a normal distribution.
Ratio: Defined as the ratio of real calf left divided by the total length of the calf. Due to little
knowledge about the distribution of this number, we can only assume that the prosthetic will work for a
ratio of 75% or less to allow for enough room for the ankle spring and damper hardware. It also will
need to be at least 25% to create a solid connection.
Foot Mass: Mass of the prosthetic foot alone, with the toe portion accounting for 10% of the total mass.
Gait Ratio: Gait ratio is defined as the ratio of the gait length to a nominal gait length. Increasing the
gait ratio increases the stride. Our Our model is only designed for a transtibial prosthetic for walking,
separate prostheses exist for the purpose of running.
Period: Defined as the time (in seconds) for one full gait cycle. Increasing the period slows the gait.
The spring and damper numbers were taken from actual tolerances from hardware.
We elicited our beliefs on what the low and high values for the variables were based off of data which
we obtained from papers and manufacturers’ specifications. The table below has all ten variables with
their corresponding nominal values that were used in the model, their low and high values and our
reasoning for our elicitation.
Variable
Person’s
Height
(meters)
Person’s
Weight
(kilograms)
Ratio of
Amputated
Leg
Gait Ratio
Gait Period
(seconds)
Prosthetic
foot Weight
(kilograms)
Toe Spring
Constant
Toe Damper
Constant
Ankle Spring
Constant
Ankle
Damper
Constant
Nominal
Value
1.759
80.42
.50
1
1.063
3.5
150
10
100
90
Low
Value
1.649
66.22
.25
.9
.5315
3.15
138
9.2
92
82.8
Hi
Value
1.869
96.41
.75
1.1
2.126
3.85
162
10.8
108
97.2
Elicitation Reasoning
These are the values for the 5th, 50th, and 95th percentile
for average male human height which were obtained from
literature.
These are the values for the 5th, 50th, and 95th percentile
for average male human weight which were obtained from
literature.
The values were determined by eliciting our beliefs on
what we thought would be the minimum and maximum
amount of leg, below the knee, that can be left after
amputation so that the prosthetic could be attached.
These values were determined by eliciting our beliefs
regarding what we thought would be a reasonable small
gait and large gait. Our lower bound was limited by a value
that would not crash the model.
The values were determined by eliciting our beliefs on
what would be a quick gait period that was not quite yet to
the point of running and a slow gait period that we would
consider to still be walking.
We elicited our beliefs as to how much a manufacturing
process to make the prosthetic foot would vary the weight.
We determined that there would be a ±10% tolerance to
the weight of the foot.
We used data from springs that were found on McMasterCarr. We found that most springs had a tolerance of ±8%.
We used data from dampers and dashpots that were found
on McMaster-Carr. We found that most dampers had a
tolerance of ±8%.
We used data from springs that were found on McMasterCarr. We found that most springs had a tolerance of ±8%.
We used data from dampers and dashpots that were found
on McMaster-Carr. We found that most dampers had a
tolerance of ±8%.
Variables nominal value, low value, hi value and elicitation reasoning
A Sensitivity Analysis was conducted using a central composite experiment to determine the Main
Effects. With these results the three most important sources of uncertainty were determined. Below is a
figure of the DOE tool GUI that was used to define which input and output variables were going to be
run in the Central Composite experiment.
Probabilistic GUI for the Sensitivity Analysis
Once the Central Composite experiment completed running the sensitivity of each input variable was
determined with respect to each output variable. There were three input variables that had the most
affect on all three output variable. They were gait period, gait ratio, and the mass of the foot. Each one
of these variables had different sensitivities for the different individual outputs but consistently gait
period had the strongest affect on the output variables, and the gait ratio and the mass of the foot also
affected the outputs by a significant amount. The figures below show the different sensitivities of the
input variables for the outputs.
The input variable that affected the three outputs the most was the gait period. By shortening the gait
period the model was able to make more steps within the given simulation time and lengthening the
gait period lowered the amount of steps the model took. Since the gait period affected the amount of
steps the model took it made sense that this would affect the output variables the most. For example
the more steps the model made the more average power it would have to use during that simulation
time. In addition the more steps the model took the further it was going to walk within the given
simulation time. The increase in the gait period causes the model to slow down in walking and which
then allows the toe on the prosthetic to catch the ground and cause the model to hop. This drastically
limits the distance the model is able to walk and it affects the vertical displacement since it causes the
model to hop. The same issue occurs when the model’s gait period is decreased to a smaller time. The
prosthetic’s toe gets caught on the ground multiple times which affects the vertical displacement and
the distance it is able to walk.
The second input variable that affects the outputs is the gait ratio. The gait ratio affects the distance
covered by each gait. When the gait ratio is one it will take the average gait distance that the model
takes. When the gait ratio is lowered the model takes smaller steps and when it is increased the model
takes larger steps. By modifying this number all three of the outputs were affected. When the gait ratio
is lowered the toe would sometimes catch the ground causing the model to skip. By increasing the gait
ratio the model walked a much greater distance and the model uses a greater amount of power than
when it is taking smaller steps.
The third most sensitive input variable was the mass of the foot. The variance of the foot mostly affects
the amount of power the person generates and the walking distance. It affects the vertical displacement
the model travels but not as much as gait period and gait ratio. By increasing or decreasing the mass of
the foot the model has to output more or less power to move the foot the distance it desires. In the
increase in the mass of the foot lowers the amount of distance the model walks.
Sensitivity Analysis for distance
Sensitivity Analysis for power
Sensitivity Analysis for max vertical displacement
Task 3: Elicit a Detailed CDF for the Most Significant Uncertain Variables
Danny – Gait Ratio
The sensitivity analysis determined that the gait ratio had a significant affect on the outputs. By varying
the distance between each step the model made, several of the outputs were affected. Since it is one of
the important variables we elicited our beliefs to determine what kind of a distribution it would actually
produce. For the gait ratio we took into consideration the normal ratio our model walks at, the
minimum which we believe nobody would walk at and the maximum gait ratio before we considered it
to be running.
The first question we asked was: What is the maximum gait ratio a person can take before considering it
running? We determined through our beliefs that this number would be two times the normal gait.
From this answer we determined that everyone walks with a gait ratio smaller than twice the average
model gait ratio. Next we asked: What is the smallest gait ratio which people can walk at? In reality this
can go to zero but our model would crash at such a small number. We determined through running the
model that the smallest gait ratio that the model could work at was 0.8 times the normal gait ratio. For
the gait distance that corresponds to our ratio of one, we determined that we could say that 25% of
people walk at or below this point. We elicited this belief since it is a small value and after us taking
some simple calculations of our own gait cycle we realized that none of us walked with that length of a
gait. We also elicited the point at which we believed 75% of people walk below. We determined that at
roughly 1.1 times the gait was where 75% of people walked at. Once these points were elicited we
plugged into the CDF spreadsheet and came up with our CDF spline. The figure below shows the CDF
curve that was created with the elicited points. When the PDF is examined the distribution seems to be
slightly skewed to the right. The figure below shows the PDF curve.
CDF curve for gait ratio.
PDF curve for gait ratio
Beth –Gait Period
CDF - Gait Period
1.0
0.9
0.8
Probability
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0
1
2
X values
3
(Note: You can select the blue dots and drag them around
CDF for Period
4
From literature we discovered that the average gait period is 1.063 seconds. From here we decided that
a person would probably not have a normal gait 1/3 or 3 times the nominal gait, so we set these as
utilities of 0 and 1 respectively. This means that 0 people have a gait period of .3 seconds or less and
everyone has a gait period less than 3 seconds. From here we decided that the 5 and 95 percentiles are
at half the nominal and twice the nominal gait period. We set the 5th percentile by looking at how the
model behaves with this period. At this point, the model looks like it is on the edge of walking and
running. Since our prosthesis is only designed for walking, this is the upper limit we would like to
simulate. The upper limit is set at twice a nominal gait period using the same logic. If half of the
nominal causes a person to almost run, twice the gait period must cause a person to something less
than walking, which we would not like to simulate.
PDF - Gait Period
1.2
Probability Density
1
0.8
0.6
0.4
0.2
0
0
1
2
X values
3
4
PDF for gait period
The PDF for gait period is represented by a positive skew. This means that it is more probable that a
person will walk with a gait period less than nominal, meaning that a user of the prosthetic will probably
walk with a slower than nominal gait.
Carlos - Foot Mass
After a group discussion on the average value a prosthetic foot will weight, based on the density of the
prosthetic foot material (steel) and based on our estimated value for its volume, an agreement on a
prosthetic foot’s weight of 3.5 kg was reached. Knowing that manufacturing accuracy is high nowadays,
we can expect the actual weight of the foot to be between 10% of the estimated value (3.15 kg-3.85 kg).
In order to construct the CDF, the two extreme values to be considered will be 3 as the lower bound,
since we are sure the weight of the foot will never be less than that value, and 4 for the upper bound
since we are sure it will not exceed that value.
Elicitation questions:
1.
How much am I willing to pay to enter a gamble in which I will receive $1 if the prosthetic foot is
less than 3.8 kg and $0 if it is more than that? Knowing that manufacturing methods are highly
accurate nowadays, it will be logical to bet a high amount of money considering the we are at
the higher end of the 10% manufacturing confidence interval. For this reason, a bet of $0.90 is
logical.
2. How much am I willing to pay to enter a gamble in which I will receive $1 if the prosthetic foot is
less than 3.3 kg and $0 if it is more than that? In this case the odds of winning are much less.
Again taking into account manufacturing accuracy and the fact that we are not very close to the
estimated value the foot’s mass should have, a bet of only $0.20 will be logical. The fact we are
still within the confidence interval, helped to raise the amount of the bet.
3. How much am I willing to pay to enter a gamble in which I will receive $1 if the prosthetic foot is
less than 3.6 kg and $0 if it is more than that? Here the odds of winning are higher than the
odds of loosing but for a minimal amount. A bet of $0.70 will be too high since I will only get an
extra $0.30 cents but risk losing $0.70. On the other hand, a bet of $0.50 seems to low since I
still have more chances of winning that loosing. For these reasons a bet of $0.60 seems
reasonable.
4. How much am I willing to pay to enter a gamble in which I will receive $1 if the prosthetic foot is
less than 3.4 kg and $0 if it is more than that? In this scenario the odds of loosing are higher
than the odds of winning but not dramatically. Taking into account that we are well within the
manufacturer confidence interval, a bet of $0.30 seems reasonable. $0.40 seems too high since
I will be getting only an extra $0.20 if I win. A bet of $0.20 will be to low since this is what I bet
in the case of the 3.3 kg weight which had even less chances of winning. All the different weight
considered and their corresponding bets are presented below in Table 1.
Table1. Prosthetic Foot’s Weights and Bet amounts.
Prosthetic Foot’s Weight (Kilograms)
3.8
3.3
3.6
3.4
4.0
3.0
Bet (dollars)
0.90
0.20
0.60
0.30
1.0
0.0
Using the CDF tool provided, a plot of the Cumulative Distribution Function has been generated
as presented below.
The cumulative distribution presented above appears to have an inflection point near the 3.5 Kg mark.
This is in fact what we expected it to have since the probability of manufacturing a foot with that specific
weight is highest at that point. It is also noted that there is no very steep regions or abrupt changes in
CDF which is expected since we are stating our beliefs only taking into account the expected weight of
the foot and the manufacturer’s confidence intervals. In this case, any abrupt change would have been
an indication of irrational behavior. Finally, it can be noted that the graph is symmetrical. This is
reasonable since the is no reason to suspect there will be more chances of manufacturing a foot with a
higher weight that there is chances of manufacturing one with a lower weight.
Sean – Height and Weight
For the height and weight variables, we had the 5th, 50th, and 95th percentile values from the same
anthropometric data source we used to construct the relative lengths and masses of the various body
parts. We used the data for a male as that is what are model was designed with. To make the CDF more
accurate, 8 additional points were used based on beliefs held about height. Values were expressed in
feet and inches during the initial elicitation, and then converted to meters to work with the model. The
first step was to determine the 0 and 100 percentile points on the CDF. The zero point was chosen as
1.22m (4 ft). There are shorter people and children out there, but we based our model on average adult
data, so our model is likely no longer valid at values this low. Likewise, the maximum value was chosen
as 2.4m (7ft 10in). There are a few people in the world taller than this, but the amount is so small that it
is statistically zero. Likewise, the 1st and 99th percentile were determined chosen as 1.52m (5 ft) and
2.05m (6ft 9in) respectively. These were chosen because there are very few people who fall outside of
this range, and the CDF needed to be adjusted to more accurately represent the population. To finish
the graph, the 10th, 30th, 70th, and 90th percentiles were elicited. To determine these, I asked the
questions “What percentage of the population do I believe is shorter than this height?” and also by
considering the opposite “What percentage of the population do I believe is tall than this?” So for each
percentile, values were chosen until the answer felt satisfactory. For the 10th percentile, it is fair to say
that 90% of males are taller than 1.68m (5ft 6in). For the 30th percentile, it is fair to say that 70% of
males are taller than 1.73m (5ft 8in) or that only 30% are shorter than that height. For the 70th
percentile, it is fair to say that 70% of males are shorter than 1.78m (5ft 10in) or that only 30% are taller
than that height. For the 90th percentile, it is fair to say that 90% of males are shorter than 1.83m (6ft).
The PDF resembles a normal distribution, which is to be expected for the height of the male population.
Cummulative Distribution Function
1.0
0.9
0.8
Probability
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
1
1.5
2
2.5
X values
Height CDF
The CDF looks how one would expect. The slope is at the area closest to the average value. This makes
sense because most males are about the same height and are only a few inches different in height.
However, as the graph reaches the more extreme values, the slope becomes flat very quickly. This too
makes sense, because as you approach the extremes of adult height, you expect the number of people
outside of these values to decrease quickly.
Task 4: Determine the Distribution of the Output of your Model
Snapshot of Probabilistic Analysis GUI
See rest of input variables in window below.
Snapshot of ModelCenter model and variable browser
Distance (m)
400
350
300
250
200
150
100
50
0
-0.1 1.9 4.0 6.0 8.0 10.0 12.1 14.1 16.1 18.1 20.2 22.2 24.2 26.2 28.3
Histogram of distance travelled over 5 seconds
The histogram for distance travelled has two peaks. We expected to see a normal distribution centered
around the nominal distance travelled at around 6 meters. Since we did 2000 runs, it is possible that
more runs (probably around 10,000) would give a normal distribution. In Monte Carlo analysis, the
more runs you do, the less course the histogram looks and less it changes. Our simulation took about a
minute per run, so doing 10,000 or more was not possible, especially due to continual crashing due to
lack of memory and spontaneous Windows updates.
Note: Due to the length of the simulation, we broke up the runs and each did a set, which is why our
trade study is just a few samples out of the 2000.
Y Displacement (m)
1600
1400
1200
1000
800
600
400
200
0
0.931.271.611.952.282.622.963.303.633.974.314.644.985.325.665.996.33
Histogram of maximum y displacement over 5 seconds. Y displacement is measured at the hip. This
value represents the amount of bouncing that occurs when walking.
Y displacement does not show much variation. This means that the tolerances we put into the
simulation do not have much of an effect on the amount of “bounciness” a person will experience while
wearing the prosthetic. This is beneficial for marketing of the prosthetic because we can guarantee
minimal “bounciness” for a range of walking speeds and gait lengths.
Power (W)
1600
1400
1200
1000
800
600
Frequency
400
200
0
Histogram of average power generated at hip. This represents the assistive power required by the user
to move the prosthetic.
The histogram for power is mostly represented by a constant of 11 kW. This makes sense because the
user will have to generate the same amount of power regardless of rate, the only variable that would
change this number is a change in mass. Since the mass of the prosthetic foot is only variable by
manufacturing tolerances, it makes sense that there is not much variation in power. However, the value
of 11kW does not make sense. This is far more than the average human can generate. There is
probably something wrong with the model, or placement of power sensor, that needs to be altered.
Task 5: Lessons learned
Danny:
While working on this project I was able to learn and understand the programs of Dymola
and Model Center and I have been more fluent at using Modelica. After HW 3, as a group,
we had thought that this HW assignment would not be that difficult or time consuming, but
after trying to get the desired outputs for model center proved to be challenging, we
realized that this was going to be much more difficult than we had previously thought. One
lesson that I learned was that if the model works by itself it doesn’t mean that we are easily
going to get the outputs that we desire. As I found out there were multiple manipulations
and modifications that were needed to be done in order to get the outputs that we wanted
to measure. A simple task such as the average power ended up being an extremely time
consuming task. After figuring out how to modify the model I was able to understand how
to use Model Center and run simulations in it. I can now say that I am fluent in creating
wrappers for the model to run in model center and I can now set up a Sensitivity Analysis
and conduct a Monte Carlos simulation
This HW has also allowed me to apply the concepts which we have discussed in class such
as eliciting our beliefs to come up with our CDFs and I now better understand the concepts
of uncertainty in design. I now feel like I can actually use model center comfortably. One
thing that I found that was very helpful and useful was the discussion session you held
during class. It was extremely helpful, and I believe it is a format that you should adopt for
the course next semester. I feel as though people will be able to better understand the
concepts, software, and the projects much better this way.
Beth:
The most difficult part of this assignment was changing the Modelica model to access the input and
output variables to interface with ModelCenter. We had trouble defining global variables and using
them in various mathematical functions to output the value we desired. What we learned is that
Modelica is actually much more straightforward than we had thought. At first we tried writing scripts to
get average power by skipping the first period and integrating, but once that failed we learned from
Chris that we just needed to set up a few blocks and connect them. This is much more intuitive than we
are used to dealing with in a program like Matlab. We also had some coding problems with the
wrapper, which we never fully figured out. Most of the problems were fixed by simply starting over
again, though this is common with coding where a mistake like leaving out a semi colon is not blatantly
obvious.
This assignment took more time than we had anticipated. This is mostly due to our lack of knowledge
when it comes to interfacing these two programs and working with unfamiliar code. There were a few
times where I spent a few hours on a problem, only to find the solution that should have taken a few
minutes to fix if I had sat and thought about it logically. The most important lesson I learned from this
assignment was that I should take my time troubleshooting. If the model does not work the way it had
only moments before, there is probably a simple fix. For instance, I was entering the foot weight as an
output when all of a sudden the model started tripping. I assumed that I had broken some of the basic
functionality, and four hours later I realized that I hadn’t appropriately accounted for the decreased
weight of the toe. When the total mass of the foot was placed on the toe, the spring could not hold up
the entire weight, causing it to drag on the ground and trip the model. This is plainly obvious in
hindsight, but at the time I tried reworking the model.
I expected our system to behave in a stiff manner, meaning that a small change in the spring and
damper constants causes a large difference in the behavior of the system. However, the springs and
dampers could vary by a large amount and still result in a working model. I learned a lot from this
assignment because of the manner in which we approached the problem. Instead of jumping in and
varying a lot of different things, we made small tweaks to make sure the model still worked. In the end,
our model is much improved with outputs that are measurable and make sense with regard to our
objectives of making a prosthetic that increases user mobility.
Carlos:
In this assignment, several things presented challenges in different magnitudes. Trying to get
model center to work was relatively easy but creating the function wrappers was a bit more
confusing. I had trouble understanding how a program such as model center will “use” another
software such as “dymola” to perform simulations. The concept of saving an executable file
and then use it in software was new to me. Coming out with different bets under different
conditions was somewhat challenging since it required me to think through the problem deeply
and make sure I was not being biased. This is also the first time I get exposed to Monte-Carlo
simulations. I was having trouble understanding how it worked and especially the fact that we
could divide the simulation and then combine the results. I though 1 person was going to run
the simulation from one interval, say 0 to 250, and another person from 250-500 and so on and
the combine the results in a logical way. The issues came from trying to understand how a
person could run a simulation for a certain interval without having information on past results.
It turned out to be a lot simpler that I expected it to be the fact that the software simulates for
randomly chosen values and ADDs up the results in a histogram clearly explains why the
simulation could be divided as it was.
From my experience with this assignment, I believe the approach taken was correct. Normally,
it is difficult to understand the uncertainty in the analysis and even more difficult to estimate
the effect that uncertainty will have in our final calculations. Going through the process of
listing all variables and uncertainties provided an opportunity to deeply thing about our model
and understand the specifics of its different components. The performing the sensitivity
analysis allowed to focus only in the most important variables, which I think was the correct
approach since I would not make sense to expend a great amount of time trying to understand
the specifics of a variable that does not have a lot of impact in our desired result.
I was surprised to see that the power variance was so low for the many different input
combinations performed in the Monte Carlo. I would have expected these values to depend
greatly on the person’s weight and displacement as well as damper and spring coefficients and
therefore be a little more spread out. This assignment, and all the simulations it involves,
helped me to verify the robustness of our model and be confident to be in the right track for
the optimization assignment coming next. Taking into account the complexity of the model, I
expected it to be more unstable and I am definitely pleased to its performance in this
assignment
Sean:
The first thing that I learned was that eliciting uncertainties is not always simple. Even if you believe
uncertain value follows a normal distribution, you cannot always be sure. One would expect that a part
made with a tolerance would follow a normal distribution with the tolerances being the ends of the
distribution, however, manufacturing experience tells me that is not always true. The most important
thing I learned is that even if your model is great and you specify the values of everything, there will
always be uncertainties in those values and they will compound in your results. I learned how to use a
Monte Carlo approach to construct the distribution of the output to determine the effect of input
uncertainty. Initially, I had a difficult time grasping how to construct the CDF’s, because I was confusing
it with optimization of variables. Once I realized it was only used to describe uncertainties due to things
like tolerances and small variations, the process became clearer.
From the results, I learned that there were still some issues with the implementation of our model that
needs to be resolved before we work on optimization. When height uncertainty had such a low impact
on the outputs, I suspected there might be an issue with the model, but unfortunately, I did not discover
it until it was too late to correct it and still finish this assignment. I would not make much changes to the
approach to this assignment, but I would change the approach to the previous assignment. Had I known
how Model Center would use the inputs and outputs, I would have created the model to accommodate
this from the start. Having to go back and modify the model to be compatible with the input of 10
variables took some time, and would have been easier to implement (and more importantly, debug) in
the previous assignment.
Task 6: Project web-page
http://www.srl.gatech.edu/education/ME6105/Projects/Fa09/bcds/