Development of Verification and
Validation Procedures for Computer
Simulation use in Roadside Safety
Applications
NCHRP 22-24
DEFINITIONS AND PROCEDURES
Worcester Polytechnic Institute
Battelle Memorial Institute
Politecnico di Milano
FEA IN ROADSIDE HARDWARE DESIGN
• Specialty codes prior to 1990 (i.e.,
NARD, Guard, BarrierVII, etc.)
• Shift to DYNA/LSDYNA in mid-1990’s
• Today
– Almost exclusively LSDYNA
– Used in nearly all new product
developments
– Requests for approval now coming
based partly or entirely on LSDYNA
results.
– Decision-makers need a way to judge
good from bad results. Who do you
trust? How do you make an
acceptance decision based on
simulations?
MOTIVATION
• We have formal standards on how to perform
and evaluate full-scale crash tests.
• Decision makers want a formal standard on
how to perform and evaluate FEA simulations
used in the approval process.
• NCHRP 22-24 was initiated to develop these
procedures.
• Develop a procedure and format for validation and
verification report for simulations that can be used like
a crash test report.
OBJECTIVE
… to develop guidelines for verification and validation
of detailed finite element analysis for crash
simulations of roadside safety features. The focus of
these guidelines will be on establishing accuracy,
credibility, and confidence in the results of crash test
simulations intended (1) to support policy decisions
and (2) to be used for approval of design
modifications to roadside safety devices that were
originally approved with full-scale crash testing.
Existing V&V Procedures
Several organizations have developed V&V procedures in recent years.
•
•
•
•
•
•
•
•
•
•
NASA • Lockhead
These are general purpose, broadbased guides that out-line general
DoD
procedures and provide
AIAA
terminology definitions. They are
not step-by-step guides.
Sandia
Los Almos
LLNL
ASME Guide draws on all the above guides in
developing its recommendations.
ASME
FHWA/NARD
These are domain-specific guides
with more step-by-step
FAA
• EU (rail)
procedures and defined metrics.
FRA
• Chrysler
ASME V&V 10-2006
History
1999
An ad hoc verification & validation specialty
committee was formed under the auspices of
the United States Association for
Computational Mechanics (USACM).
2001
ASME approved the committee’s charter:
To develop standards for assessing the
correctness and credibility of modeling and
simulation in computational solid mechanics.
2006
2007
Committee was assigned the title and
designation of the ASME Committee for
Verification & Validation in Computational
Solid Mechanics (PTC 60).
ASME published the “Guide for verification
and validation in computational solid
mechanics.” ASME V&V 10-2006.
Developing a series of “best practices
guides”
ASME V&V 10-2006
Committee Members
The committee derives its authority from the
diversity of its membership and consensus of opinion.
M. C. Anderson, Los Alamos National Laboratory
J. A. Cafeo, General Motors Corporation
R. L. Crane, The American Society of Mechanical Engineers
S. W. Doebling, Los Alamos National Laboratory
J. H. Fortna, ANSYS
M. E. Giltrud, Defense Threat Deduction Agency
J. K. Gran, SRI International
T. K. Hasselman, Acta Inc.
H. M. Kim, Boeing
R. W. Logan, Lawrence Livermore National Laboratory
H. U. Mair, Institute for Defense Analyses
A. K. Noor, Old Dominion University
W. L. Oberkampf, Sandia National Laboratories
J. T. Oden, University of Texas
D. K. Pace, Consultant
T. Paez, Sandia National Laboratories
A. B. Pifko, Consultant
L. Proctor, MSC Software
J. N. Reddy, Texas A & M University
P. J. Roache, Consultant
L. E. Schwer, Schwer Engineering
P. E. Senseny, Consultant
M. S. Shephard, Rensselaer Polytechnic Institute
D. A. Simons, Northrop Grumman
B. H. Thacker, Southwest Research Institute
T. G. Trucano, Sandia National Laboratories
R. J. Yang, Ford Motor Company
Y. Zhao, St. Jude Medical
10
ASME V&V 10-2006
•The Guide does provide a:
• Framework and process for V&V
activities.
• Standard definitions for V&V terms.
•The Guide does not provide:
• A step-by-step procedure for V&V.
• Specific recommendation for metrics.
ASME V&V 10-2006
Some Definitions
• Validation -- The process of determining the degree to
which a model is an accurate representation of the real
world from the perspective of the intended uses of the
model.
– Model results are compared to physical experiments.
• Verification -- The process of determining that a
computational model accurately represents the underlying
mathematical model and its solution.
– Model results are compared to known mathematical solution.
• Calibration -- The process of adjusting physical modeling
parameters in the computational model to improve
agreement with experimental data.
– Physical experiments used to estimate model parameters.
ASME V&V 10-2006
Validation
The process of determining the degree to which a
model is an accurate representation of the real
world from the perspective of the intended uses of
the model.
ASME V&V 10-2006
Verification
The process of determining that a computational
model accurately represents the underlying
mathematical model and its solution.
ASME V&V 10-2006
Calibration
The process of adjusting physical modeling
parameters in the computational model to
improve agreement with experimental data.
ASME V&V 10-2006
Model Development
Before we begin to develop a model, a reality of interest is
identified (i.e., what is the physical system to be modeled).
“We need to know the wing tip deflection of the ABC experimental aircraft under
a distributed load of X Newtons/meter,” in this case the reality of interest is the
aircraft wing.
The airplane is manufactured by Karel Klenor - KLN, Choceň, the firm
is one of the biggest producers of composites in the Czech Republic www.kln.cz
ASME V&V 10-2006
Model Development
Conceptual Model – “the collection of
assumptions and descriptions of physical
processes representing the solid mechanics
behavior of the reality of interest from which the
mathematical model and validation experiments
can be constructed.”
ASME V&V 10-2006
Model Development
Mathematical Model – “The mathematical
equations, boundary values, initial conditions, and
modeling data needed to describe the conceptual
model.”
 EI  x  y  w  x 


0 xL
y  0   y  0   y  L   y  L   0
ASME V&V 10-2006
Model Development
Computational Model – “The numerical
implementation of the mathematical model,
usually in the form of numerical discretization,
solution algorithm, and convergence criteria.”
Commercial Software
ASME V&V 10-2006
V&V Process
ASME V&V 10-2006
V&V Process
Verification
ASME V&V 10-2006
V&V Process
Verification
Validation
ASME V&V 10-2006
Validation Process
The goal of the validation process is to
assess the predictive capability of the
model by comparing the predictive
results of the model with validation
experiments.
Three key elements of Validation:
1. Precision Testing
2. Uncertainty Quantification
3. Comparative Metrics
25
ASME V&V 10-2006
Comments on the V&V Process
•
•
•
The V&V process diagram is valid not only for
whole models but for components,
assemblies, parts, etc.
While most roadside safety work uses
LSDYNA, this process and definitions are
applicable to any numerical simulation
software (e.g., MADYMO, BVII, HVOSM, HVE,
etc.).
We can not usually do code verification – we
do not generally have access to the code.
–
•
We can do calculation verification – this is
another word for benchmarking.
–
•
Counter example: Yvonne Murray’s soil and
timber models for LSDYNA.
Example: do different versions of LSDYNA
produce the same result? Do different
computational platforms produce the same
result?
Notice the comparison is quantitative.
–
Qualitative validation is not really validation
because it is subjective.
Recommendation
• The project team recommends that
we adopt the ASME V&V 10-2006
Guide as a basis for the basic V&V
process and definition of terms
because …
• The 22-24 panel definitions are more
or less consistent with the ASME
definitions and …
• The ASME definitions have been
adopted by a broad cross-section of
the computational solid mechanics
community and ...
• The ASME process includes all the
essential pieces needed in
computation roadside safety.
• Discussion?
Hierarchical Modelling
• The ultimate goal is to use a
validated model to
extrapolate results to an
untested situation.
• We need to have confidence
in the model before we can
use it to predict untested
situations.
Hierarchical Modelling
Barrier assembly
Vehicle assembly
Whole model
level
Top Rail assembly
Middle Rail assembly
Post part
Assembly
Level
Rubrail part
Guardrail part
Posts parts
Part Level
Spacer part
Blockout part
Main -rail part
Stiffner parts
Hierarchical Modelling
Hierarchical Modelling
Hierarchical Modelling