Available online at www.sciencedirect.com
ScienceDirect
IFAC PapersOnLine 54-1 (2021) 1132–1137
State-of- practice survey in industry on the deployment of simulation in systems
State-ofpractice survey
in industry
on
the
deployment
of
simulation
in
systems
State-ofon
the
deployment
of
simulation
in
systems
engineering
State-of- practice
practice survey
survey in
in industry
industry
on
the
deployment
of
simulation
in
systems
on
the
deployment
of
simulation
in
systems
State-of- practice survey in industry
engineering
engineering
engineering
engineering
K. E.BEMMAMI*.
P. DAVID**
K.
E.BEMMAMI*.
P.
DAVID**

K.
E.BEMMAMI*.
P.
DAVID**
K.
E.BEMMAMI*.
P.

K. E.BEMMAMI*.
E.BEMMAMI*.
P. DAVID**
DAVID**

* Univ. Savoie
Mont Blanc,
SYMME
Annecy, France
K.
P.
DAVID**

** Univ.
Savoie
Mont
Blanc,
SYMME
Annecy,
France
 kamel-eddin.bemmami@univ-smb.fr)
(Tel: +33 (0)
751-305-271
; e-mail:
Univ.
Savoie
Mont
Blanc,
SYMME
Annecy,
France
** Univ.
Savoie
Mont
Blanc,
SYMME
Annecy,
France
(Tel:
+33
(0)
751-305-271
;
e-mail:
kamel-eddin.bemmami@univ-smb.fr)
Univ.
Savoie Alpes,
Mont
Blanc,
SYMME
Annecy,
France
**Univ.
Grenoble
CNRS,
G-SCOPAnnecy,
Grenoble,
France
*
Univ.
Savoie
Mont
Blanc,
SYMME
France
(Tel:
+33
(0)
751-305-271
;
e-mail:
kamel-eddin.bemmami@univ-smb.fr)
(Tel:
+33
(0)
751-305-271
;
e-mail:
kamel-eddin.bemmami@univ-smb.fr)
**Univ.
Grenoble,
France
Grenoble
Alpes,
CNRS,
G-SCOP
(Tel:
+33
(0)
751-305-271
;; e-mail:
kamel-eddin.bemmami@univ-smb.fr)
(Tel:
+33
(0)
476-574-328
; e-mail:
pierre.david@grenoble-inp.fr)
(Tel:
+33
(0)
751-305-271
e-mail:
kamel-eddin.bemmami@univ-smb.fr)
**Univ.
Grenoble
Alpes,
CNRS,
G-SCOP
Grenoble,
France
**Univ.
Grenoble
Alpes,
CNRS,
G-SCOP
Grenoble,
France
(Tel:
+33
(0)
476-574-328
;; e-mail:
pierre.david@grenoble-inp.fr)
**Univ.
Grenoble
Alpes,
CNRS,
G-SCOP
Grenoble,
France
**Univ.
Grenoble
Alpes,
CNRS,
G-SCOP
Grenoble,
France
(Tel:
+33
(0)
476-574-328
e-mail:
pierre.david@grenoble-inp.fr)
(Tel:
+33
(0)
476-574-328
;
e-mail:
pierre.david@grenoble-inp.fr)
(Tel:
+33
(0)
476-574-328
;
e-mail:
pierre.david@grenoble-inp.fr)
(Tel: +33 (0) 476-574-328 ; e-mail: pierre.david@grenoble-inp.fr)
Abstract: Model-based systems engineering (MBSE) has been proposed as an approach to manage the
Abstract:
Model-based
systems
engineering
(MBSE)
has
been
proposed
approach
manage
the
complexity
of modern product
development
through the
continuous
use as
of an
models.
The to
use
of model
Abstract:
Model-based
systems
engineering
(MBSE)
has
been
proposed
as
an
approach
to
manage
the
Abstract:
Model-based
systems
engineering
(MBSE)
has
been
proposed
as
an
approach
to
manage
the
complexity
of
modern
product
development
through
the
continuous
use
of
models.
The
use
of
model
Abstract:
Model-based
systems
engineering
(MBSE)
has
been
proposed
as
an
approach
to
manage
the
simulation
is
a
core
principle
of
the
MBSE
approach.
In
the
first
stages
of
projects,
it
supports
defining
Abstract:
Model-based
systems
engineering
(MBSE)
has
been
proposed
as
an
approach
to
manage
the
complexity
of
modern
product
development
through
the
continuous
use
of
models.
The
use
of
model
complexity
of
modern
product
development
through
the
continuous
use
of
models.
The
use
of
model
simulation
is
a
core
principle
of
the
MBSE
approach.
In
the
first
stages
of
projects,
it
supports
defining
complexity
of
modern
product
development
through
the
continuous
use
of
models.
The
use
of
model
the
expected
system
features,
when
in
the
later
stages
it
estimates
the
dynamic
behavior.
Simulation
is
complexity
of
modern
product
development
through
the
continuous
use
of
models.
The
use
of
model
simulation
is
a
core
principle
of
the
MBSE
approach.
In
the
first
stages
of
projects,
it
supports
defining
simulation
is
aa core
principle
of
the
MBSE
approach.
In
the
first
stages
of
projects,
it
supports
defining
the
expected
system
features,
when
in
the
later
stages
it
estimates
the
dynamic
behavior.
Simulation
is
simulation
is
core
principle
of
the
MBSE
approach.
In
the
first
stages
of
projects,
it
supports
defining
pushed
to
obtain
results
earlier
and
cheaper
than
with
testing
and
prototyping.
However,
the
development
simulation
is
a
core
principle
of
the
MBSE
approach.
In
the
first
stages
of
projects,
it
supports
defining
the
expected
system
features,
when
in
the
later
stages
it
estimates the
dynamic
behavior.
Simulation
is
the
expected
system
features,
when
in
the
later
stages
it
the
dynamic
behavior.
Simulation
is
pushed
to
obtain
earlier
and
cheaper
than
with
testing
However,
the
development
the
expected
system
features,
when
in
theexpensive
later
stages
it estimates
estimates
thesimulation
dynamic
behavior.
Simulation
is
of simulation
canresults
be features,
a very
tedious
and
task.
Theand
useprototyping.
ofthe
is a specific
part of an
the
expected
system
when
in
the
later
stages
it
estimates
dynamic
behavior.
Simulation
is
pushed
to
obtain
results
earlier
and
cheaper
than
with
testing
and
prototyping.
However,
the
development
pushed
to
obtain
results
earlier
and
cheaper
than
with
testing
and
prototyping.
However,
the
development
of
simulation
can
be
a
very
tedious
and
expensive
task.
The
use
of
simulation
is
a
specific
part
of
an
pushed
to obtain
obtain
results
earlier
and
cheaper
thanproject
withtask.
testing
and
prototyping.
However,
thesource
development
MBSE
approach
that
is very
crucial
in an
efficient
management
since
it might
be
thethe
ofofhigh
pushed
to
results
earlier
and
cheaper
than
with
testing
and
prototyping.
However,
development
of
simulation
can
be
a
tedious
and
expensive
The
use
of
simulation
is
a
specific
part
an
of
simulation
can
be
very
tedious
and
expensive
task.
The
simulation
is
aa specific
part
of
an
MBSE
approach
that
crucial
in
efficient
management
it
might
the
source
of
simulation
canneeds
be aaais
very
tedious
and
expensive
task.
The use
use of
ofsince
simulation
isbe
specific
partof
ofhigh
an
costsimulation
and
skillscan
but
also
aan
source
of project
precious
knowledge
on
the
future
Simulation
of
be
very
tedious
and
expensive
task.
The
use
of
simulation
is
asystem.
specific
part
of
an
MBSE
approach
that
is
crucial
in
an
efficient
project
management
since
it
might
be
the
source
of
high
MBSE
approach
that
is
crucial
in
an
efficient
project
management
since
it
might
be
the
source
of
high
cost
and
skills
but
also
aaan
source
of
precious
knowledge
on
the
future
system.
Simulation
MBSE
approach
that
is
crucial
efficient
project
management
it
might
be
the
of
high
opportunities
areneeds
numerous,
butin
the
project
manager
must
identify since
the
more
relevant
forsource
their
project.
MBSE
approach
that
is
crucial
in
an
efficient
project
management
since
it
might
be
the
source
of
high
cost
and
skills
needs
but
also
source
of
precious
knowledge
on
the
future
system.
Simulation
cost
and
skills
needs
but
also
aa source
ofmanager
precious
knowledge
on
the
future
system.
Simulation
opportunities
are
but
the
project
must
identify
the
relevant
for
their
project.
cost
and
needs
but
precious
knowledge
on
the
system.
Simulation
This paper
aims
atnumerous,
documenting
the
currentof
state
of practice
in industry
on future
the
usage
of simulation
in
cost
and skills
skills
needs
but also
also
a source
source
ofmanager
precious
knowledge
on more
the
future
system.
Simulation
opportunities
are
numerous,
but
the
project
must
identify
the
more
relevant
for
their
project.
opportunities
are
numerous,
but
the
project
manager
must
identify
the
more
relevant
for
their
project.
This
paper
aims
at
documenting
the
current
state
of
practice
in
industry
on
the
usage
of
simulation
in
opportunities
are
numerous,
but
the
project
manager
must
identify
the
more
relevant
for
their
project.
MBSE
processes.
The
paper
presents
a
survey
conducted
amongst
French
companies,
on
how
they
apply
opportunities
are
numerous,
but
the
project
manager
must
identify
the
more
relevant
for
their
project.
This
paper
aims
at
documenting
the
current
state
of
practice
in
industry
on
the
usage
of
simulation
in
This
paper
aims
atThe
documenting
the current
state
of
practice
in
industry
on
the usage
of
simulation
in
MBSE
processes.
paper
presents
aa survey
conducted
amongst
French
companies,
on
how
they
apply
This
paper
aims
documenting
current
state
of
practice
in
industry
on
usage
of
simulation
in
MBSE,
Verification
Validation
& the
Testing
VVT,
and
perceived
benefits
and
barriers,
and
This
paper
aims at
atThe
documenting
the
current
state
ofsimulation.
practice
inThe
industry
on the
the
usage
of
simulation
in
MBSE
processes.
paper
presents
survey
conducted
amongst
French
companies,
on
how
they
apply
MBSE
processes.
The
paper
presents
a
survey
conducted
amongst
French
companies,
on
how
they
apply
MBSE,
Verification
Validation
&
Testing
VVT,
and
simulation.
The
perceived
benefits
and
barriers,
and
MBSE
processes.
The
paper
presents
a
survey
conducted
amongst
French
companies,
on
how
they
apply
the
parameters
influencing
VVT
strategies
and
the
use
of
simulation
are
alternately
analyzed.
MBSE
processes.
The
paper
presents
a
survey
conducted
amongst
French
companies,
on
how
they
apply
MBSE,
Verification
Validation
&
Testing
VVT,
and
simulation.
The
perceived
benefits
and
barriers,
and
MBSE,
Verification
Validation
&
Testing
VVT,
and
simulation.
The
perceived
benefits
and
the
parameters
influencing
VVT
and
the
use
of
simulation
are
alternately
analyzed.
MBSE,
Verification
Validation
&strategies
Testing
VVT,
and
simulation.
The
perceived
benefits
and barriers,
barriers, and
and
MBSE,
Verification
Validation
&
Testing
VVT,
and
simulation.
perceived
benefits
and
barriers,
and
the
parameters
influencing
VVT
strategies
and
the
use
of
simulation
alternately
analyzed.
license
BY-NC-ND
CC
theare
under The
article
access
is an open
This
Authors.
The
2021
©
Copyright
the
parameters
influencing
VVT
strategies
and
the
use
of
simulation
are
alternately
analyzed.
Keywords:
Model-based
Systems
engineering
(MBSE),
Simulation,
State
of
practice
the
parameters
influencing
VVT
strategies
and
the
use
of
simulation
are
alternately
analyzed.
the
parameters
influencing
VVT
strategies
and
the
use
of
simulation
are
alternately
analyzed.
licenses/by-nc-nd/4.0)
(http://creativecommons.org/
Keywords:
Model-based
Systems
engineering
(MBSE),
Simulation,
State
of
practice
Keywords:
Model-based
Systems
engineering
(MBSE),
Simulation,
State
of
practice
Keywords:
Model-based
Systems
engineering
(MBSE),
Simulation,
State
of

Keywords: Model-based
Model-based Systems
Systems engineering
engineering (MBSE),
(MBSE), Simulation,
Simulation, State
State of
of practice
practice
Keywords:
practice

1. INTRODUCTION
A significant amount of work has been devoted to develop


1.
INTRODUCTION
A
significant
amount
of
work
has
been
devoted
to
develop

the
modelling amount
infrastructure
so that
multi-domain
knowledge
1. INTRODUCTION
A
significant
of
work
has
been
devoted
to
develop
INTRODUCTION
A
significant
amount
of
work
has
been
devoted
to
develop
the
modelling
infrastructure
so
that
multi-domain
knowledge
Mastering the use of1.
simulation
tools
in
Systems
engineering
1.
INTRODUCTION
A
significant
amount
of
work
has
been
devoted
to
develop
can
be centrally
managed
and
shared
among
stakeholders.
1.simulation
INTRODUCTION
A
significant
amount
of
work
has
been
devoted
to
develop
the
modelling
infrastructure
so
that
multi-domain
knowledge
Mastering
the
use
of
tools
in
Systems
engineering
the
modelling
infrastructure
so
that
multi-domain
knowledge
can
be
centrally
managed
and
shared
among
stakeholders.
(SE)
processes
is
a
crucial
subject
for
engineers.
The
digital
Mastering
the
use
of
simulation
tools
in
Systems
engineering
the
modelling
infrastructure
so
that
multi-domain
knowledge
However,
the
simulation
capacity
needed
to
experiment
a full
the
modelling
infrastructure
so
that
multi-domain
knowledge
can
be
centrally
managed
and
shared
among
stakeholders.
Mastering
the
use
of
simulation
tools
in
Systems
engineering
(SE)
processes
is
a
crucial
subject
for
engineers.
The
digital
can
be
centrally
managed
and
shared
among
stakeholders.
Mastering
the numerous
use
ofcrucial
simulation
tools
inengineers.
Systems
engineering
However,
the simulation
simulation
capacity
needed
to experiment
experiment
a full
full
world
brings
advantages
inin
terms
of engineering
information
Mastering
the
use
of
simulation
tools
Systems
(SE)
processes
is
a
subject
for
The
digital
can
be
centrally
managed
and
shared
among
stakeholders.
system
dynamic
behaviour
is
rarely
available
at
companies
can
be
centrally
managed
and
shared
among
stakeholders.
However,
the
capacity
needed
to
a
(SE)
is
subject
for
engineers.
The
digital
world
brings
numerous
advantages
in
terms
of
information
the
simulation
capacity
needed
to
experiment
a
full
(SE) processes
processes
is aaa crucial
crucial
subject
for
engineers.
The
digital However,
system
dynamic
behaviour
is
rarely
available
at
companies
management,
processing,
and
visualization,
but
demands
(SE)
processes
is
crucial
subject
for
engineers.
The
digital
world
brings
numerous
advantages
in
terms
of
information
However,
the simulation
simulation
capacity
needed
to experiment
experiment
full
that
stilldynamic
often
use
traditional
systems
engineering
practices
However,
the
capacity
needed
to
aa full
system
behaviour
is
rarely
available
at
companies
world
brings
numerous
advantages
in
terms
of
information
management,
processing,
and
visualization,
but
demands
system
dynamic
behaviour
is
rarely
available
at
companies
world
brings
numerous
advantages
in terms
terms
ofbut
information
that
stilldynamic
often
use
traditional
systems
engineering
practices
careful
attention
for their
relevant
and
efficient
utilization.
world
brings
numerous
advantages
in
of
information
management,
processing,
and
visualization,
demands
system
dynamic
behaviour
iscompanies
rarely available
available
atreluctant
companies
(Zeigler
et
al,
2018).
Many
are
still
in
system
behaviour
is
rarely
at
companies
that
still
often
use
traditional
systems
engineering
practices
management,
processing,
and
visualization,
but
demands
careful
for
their
relevant
and
efficient
utilization.
still
often
use
traditional
systems
engineering
practices
management,
processing,
and
visualization,
but
demands that
(Zeigler
et
al, 2018).
Many companies
aredoubts
still reluctant
in
The
use attention
of models
proposed
in the
frame
of the but
Model-Based
management,
processing,
and
visualization,
demands
careful
attention
for
their
relevant
and
efficient
utilization.
that
still
often
use
traditional
systems
engineering
practices
deploying
simulation
benches
mostly
since
remain
on
that
still
often
use
traditional
systems
engineering
practices
(Zeigler
et
al,
2018).
Many
companies
are
still
reluctant
in
careful
attention
for
their
relevant
and
efficient
utilization.
The
use
of
models
proposed
in
the
of
Model-Based
et
al,
2018).
Many
companies
are
still
reluctant
in
careful
attention
for
their
relevant
and
utilization.
deploying
simulation
benches
mostly
since
doubts
remain
on
Systems
Engineering
(MBSE)
shallframe
helpefficient
to the
decrease
project (Zeigler
careful
attention
for
their
relevant
and
efficient
utilization.
The
use
of
models
proposed
in
the
frame
of
the
Model-Based
(Zeigler
et
al,
2018).
Many
companies
are
still
reluctant
in
the
returneton
Moreover,
project
leaders
still have
(Zeigler
al,investment.
2018). benches
Many
companies
aredoubts
still reluctant
in
deploying
simulation
mostly
since
remain
on
The
use
of
models
proposed
in
the
frame
of
the
Model-Based
Systems
Engineering
(MBSE)
shall
help
to
decrease
project
deploying
simulation
benches
mostly
since
doubts
remain
on
The
use
of
models
proposed
in
the
frame
of
the
Model-Based
the
return
on
investment.
Moreover,
project
leaders
still
have
risks,
shorten
project,
decrease
development
costs,
and project
ensure difficulties
The
use
of
models
proposed
in
the
frame
of
the
Model-Based
Systems
Engineering
(MBSE)
shall
help
to
decrease
deploying
simulation
benches
mostly
since
doubts
remain
on
to
get
clear
evidence
on
whether
or
not
simulation
deploying
simulation
benches
mostly
since
doubts
remain
on
the
return
on
investment.
Moreover,
project
leaders
still
have
Systems
Engineering
(MBSE)
shall
help
to
decrease
project
risks,
shorten
project,
decrease
development
costs,
and
ensure
the
return on
investment.
Moreover,
project
leaders
still
have
Systems
Engineering
(MBSE)
shall
to
decrease
project
to
get
clear
evidence
whether
or
not
simulation
higher
product
quality.
For computer
new
design difficulties
Systems
Engineering
(MBSE)
shall help
helpscience,
to costs,
decrease
project
risks,
shorten
project,
decrease
development
and
ensure
the
on
investment.
Moreover,
project
leaders
still
is
anreturn
appropriate
practice
in
theiron
project
context.
the
return
on
investment.
Moreover,
project
leaders
still have
have
difficulties
to
get
clear
evidence
on
whether
or
not
simulation
risks,
shorten
project,
decrease
development
costs,
and
ensure
higher
product
quality.
For
computer
science,
new
design
difficulties
to
get
clear
evidence
on
whether
or
not
simulation
risks,
shorten
project,
decrease
development
costs,
and
ensure
is
an
appropriate
practice
in
their
project
context.
patterns
and
methods
have
been
created.
This
has
driven
to
risks,
shorten
project,
decrease
development
costs,new
and ensure
higher
product
quality.
For
computer
science,
design
difficulties
to get
get practice
clear evidence
evidence
on
whether
or not
not simulation
simulation
difficulties
to
clear
on
whether
or
is
an
appropriate
in
their
project
context.
higher
product
quality.
For
computer
science,
new
design
patterns
and
methods
have
been
created.
This
has
driven
to
is
appropriate
practice
in
This
aims at
clarifying
the project
current context.
status of Simulation
higher
product
quality.have
Forbeen
computer
science,
new
design
develop
moremethods
complex
systems,
often science,
combining
disparate
higher
product
quality.
For
computer
new
design
patterns
and
created.
This
has
driven
to
is an
an paper
appropriate
practice
in their
their
project
context.
is
an
appropriate
practice
in
their
project
context.
patterns
methods
have
been
This
driven
to
paper
aims
at
clarifying
the
current
status
of
Simulation
develop
more
complex
systems,
often
combining
disparate
and
MBSE
in companies
and
to identify
the way
of
progress
patterns and
and
methods
have
been created.
created.
This has
has
driven
to This
This
paper
aims
at
clarifying
the
current
status
of
Simulation
technologies
and
elements
(Huldt
and Stenius,
2018).
This
patterns
and
methods
have
been
created.
This
has
driven
to
develop
more
complex
systems,
often
combining
disparate
This
paper
aims
at
clarifying
the
current
status
of
Simulation
develop
more
complex
systems,
often
combining
disparate
and
MBSE
in
companies
and
to
identify
the
way
of
progress
This
paper
aims
at
clarifying
the
current
status
of
Simulation
technologies
and
elements
(Huldt
and
Stenius,
2018).
This
for
MBSE
and
simulation
tools,
frameworks,
or
develop
moreand
complex
systems,
often
combining
disparate
This
paper
aims
at
clarifying
the
current
status
of
Simulation
and
MBSE
in
companies
and
to
identify
the
way
of
progress
computerized
design
has systems,
considerably
evolved
and
has
given
develop
more
complex
often
combining
disparate
technologies
elements
(Huldt
and
Stenius,
2018).
This
and
MBSE
in
companies
and
to
identify
the
way
of
progress
technologies
and
elements
(Huldt
and
Stenius,
2018).
This
for
MBSE
and
simulation
tools,
frameworks,
or
and
MBSE
in
companies
and
to
identify
the
way
of
progress
computerized
design
has
considerably
evolved
and
has
given
methodologies.
The
purpose
of
the
survey
was
first
to
technologies
and
elements
(Huldt engineering
andevolved
Stenius,and
2018).
This and
MBSE
in companies
and to identify
theframeworks,
way of progress
for
MBSE
and
simulation
tools,
or
rise
to some very
powerful
systems
tools.
technologies
and
elements
(Huldt
and
Stenius,
2018).
This
computerized
design
has
considerably
has
given
for
MBSE
and
simulation
tools,
frameworks,
or
computerized
design
has
considerably
evolved
and
has
given
methodologies.
The
purpose
of
the
survey
was
first
to
for
MBSE
and
simulation
tools,
frameworks,
or
rise
to
some
very
powerful
systems
engineering
tools.
establish
an
overview
of
the
state
of
practices
on
simulation
computerized
design
has
considerably
evolved
and
has
given
for
MBSE
and
simulation
tools,
frameworks,
or
methodologies.
The
purpose
of
the
survey
was
first
to
computerized
design
has considerably
evolved
and
has
given methodologies. The purpose of the survey was first to
rise
to
some
very
powerful
systems
engineering
tools.
rise
to
some
very
powerful
systems
engineering
tools.
The
potential
to
increase
verification
activities
through
establish
an
overview
of
the
state
of
practices
on
simulation
methodologies.
The purpose
purpose
of itof
thewas
survey
was
first
to
usage
in anSE/MBSE.
Secondly,
to identify
some
rise
to
some
powerful
systems
engineering
tools. through methodologies.
The
of
the
survey
was
first
to
establish
overview
of
the
state
practices
on
simulation
rise
topotential
someisvery
very
powerful
The
increase
verification
an
overview
the
of
practices
on
simulation
usage
in
Secondly,
it
to
identify
some
simulation
ato
key
pointsystems
for
theengineering
futureactivities
of tools.
SE. through
For its establish
The
potential
to
increase
verification
activities
establish
anSE/MBSE.
overview of
of
the state
state
of was
practices
on
simulation
barriers
to
MBSE/simulation
fulfilment.
In
the
remaining
of
establish
an
overview
of
the
state
of
practices
on
simulation
usage
in
SE/MBSE.
Secondly,
it
was
to
identify
some
The
potential
to
increase
verification
activities
through
simulation
point
for
the
future
of
SE.
For
its
is
key
usage
in
SE/MBSE.
Secondly,
it
was
to
identify
some
The potential
potential
toapplication
increase
verification
activities
through
barriers
MBSE/simulation
fulfilment.
In
remaining
of
promoters,
theaato
of simulation
techniques
can
The
increase
verification
activities
through
simulation
is
key
point
for
the
future
of
SE.
For
its
usage
into
SE/MBSE.
Secondly,
it and
wasMBSE
tothe
identify
some
the
paper,
the
context of
simulation
is presented.
usage
in
SE/MBSE.
Secondly,
it
was
to
identify
some
barriers
to
MBSE/simulation
fulfilment.
In
the
remaining
of
simulation
is
a
key
point
for
the
future
of
SE.
For
its
promoters,
the
application
of
simulation
techniques
can
barriers
to
MBSE/simulation
fulfilment.
In
the
remaining
of
simulation
is
a application
key point
point
for
the
future of
of
SE. support
For can
its the
paper,
the
context
of
simulation
and
MBSE
is
presented.
significantly
increase
design
knowledge
and
simulation
is
a
key
for
the
future
SE.
For
its
promoters,
the
of
simulation
techniques
barriers
to
MBSE/simulation
fulfilment.
In
the
remaining
of
Then,
the
survey
procedure
isfulfilment.
detailed,
andthethe
results
are
barriers
to
MBSE/simulation
In
remaining
of
the
paper,
the
context
of
simulation
and
MBSE
is
presented.
promoters,
the
application
of
simulation
techniques
can
significantly
increase
design
knowledge
and
support
the
paper,
the
context
of
simulation
and
MBSE
is
presented.
promoters,
the
application
of
simulation
techniques
can
Then,
the
survey
procedure
is
detailed,
and
the
results
are
Verification,
Validation
& ofTesting
(VVT),
and support
failure
promoters,
the
application
simulation
techniques
can
significantly
increase
design
knowledge
and
the
paper,
the
context
of
simulation
and
MBSE
is
presented.
examined.
A
conclusion
is
given
on
the
opportunities
opened
the
paper,
the
context
of
simulation
and
MBSE
is
presented.
Then,
the
survey
procedure
is
detailed,
and
the
results
are
significantly
increase
design
knowledge
and
support
Verification,
Validation
&
Testing
(VVT),
and
failure
Then,
the
survey
procedure
is
detailed,
and
the results
are
significantly
increase
design
knowledge
and
examined.
A conclusion
is given
on the opportunities
opened
correction
in an
early product
development
stage.
It support
should
significantly
increase
design
knowledge
and
support
Verification,
Validation
&
Testing
(VVT),
and
failure
Then,
the
procedure
is
and
are
by
the survey
observations.
Then,
the survey
survey
procedure
is detailed,
detailed,
and the
the results
results
are
examined.
A
conclusion
is
given
on
the
opportunities
opened
Verification,
Validation
&
Testing
(VVT),
and
failure
correction
in
an
early
product
development
stage.
It
should
examined.
A
conclusion
is
given
on
the
opportunities
opened
Verification,
Validation
&
Testing
(VVT),
and
failure
by
the
survey
observations.
increase
the
confidence
of
the
developers
in
the
design
and
Verification,
Validation
&
Testing
(VVT),
and
failure
correction
in
an
early
product
development
stage.
It
should
examined.
A
conclusion
is
given
on
the
opportunities
opened
examined.
A
conclusion
is
given
on
the
opportunities
opened
by
the
survey
observations.
correction
in
an
early
product
development
stage.
It
should
increase
the
confidence
of
the
developers
in
the
design
and
by
observations.
2. Deployment
MBSE and simulation
correction
in
an
product
development
stage.
It
should
fosters
the
correction
intrinsic
conceptual
design
correction
in
an early
early of
product
development
stage.
It failures
should
increase
the
confidence
of
the
developers
in
the
design
and
by the
the survey
survey
observations.of
by
the
survey
observations.
increase
the
confidence
of
the
in
design
and
2. Deployment
of MBSE and simulation
fosters
the
conceptual
design
failures
increase
thecorrection
confidence
ofintrinsic
the developers
developers
in the
the
design
and
2.
Deployment
of
MBSE
and
simulation
that
usually
cause
highof
rework
costs
if found
during
system
increase
the
confidence
of
the
developers
in
the
design
and
fosters
the
correction
of
intrinsic
conceptual
design
failures
2.
Deployment
MBSE
and
simulation
MBSE emerged
to supportof
the
specification,
analysis, design,
fosters
the
correction
of
intrinsic
conceptual
design
failures
that
usually
cause
if
found
during
system
2.
Deployment
of
MBSE
and
simulation
fosters
the
correction
ofrework
intrinsic
conceptual
design
failures
2.
Deployment
of
MBSE
and
simulation
testing
(Hoppe
ethigh
al,
2007).costs
The
combination
of
the MBSE
emerged
to
support
the
specification,
analysis, design,
fosters
the
correction
of
intrinsic
conceptual
design
failures
that
usually
cause
high
rework
costs
if
found
during
system
and
verification
of
complex
systems
using
integrated
system
MBSE
emerged
to
support
the
specification,
analysis,
design,
that
usually
cause
high
rework
costs
if
found
during
system
testing
The
the
(Hoppe
et
al,
2007).
MBSE
emerged
to
support
the
specification,
analysis,
design,
that usually
usually
causeverification
high
rework
costs
if combination
found
during
system
simulation
and
tools
should
help
the of
and
verification
of
complex
systems
using
integrated
system
that
cause
high
rework
costs
if
found
during
system
testing
(Hoppe
et
al,
2007).
The
combination
of
the
MBSE
emerged
to
support
the
specification,
analysis,
design,
models
with
dedicated
software
tools
(MacCalman
et al,
MBSE
emerged
to
support
the
specification,
analysis,
design,
and
verification
of
complex
systems
using
integrated
system
testing
(Hoppe
et
al,
2007).
The
combination
of
the
simulation
verification
tools
should
help
system
verification
of
complex
systems
using
integrated
system
testing
(Hoppe
et al,
al, the
2007).
The
combination
of
the and
designer
to and
understand
traceThe
results
given the
by of
model
models
with
dedicated
software
tools
(MacCalman
et the
al,
testing
(Hoppe
et
2007).
combination
the
simulation
and
verification
tools
should
help
the
system
and verification
verification
of complex
complex
systems
using
integrated
system
2016).
Companies
and
organizations
have
recognized
and
of
systems
using
integrated
system
models
with
dedicated
software
tools
(MacCalman
et
al,
simulation
and
verification
tools
should
help
the
system
designer
to
understand
the
trace
results
given
by
model
models
with
dedicated
software
tools
(MacCalman
et
al,
simulation
and
verification
tools
should
system
checkers,
ultimately
enhancing
the
use help
of the
formal
2016).
Companies
and
organizations
have
recognized
the
simulation
and
verification
tools
should
help
the
system
designer
to
understand
the
trace
results
given
by
model
models
with
dedicated
software
tools
(MacCalman
et
al,
potentialCompanies
of MBSE
and software
foresee
benefits
in terms
of higher
models
with
dedicated
tools have
(MacCalman
et the
al,
2016).
and
organizations
recognized
designer
to
understand
the
trace
results
given
by
model
checkers,
ultimately
enhancing
the
use
of
the
formal
2016).
Companies
and
organizations
have
recognized
the
designer
to
understand
the
trace
results
given
by
model
verification
method
(Seidner
ettrace
al, 2010).
potential
of
MBSE
and
foresee
benefits
in
terms
of
higher
designer
to
understand
the
results
given
by
model
checkers,
ultimately
enhancing
the
use
of
the
formal
2016).
Companies
and
organizations
have
recognized
the
2016).
Companies
and
organizations
have
recognized
the
potential
of
MBSE
and
foresee
benefits
in
terms
of
higher
checkers,
ultimately
enhancing
the
use
of
the
formal
verification
method
(Seidner
et
al,
2010).
of MBSE
and foresee
benefits in
terms of
higher
checkers,
ultimately
enhancing
the
checkers,
ultimately
enhancing
the use
use of
of the
the formal
formal potential
verification
method
(Seidner
et
al,
2010).
potential
potential of
of MBSE
MBSE and
and foresee
foresee benefits
benefits in
in terms
terms of
of higher
higher
verification
method
(Seidner
et
al,
2010).
verification
method
(Seidner
et
al,
2010).
verification
method© (Seidner
al, 2010).
2405-8963
Copyright
2021 The et
Authors.
This is an open access article under the CC BY-NC-ND license.
Peer review under responsibility of International Federation of Automatic Control.
10.1016/j.ifacol.2021.08.133
K.E. Bemmami et al. / IFAC PapersOnLine 54-1 (2021) 1132–1137
quality and shorter lead times (Huldt and Stenius, 2018).
However, according to INCOSE SE Vision 2025 “MBSE has
grown in popularity as a way to deal with the limitations of
document-based approaches but is still in an early stage of
maturity similar to the early days of CAD/CAE” (Beihoff et
al., 2014). This status is a result of several barriers to the
transition from the traditional SE approach to MBSE. For
example, there exist required changes from legacy practices
relied on documentation, in organizations and their network
of suppliers and clients (Friedenthal and Steiner, 2012). Other
barriers are the high cost for such technical transition, the
lack of strong management structures to support and guide
the implementation, the lack of knowledge to integrate a
model-based approach with current business processes, and a
lack of available trained resources (Huldt and Stenius, 2018).
For supporters of MBSE, the superposition of modelling and
simulation in systems engineering provides a prospect that is
an integral part of a cost-effective process to meet user
requirements and needs (Kossiakoff and Sweet, 2002).
Simulation provides the system designer with a fair insight
upon its behaviour, which is particularly needed in the early
design phases of a complex system (Lerat and Roux, 2010).
For example, analysing simulation model output results are
one way a systems engineer can early verify and/or validate a
system requirement (MacCalman et al, 2016). In many
instances, the modelling, simulation, and analysis activities
are interwoven, particularly during the development,
verification, and validation phases (Aeronautics, 2016).
Simulation techniques are also seen as an essential element
for early VVT. In the context of the pilot project conducted
by (Hoppe et al, 2007) , simulation techniques were used
more intensively, trying to enhanced VVT planning and
execution in the first phases of the project (Hoppe et al,
2007). In this study, failures and intensive VVT in the latter
phases were avoided, resulting in an overall VVT cost
reduction of 18%, which was found significant. Hoppe et al.
also found a reduction in test redundancy and improved
quality coordination with the pilot project engineering
services. (Hoppe et al, 2007).
Despite the importance of the simulation contributions,
project managers still have some hesitations when it comes to
its deployment. These hesitations may be due to a lack of
trust, when in some way the simulated model is seen abstract
or simplistic (Schamai, 2013). In another point, the heavy
deployment workload generally related to the complexity of
the system as well as the cost of deployment represent
obstacles that are largely based on the feelings of project
managers as reflected in practice studies (Huldt and Stenius,
2018; Laing et al., 2020). In this respect, the NASA standard
7009A (Aeronautics, 2016) proposes a set of questions to
answer before simulation deployment. These questions are
related to the pedigree (and quality) of the data used to
develop the model, the uncertainty characterization of the
model, and the correctness of the model implementation per
their requirements/specifications.
Many parameters can be considered to assess the necessity of
modelling and simulation tasks. The cost of modelling, skill
availability, level of confidence in the estimation, reusability
of the model, may be criteria to analyse. The presented
1133
survey aims at better understanding the way the decision of
introducing simulation or not could be made.
This state of practice aims at understanding enablers and
barriers to simulation utilization at various companies. It is
also used to identify the primary criteria used in companies to
decide when modelling and simulation effort might be made
within projects.
3. CRRA Survey
3.1. Survey description
The survey was conducted during a workshop on MBSE &
VV (Verification, Validation) organized by the CRRA
chapter of AFIS1 in January 2019. The workshop gathered 21
participants from France belonging to 18 different
organizations from various fields. The respondents were not
filtered and belong to a specific set interested in sharing
knowledge and thoughts on MBSE. Among the represented
organizations 60% of companies have more than 5 years’
experience in setting up strong SE processes. It is also to be
mentioned that 68% of participants have more than 5 years’
experience in the SE field. This panel can be compared to the
one used in (Vogelsang et al., 2017) that gathered 20
respondents in a study on MBSE adoption. The used panel
and the one from (Vogelsang et al., 2017) are smaller than
the one of (Huldt and Stenius, 2018) (66 respondents), but
this latter study has been performed through an online
questionnaire. The panel used in this work is complementary
to the one of Vogelsang et al or Huldt and Stenius as it
covers a new geographical area (e.g. France, where
Vogelsang et al covered Germany, Huldt and Stenius covered
the US and Sweden). It is also a more heterogeneous panel
from the companies’ domain perspective, aerospace and
defence were representing more than 70% of (Huldt and
Stenius, 2018) panel when 60% of (Vogelsang et al., 2017)
panel is from avionics or automotive industry. The panel of
this study gathers companies from the electrical power
management domain (16%), defence, civil nuclear industry,
or oil & gas and less than 10% by domain for all remaining
ones.
The objectives of the survey are triple: Analysing SE, MBSE,
and Simulation practices in participants’ organization;
eliciting barriers and enablers for MBSE and Simulation
adoption; finding the relevant criteria for Simulation
deployment in projects. Participants were consulted with a
questioner organized in 40 questions presented in 4 sections:
1

The status of current SE and MBSE practices in
companies (maturity level, challenges, expected
benefits).

The status of current practices of VVT in companies
(VVT strategy, criteria for VVT deployment,
practices).

The use of computer-based simulation in companies
(use, criteria for deployment, satisfaction/barriers).
AFIS: French Association of Systems Engineering,
https://www.afis.fr; French chapter of INCOSE, CRRA Rhône-Alps
Chapter of AFIS
K.E. Bemmami et al. / IFAC PapersOnLine 54-1 (2021) 1132–1137
1134

The use of multi-criteria decision methods in
companies.
The questions were developed in a way that cross-referenced
the factors governing the practices of the VVT activities and
those of simulation, for the close link that exists between the
two activities.
Some questions were inspired from the survey conducted by
(Huldt and Stenius, 2018) to analyse the link between the
MBSE approach and simulation activities and trying to better
understand the use of simulation and their impact on certain
practices in SE field.
Figure 1 shows the results. The main mentioned challenges
are, the difficulties of integrating the MBSE with other
existing processes according to 83% of participants, the need
to improve skills according to 74% of participants, as well as
the way tools are used according to 52% of participants.
Many organizational challenges were also pointed out as
formalizing the processes, creating a new structure for the
management of MBSE, or having a good sponsor in the
company organization.
The questionnaire has been sent by email to other MBSE user
community in France that permit to gather 4 more answers.
3.2. Observed MBSE practices
A set of questions explored SE and MBSE practices of
participants. Maturity and experience were investigated.
Then, the targeted benefits and the challenges to improve
MBSE in companies were analysed. The participating
organizations have various maturity level in SE, but 90% of
them invest in developing their process (see Table 1).
Table 1. Maturity Level of SE process
Maturity
degree of SE
process in
organization
Ratio %
Hight
level
Advanced
level
In
development
Not
formalized
15.00%
30.00%
45.00%
10.00%
Many companies from the panel are quite new to SE
practices as 40% of respondents declare having less than 5
years of experience in SE. But a quarter are well-established
companies in SE with more than 10 years of experience (see
table 3). Nevertheless, interviewees were more experienced
with 40% of the participants having more than 10 years of
experience in SE. Even if many companies are quite new to
SE, it can be observed that the huge majority implied in
developing an SE approach develops an MBSE approach.
Indeed 80% of companies declared using an MBSE approach
(reminding that 10% do not have an identified SE process). It
can be concluded that for the panel, setting up SE processes
quasi implies introducing MBSE approaches in the company.
This was true for experienced companies as for new users of
SE that directly invest in MBSE. The participants were asked
to give companies motivations in introducing MBSE with a
semi directed question whose results showed an equal
motivation between “improving communication among
stakeholders”, “increasing ability to manage system
complexity” and “improving quality and knowledge
management” (30% of participants). Complementary shared
motivations were “providing customer support” (10%),
“accelerating the project” (10%).
The participant companies have difficulties to master the
MBSE deployment since most interviewees (70%) indicated
that the level of maturity of their MBSE process was in
development or not formalized. Only 15% of companies are
confident in the maturity of their MBSE process. The
exploration of companies’ MBSE practices has been ended
by the search of the perceived challenges to improve MBSE.
Fig. 1. Main perceived Challenges to improve MBSE
It could be analysed that a lot of challenges are due to the
novelty of MBSE approaches for the company of the survey
panel. Even if the MBSE principles are now developed for
years, it is a very topical subject for companies interested in
adopting structured SE. Our panel covers a broad scope of
industrial domains, it confirms that MBSE raises a lot of
interest but imposes important changes in companies’
structures. It changes the development of software
frameworks, knowledge management, the skills needed in
projects, the relation with clients, the project management…
Many participants felt they need strong top management
support for deploying MBSE since working standards
changes and consequent budget is needed.
The next set of questions examined the VVT practices
deployed in companies since it is a key stage of SE processes
when simulation may play an important part.
3.3. Observed VVT practices
To address VVT practices, participants were invited to
answer questions related to various aspects (the state of VVT
practices; the considered criteria to perform a VVT action;
the important phases of the system life cycle for the VVT;
early VVT constraints; VVT & Simulation link).
VVT strategies are correctly implemented in most
organizations. 80% of participants report having a clear
vision of the VVT strategy of their organizations.
Simulation possibilities give the opportunity to make early
VVT actions. To understand the importance of early VVT in
companies’ strategy, few questions addressed this topic. It
appears that 58% of respondents mention that their
organization perform early VVT. This rate is quite modest
when compared to the importance and expected benefits of
early VVT as it was shown in (Laing et al., 2020). To
corroborate this, a high majority of participants (80%) claim
that VVT activities must be realized even earlier than in their
K.E. Bemmami et al. / IFAC PapersOnLine 54-1 (2021) 1132–1137
current practices. The early VVT actions use in companies
concern defining stakeholders' needs and system
requirements phases (see Figure 2).
Fig. 2. Main phases for the use of an early VVT strategy
in a project
The obtained results reveal the relevance of addressing new
techniques to support the early VVT deployment. To succeed
in such an approach, the panel were questioned on the main
impediments to early VVT. According to the participants
time and cost (according to 33% of participants), and
technical resources availability (according to 20% of
participants) are the main barriers (See Table 2).
1135
errors (Highly regarded for 54% of participants) (see Table
2). The decision to perform or not VVT actions are related to
the assessment of the benefits that these actions will bring,
and the risks taken in the opposite case, as reported by 70%
of participants. They claim doing this assessment within their
organization and mainly considering the following criteria
(see Table 2).

The importance of the requirements verified/tested

The complexity of the system to be checked or
tested for its behaviour or the environment in which
it operates

The impact of undetected errors

The reachable results confidence

The benefits of detecting errors
It may be observed that the functional architecture design and
system requirements analysis phases are seen as the most
critical phases for VVT activities according to the vast
majority of participants (see Figure 3). This information
highlights the need to better integrate verification and
validation activities in MBSE.
Table 2. Important criteria to consider when deciding to
perform a VVT action
To what extent the criteria below are important to consider when
deciding to perform a VVT action
-
High
Consideration
75%
To
some
extent
25%
To a
small
extent
5%
Not
at
all
0%
Importance of
verified/tested
requirements
Maturity of verified/tested
requirements
Complexity of
system/behavior/environme
nt to be verified/tested
Reachable result accuracy
30%
45%
10%
5%
50%
35%
15%
0%
30%
35%
15%
5%
55%
15%
15%
5%
40%
45%
5%
0%
Impact of undetected
errors
Cost of VVT actions
55%
20%
15%
0%
25%
35%
30%
0%
Duration of VVT actions
10%
35%
35%
0%
Reachable results
confidence
Benefits of detecting errors
Available skills
10%
25%
35%
20%
Project lifecycle phase
30%
30%
20%
0%
Fig. 3. Most important lifecycle phases to consider for
VVT actions
Concerning the link between VVT activities and simulation,
most of respondents (75%) affirms using this latter as a
support to verification, validation, and testing activities. The
next set of questions examined the deployment of simulation
in companies and their impact on several aspects of
organizations' activities.
3.4. Observed simulation practices
3.4.1. State of practice in simulation
When moving to questions related to VVT activities it was
noted that although cost and time criteria are a direct cause of
avoidance/deletion or limitation of VVT actions for more
than 50% of the participant, they are not the most considered
when making decisions regarding the deployment of VVT
activities (see Table 2). Indeed, it is less quoted than the
importance of the reviewed/tested requirements (Highly
considered for 75% of participants), the complexity of the
system, its behaviour or its environment (Highly considered
for 58% of participants), and the incidence of undetected
The first set of questions of this section were analysing the
current state of practice in simulation. Firstly, simulation
asserts its position as a support for system validation action.
When addressing the objectives of the use of simulation in
the organizations today, it appears that simulation is used
mainly for more than 60% of participants in system-level
verification and validation. Meanwhile, the deployment of
simulation is mainly related to the organization's field. As
K.E. Bemmami et al. / IFAC PapersOnLine 54-1 (2021) 1132–1137
1136
reported by more than 50% of participants, simulation is
commonly used in software and mechanical domains.
The aim of the simulation is generally to assess the
performance of the systems for more than 60% of the
participants, and for more than 40% for the consistency check
(to determine whether there are any internal conflicts in the
data), and the reliability of the state of the system.
Table 4. Quality improvement of the activities observed in
organizations when using simulation
To what extent has simulation increased the quality of
43%
5%
0%
0%
19%
38%
33%
5%
0%
0%
24%
Verification and
Validation
Architectural view
24%
33%
10%
0%
0%
19%
Some
improvement
No change
Some
impairment
Big
impairment
41,18%
35,29%
5,88%
0%
0%
19%
14%
29%
5%
0%
24%
23,53
%
29,41
%
23,53
%
29,41
%
Traceability between
systems requirements
and the realization
Technical reviews
19%
24%
14%
5%
0%
24%
35,29%
29,41%
11,76%
0%
0%
11,76%
29,41%
35,29%
0%
0%
33%
43%
0%
0%
0%
14%
17,65%
23,53%
35,29%
0%
0%
No change
Big
Improvement
Table 5. Performance improvement of the activities
observed in organizations when using simulation
To what extent has simulation improved the performance of
0%
0%
23,53
%
23,53
%
The next set of questions focuses on the importance of
simulation in projects and its impact on overall project
aspects.
3.4.2. Analysis of the importance of simulation in
projects
The participants were invited to complete their answers on
the use of simulation by discussing how they perceive the
efficiency of simulation in the project and how the
deployment of simulation is decided.
Firstly, 95% of participants feel that simulation increase the
quality of the tests and delivered projects. No impairment has
been noted. and 5% see no changes in test quality. This result
clearly shows the very high trust placed by participants in the
utilization of simulation techniques to delivered high-quality
projects. Nevertheless, it is not sufficient to justify full
investment in simulation technologies. According to
participants, an improvement in the performance and quality
of certain practices has been observed thanks to the use of
simulation (see Table 4 & 5). It is broadly shared for
architecture and design, requirements analysis, verification
and validation, and technical reviews. We observe when
analysing these results that simulation is largely recognized
to be useful for technical activities (design, requirement
analysis, V&V, technical review). But when moving to
Architecture and
design
Requirement
analysis
Verification and
Validation
Architectural view
Traceability between
systems
requirements and
the realization
Technical reviews
Not
applicable
35,29%
0%
Big
impairment
41,18%
5,88
%
0%
Some
impairment
5,88%
No change
17,65%
Some
improvement
47,06%
Big
Improvement
Architecture
and design
Requirement
analysis
Architectural
view
Traceability
between
systems
requirements
and the
realization
Technical
reviews
Verification
and
Validation
Not
applicable
33%
To what extent has simulation increased the efficiency to perform
Big
impairment
Some
improvement
Architecture and
design
Requirement analysis
Table 3. Simulation efficiency impact
Some
impairment
Big
Improvement
Not applicable
The impact of simulation on the effectiveness of system
architecture and design, requirements analysis, technical
reviews, and system verification and validation is particularly
important (see Table 3).
activities aiming at consolidating the system-level view as the
architectural view or traceability tasks, the participants have
heterogeneous opinions. It may be concluded that simulation
is seen as a strong support to enhance the quality of the
system development. But it is currently not seen by most of
the actors as an obvious support to system-level analysis.
39%
33%
6%
0%
0%
22%
33%
28%
11%
0%
0%
28%
11%
28%
33%
0%
0%
22%
17%
22%
33%
0%
0%
28%
44%
17%
6%
6%
0%
22%
39%
33%
0%
0%
0%
22%
In addition to these questions, 73% of participants judged
that simulation is increasing the communication and
information exchange within the project. It is also appreciated
for the help in anticipating system failures (73%). Finally,
66% claimed that the use of simulation facilitates the
integration of the system.
Despite the high trust in simulation utility in many project
phases, many impediments to the use of simulation were
revealed with an open question. 40% of respondents
mentioned that the use of simulation is costly and demands
highly skilled persons. 22% of respondents mentioned a
difficulty to cope with model complexity. On the impact of
simulation on the overall cost of the project, opinions differ,
suggesting that it may depend on the type and complexity of
projects. The results in Table 6 mainly suggest that it is very
K.E. Bemmami et al. / IFAC PapersOnLine 54-1 (2021) 1132–1137
difficult for practitioners to have a clear understanding on the
impact of simulation on the project cost. Thus, uncertainty on
the simulation ROI is an important barrier to convince
managers in investing in these technologies.
Table 6. Impact of simulation on the project cost
Increase the
global cost (+)
No impact
Decrease the
global cost (-)
Decrease the
global cost (- -)
Ratio %
Increase the
global cost (+ +)
Evaluate the impact of
simulation on the
global cost of
SE/MBSE project
18%
24%
12%
35%
12%
As a short sum up, it could be noted that the benefits of using
simulation are recognized but the financial impact of its use
is uncertain. To better understand how simulation
deployment is decided in companies, participants were asked
on the criteria utilized to make such decisions (Fig. 4). The
use of simulation is then driven by the costs and the
perceived complexity of the project. And by the perceived
cost and duration of the VVT actions. It is interesting to
observe that the decision is to be made with criteria that
companies have difficulties to master (cost impact).
Moreover, it can be observed that the criteria pushed by
NASA (Aeronautics, 2016) to analyse simulation needs (e.g.
reachable result accuracy and confidence) are among the least
cited by participants. Many additional concerns are to be
considered in the deployment of simulation as the potential
avoidance of errors, project characteristics as available skills
or lifecycle phases. This result demonstrates that the decision
to deploy simulation in a project is multi-criteria and hard to
clearly answer.
1137
architecture and design, requirements analysis, and
verification and validation. Nevertheless, it is very difficult
for practitioners to have a clear understanding on the impact
of simulation on the project cost. The studies revealed that
the return on investment of such practice must be proven to
convince managers on their potential benefits. Moreover, it is
admitted that actions are to be taken to master the MBSE and
simulation deployment costs. These let us think that further
developments to tackle these two activities during the first
phases of deployment of MBSE and simulation in new
organizations is needed.
References:
Aeronautics, N. (2016) ‘STANDARD FOR MODELS AND
SIMULATIONS’, NASA-STD-7009A, pp. 1–72.
Beihoff, B. et al. (2014) ‘A World In Motion: SE Vision
2025’, Incose, pp. 1–9.
Hoppe, M., Engel, A. and Shachar, S. (2007) ‘SysTest:
Improving the verification, validation, and testing
process— Assessing six industrial pilot projects’, Systems
Engineering, 10(4), pp. 323–347.
Huldt, T. and Stenius, I. (2018) ‘State-of-practice survey of
model-based systems engineering’, Systems Engineering,
(July), pp. 1–12.
Kossiakoff, A. and Sweet, W. N. (2002) Systems Engineering
Principles and Practice.
Laing, C. et al. (2020) ‘Questioning integration of
verification in model-based systems engineering: an
industrial perspective⋆’, Computers in Industry. Elsevier
B.V., 114, p. 103163.
MacCalman, A., Lesinski, G. and Goerger, S. (2016)
‘Integrating External Simulations Within the Model-Based
Systems Engineering Approach Using Statistical
Metamodels’, Procedia Computer Science. Elsevier
Masson SAS, 95, pp. 436–441.
Sanford Friedenthal, Rick Steiner, A. M. (2012) A Practical
Guide to SysML. 3rd edn. Edited by Morgan Kaufmann.
Elsevier.
Schamai, W. (2013) Model-Based Verification of Dynamic
System Behavior against Requirements.
Seidner, C., Lerat, J.-P. and Roux, O. H. (2010) ‘5.4.1
Simulation and Verification of [Dys]functional Behavior
Models: Model Checking for SE’, INCOSE International
Symposium, 20(1), pp. 681–693.
Fig. 4. Criteria to decide simulation use
4. Conclusion and Discussion
The results of the CRRA survey have shown a need for
further development in MBSE for a best integration with the
existing SE process and a need to further improve skills in SE
and modelling activities. The performed survey points out
that simulation is perceived to increase the quality of tests
performed and delivered projects, it is also considered as a
key element for VVT strategies. According to respondents,
an improvement in the quality of certain practices has been
observed following the use of simulation, mainly in
Vogelsang, A. et al. (2017) ‘Should I Stay or Should I Go?
On Forces that Drive and Prevent MBSE Adoption in the
Embedded Systems Industry’, in Felderer, M. et al. (eds).
Cham: Springer International Publishing (Lecture Notes in
Computer Science), pp. 182–198.
Zeigler, B., Mittal, S. and Traore, M. (2018) ‘MBSE with/out
Simulation: State of the Art and Way Forward’, Systems,
6(4), p. 40.