Mixed Reality and Digital Engineering Solutions
and Their Promotion in the Baltic States
Eberhard Blümel1, Leonids Novickis2 , Marco Schumann3, Stefan Leye4
1,3,4 Fraunhofer
Institute for Factory Operation and Automation IFF,
Magdeburg, Germany
2 Riga Technical University, Riga, Latvia
3nd International Workshop on
Intelligent Educational Systems and Technology-enhanced Learning
(INTEL-EDU 2012)
Riga, October 10, 2012
© Fraunhofer IFF
Outline
1. Global Challenges & Innovative
Technologies
2. Digital Engineering & Digital Factory
3. Mixed Reality Platforms
4. Center of Digital Engineering (CDE)
5. Technology Transfer & Baltic States
6. Conclusions
2
© Fraunhofer IFF
Fotos © Fraunhofer
Challenges – ”The Markets Beyond Tomorrow”
Health and nutrition
Safety and security
Affordable healthcare
Disaster prediction and
management
Information and
communication
Energy and living
Low-loss generation,
distribution and use of
electricity
Mobility and
transportation
Low-emission, reliable
mobility in urban areas
Production and
environment
Life-cycle production
3
© Fraunhofer IFF
PRODUCTION AND ENVIRONMENT
Images © Fraunhofer; istockphoto
We conduct research in the following areas:
Production that saves
energy & raw materials
Automobile and plant
engineering, robotics
Product development
Production processes
Manufacturing
technologies/methods
Materials and surfaces
Measurement and
test engineering
Digital Engineering
4
© Fraunhofer IFF
INFORMATION AND
COMMUNICATION
Images © Fraunhofer; MEV; istockphoto
We conduct research in the following areas:
Image processing
eBusiness
Broadband
communications
eGovernment
Cloud computing / grid
computing
Embedded systems
eLearning,
edutainment & games
Software
engineering
Green IT und green
through IT
Usability
5
© Fraunhofer IFF
Outline
1
Global Challenges & Innovative Technologies
2
Digital Engineering & Digital Factory
2
Mixed Reality Platforms
3
Center of Digital Engineering (CDE)
4
Technology Transfer & Baltic States
5
Conclusions
6
© Fraunhofer IFF
Product Development vs. Production Planning
Virtual Reality to close the Gap
Digitalisation
DigitalGeometry
Model
Product
Development
Cooperation
&
Communication
2D-CAD
alpha-num.
Planning
1980
1990
Production
Planning
Stand allone
Simulation
2000
Virtual Reality
Virtual Product
3D-CAD
2010
7
© Fraunhofer IFF
Digital Factory - Definition
 The digital factory is the superordinate
concept for a comprehensive network of
digital models and methods, amongst
others, of the simulation and
3-D-Visualization.
Quelle: Siemens
Pictures Of the
Future
2/2007
 Their purpose is the comprehensive
planning, realization, control and
continuously improvement of all essential
factory processes and factory resources
in connection with the product.
 The digital factory is no for sale product but a strategy!
Sourcle: VDI-Arbeitskreis „Digitale Fabrik“
8
© Fraunhofer IFF
Techniques of the Digital Factory
Product Constitution Process
Product development
Production planning
Production
Simultaneous/Concurrent Engineering
Methods
Business Pprocess Management
Product/Simulation Data Management
Tolerance Management/-simulation
CAP/PPR
CAO: Office, Project-/Knowledge Management, Groupware
CAQ
Material Flow Simulation
VIBN
Tools
PPS/ERP
FEM-Simulation
CAD/Factory Layout Planning
MKS/3D-Kinematicsi Simulation
CAM/NC
BDE/MES
RFID
Virtual Reality
User
Interface
s
Source : in Anlehnung an: Zäh und Schack (2006); Zäh u.a. (2005)
Augmented Reality
Telepresence
9
© Fraunhofer IFF
Product
Development
Production
Production Planning
Digital Factory
…
Office
Project Management
Groupware
Knowledge Management
CAM
CAQ
CAP
Layout Planung
CAD
Material Flow Simulation
FEM
Ergonomics-Simulation
Virtual Reality (VR) + Digital Mock-Up (DMU)
Robotics and
NC-Simulation
Degree of Integration of Tools in the Digital Factory
Degree of Integration of Tools in the
Product Creation Process
No Computer Aided Support
10
© Fraunhofer IFF
Central challenges to the digital factory of the next
generation
 Progressive data integration
-> future scenario “product life cycle management” (PLM)
 3D visualization as a communication and interaction medium
-> means to the control of the complexity of information
 Closing of control circuits between the development and production
-> new methods of Digital Engineering
 Personnel qualification
-> new requirement profile of Digital Engineering
11
© Fraunhofer IFF
Digital Engineering
 Digital Engineering is the universal use
of digital methods and tools over the
product development and production
process and is aimed at an improved
planning quality as well as at the
process control over the whole product
life cycle.
 physically correct transformation of all problem relevant characteristics also of the software-technical qualities (e.g., embedded systems)
 Interoperability of the used tools (technically, semantically, organizationally)
 Application in interactive experience rooms to the inclusion of the person
in the digital chain
12
© Fraunhofer IFF
Outline
1
Global Challenges & Innovative Technologies
2
Digital Engineering & Digital Factory
2
Mixed Reality Platforms
3
Center of Digital Engineering (CDE)
4
Technology Transfer & Baltic States
5
Conclusions
13
© Fraunhofer IFF
Reality-Mixed Reality-Virtuality Continuum
Virtual Reality
Virtual reality is a high-end user-computer interface that
involves real-time simulation and interactions through
multiple sensorial channels. These sensorial modalities
are visual, auditory, tactile, smell, and taste). [Burdea and Coiffet,
2003]
Reality-Virtuality
Continuum (Milgram)
Comprehenses the
transition from the real
World into the virtual
World and vice versa.
Augmented Virtuality
Augmented Reality
Mixed Reality
14
© Fraunhofer IFF
Mixed Reality Supports the Entire Product Life Cycle
Design Review
• Engineering
• Manufacturing
• Assembly
• Maintenance
Maintenance
• Assistant systems
• Visual interactive
repair instructions
Education and
Training
• Technical staff
• Operation service
• Assembly service
Functional Test
• Mechanics
• Electronics
• Control systems
Virtual
Knowledge
Base
Technical
Documentation
• Visual interactive
catalogs
• Electronic manuals
Job Preparation
• Work scheduling
• Resource optimization
• Logistics functions
Factory Planning
• Layout planning
• Process planning
15
© Fraunhofer IFF
Fraunhofer IFF VDT Platform
Data Base
Platform: Core
Plugins
• CAD data
• Scenario data file
(XWS)
Scenario concept
• Texture, sounds, text,
Extension by
coupling new
function to defined
interfaces
Authoring tool
configurations, etc.
Runtime system
Scene Graph
• Version 1.8
GUI:
• Plugin
• .NET Framework 3.5.
16
© Fraunhofer IFF
Virtual Layout and Production Planning
 Development and planning of factory processes
 Assessment and validation using simulation
 Rapid generation of information about feasibility
and cost
Benefits
 Time savings when preparing quotes
 Planning support
 Early error detection
 Decision making support
 Reduced cost and effort
17
© Fraunhofer IFF
Mobile Augmented Reality
Assistance of picking processes




Investigation of psychological and working-medical factors in
environments close to production (-> exemplary workplace)
Optimization of mobile AR systems; requirement catalog
Transferability on different real scenarios
Allow in the medium term industrial application mobile AR
18
© Fraunhofer IFF
Stationary Augmented Reality
Assistance of Assembly Procedures
Virtuality
Reality
Camera
Image
Display
Mixed
Reality
Workplace
19
© Fraunhofer IFF
Coupling of virtual models with real control
Interactive 3-D-Model
Simulation Tools
Real Operator Panel
Software
»VDT-Platform«
- SINUMERIK
Machine
Simulator
- WinMOD
- Fraunhofer RTI
- Siemens 840D
inclusive
maschinespecific NCcycles
20
© Fraunhofer IFF
Coupling of virtual models with real control
• Product Presentation (Marketing)
• Development and Testing of machine specific NC-programs
• Operator Training
21
© Fraunhofer IFF
Aviation Maintenance Training
 interactive training scenarios for service staff or
pilots
 Benefits
 better learning of procedure steps
 more understandable instructions through
interactive visualization
 customized learning systems
 sustainable learning outcomes
22
© Fraunhofer IFF
Outline
1
Global Challenges & Innovative Technologies
2
Digital Engineering & Digital Factory
2
Mixed Reality Platforms
3
Center of Digital Engineering (CDE)
4
Technology Transfer & Baltic States
5
Conclusions
23
© Fraunhofer IFF
Center for Digital Engineering
Branch-covering transfer of Digital Engineering
methods
Medicine/
Medicine Technics
Surgery-Simulation &
Visualization
Automotive
Multi physical simulation
of vehicles
Dynamics
of Complex Systems
NeuroScience
Coupling of processtechnical simulation
Stimulation by Virtual
Models
24
© Fraunhofer IFF
Aims of the Master Degree Course Digital Engineering
 Graduates are engineers with competent informatics knowledge for
development, construction and operation of complicated, technical products
and systems
 Knowledge of methods for usage of modern IT solutions in application and
research fields of Engineer's sciences
 Course contains above-average percentage in Project work, which is partly in
cooperation with partners of applied research offered (in collaboration with
industrial partners)
 Focus on interdisciplinarity in special Lectures, in project work and within the
preparation of the master thesis
25
© Fraunhofer IFF
Structure of the Master Degree Course
Digital Engineering
Semester 4
Master Thesis
(30)
Digital EngineeringTechnical
Semester 3
Project
specialization
(12)
(18)
Interdisziplinäre
Methods of
Methods of
Semester 2 s Team-Project
Informatics
Digital
(6)
(12)
Engineering
(12)
InformatikEngineer's
Human
Semester 1
Basics
Basics
Factors
(18 oder 6)
(18 oder 6)
(6)
26
© Fraunhofer IFF
Lack of Experts
The branch automobile industry lacks in 2012 about 11,000 engineers
(Pricewater / Coopers, publication on 22.09.2010)
Not only demographic reasons, but:
 Rising need in specialist knowledge and research capacities on the basis
of innovation offensives
(e.g. alternative drives, trend to extend the model range)
 Rising research budget of the branch estimated from topically 20.9
billions Euros / year by about 3.4 billions / year within two years
27
© Fraunhofer IFF
Outline
1
Global Challenges & Innovative Technologies
2
Digital Engineering & Digital Factory
2
Mixed Reality Platforms
3
Center of Digital Engineering (CDE)
4
Technology Transfer & Baltic States
5
Conclusions
28
© Fraunhofer IFF
The INTERREG 4b Project: BONITA
Baltic Organisation and Network of Innovation Transfer Associations
 16 partners from 10 countries
 8 universities, 4 science parks
 3 technology transfer organisations
 1 ICT association
 share best practice examples for
Scientific Transfer of Technologies
 improve the role of universities for
regional transfer of research
 set up a network of showrooms to
promote scientific innovation within a
region – share within the network
 set up a common organisation for
sustainable operation
29
© Fraunhofer IFF
Forms of Technology / Knowledge Transfer
 transfer through direct research assignments from industry
 transfer through cooperation projects, also with third party funding
 transfer through cooperation projects in the teaching field
 transfer of highly qualified people (students, PhD, manager, …)
 small and specialized exhibitions (showrooms) run by research institutions as
windows to scientific innovation.
 physical showroom is for demonstrating cutting edge-technologies in a
tangible and accessible fashion
 virtual showrooms have centralized access to exhibits located in different
places
30
© Fraunhofer IFF
BONITA-ShowRoom at RTU
Showroom at RTU aims to:
 Promote innovative ICT solutions
 Create a link between product’s/service or idea’s author and industry,
especially SMEs
 Network with the other technology transfer centers
Showroom at RTU is a place for:
 demonstration of existing products and solutions;
 creation of new products’ and solutions’ ideas.
Showroom at RTU includes three inter-related components (parts):
 Physical exhibition located at RTU premises;
 multimedia demonstration (exhibit) outside the Showroom’s premises;
 Web-based infrastructure: web-portal.
31
© Fraunhofer IFF
32
© Fraunhofer IFF
German-Baltic Competence Network for Mixed Reality
 Bundling, developing and interlinking of the competencies of the partners
 Development of new cooperation forms on the basis of VR-Technologies
Forum for Intelligent
Machines
(Finland)
German-Baltic Competence Network for
VR/AR-supported development of
innovative products und services
Riga Technical University
Baltic Regional
Competence Centre
Virtual and Augmented Reality
FASA e.V.
SME-Networks
(Germany)
© Fraunhofer IFF
Vidzeme University
Valmiera
Socio-technical
engineering
VTT
Technical Research
Centre of Finland
Tampere
Customer Driven
Design
 Support of SME of the processing industry for the entire application of high
technologies
(Latvia)
Methodological approach of knowledge transfer to SMEs
 Raising SMEs’ awareness
to identify the potentials for applications of
virtual reality technologies
 Performing an operational potential
analysis
to specify companies’ needs and structure
operational problems
 Qualifying skilled labor and
management
to impart basic knowledge about virtual
reality and test new effective forms of
basic and advanced training (combining
real and virtual methods in qualification)
 Developing VR tools
to test and evaluate concrete VR
applications in companies
© Fraunhofer IFF
34
FP7-Project eINTERASIA
ICT Transfer Concept for Adaptation, Dissemination and Local
Exploitation of European Research Results in Central Asia
Countries
Co-ordinator: Prof. Leonid Novickis, Riga Technical University
S & T Objectives:
The major purpose of eINTERASIA is to support international
cooperation with Central Asia’s countries by creating a
Technology Transfer Concept for adaptation, piloting, diffusion
and local exploitation of EU research results. The application of
TTC will be demonstrated in the field of eLogistics.
35
© Fraunhofer IFF
eINTERASIA –
Technology Transfer Approach
Virtual
Reality
Technology
eINTERASI
A
Capability
Maturity
Transfer Model
eINTERASIA
Showrooms
showrooms to promote
scientific innovation
within a region – share
within the network
Virtual
Reality
Technology
improve the
effectiveness of
science industry
collaboration and
regional transfer of
research
investment
share best practice
examples for Scientific
Transfer of Technologies
according to a standard
based transfer model
36
© Fraunhofer IFF
VR-based Technology Transfer Processes
Capabilit
y
Maturity
Transfer
Model
Produc
t
Proces
s
Technology
Domaines:
•
•
•
•
Logistics
Mechanics
Electronics
etc.
Customized VR-Platforms
Web - Based Framework
Show Rooms
Source: Bonita Showroom RTU
© Fraunhofer IFF
Enterprise applications
Source: Fraunhofer IFF
(2)
Education & Training
Source: Workshop at RWE premises
37
Outline
1
Global Challenges & Innovative Technologies
2
Digital Engineering & Digital Factory
2
Mixed Reality Platforms
3
Center of Digital Engineering (CDE)
4
Technology Transfer & Baltic States
5
Conclusions
38
© Fraunhofer IFF
Digital Engineering
New quality of Product Lifecycle Management
 Digital Engineering enhanced the Digital Factory by a new concept of systems
interoperability and the integration of the Humans in the digital process chain
 Knowledge transfer becomes to an integrating prozess in Digital Engineering
 Mixed Reality is becoming a cross-domain communication platform in
collaborative corporate processes
 Technology based Qualification enables sustainable human resource
development and supports innovative didactic approaches
 Technology based Qualification provide digital platforms for
 Education and training in industrial application areas
 Education and training for digital technologies and processes
39
© Fraunhofer IFF
Global Trends 2025
- Requirements for Technology Transfer
Shanghai
Increasing
Energy
Consumption
Digital
Networking and
Technology
Development
© Fraunhofer IFF
Growth of MegaCities
New solutions for
mobility
Globalization and
increasing
transport volumes
Global Migration
Demographic
Development
Economic
structural
changes
Folie
40
40
Contact
Virtual Development and Training Centre VDTC
of the Fraunhofer Institute for Factory Operation and Automation
Sandtorstr. 22
39106 Magdeburg
Dr. Eberhard Bluemel
Head of Fraunhofer IFF EU Office
Tel. + 49 391 - 4090 110
Fax + 49 391 - 4090 115
email eberhard.bluemel@iff.fraunhofer.de
41
© Fraunhofer IFF
Thank you for your attention!
42
© Fraunhofer IFF