The Design of Virtual Reality Flight Simulation Platform Based on Rigidbody Kinematics
Gang WANG, Wei MEI, Heng LIU, Ying XIAO
The Design of Virtual Reality Flight Simulation Platform Based on Rigidbody Kinematics
Gang WANG, Wei MEI, Heng LIU, Ying XIAO
Electronic and Optical Engineering Department, Ordnance Engineering College,
Shijiazhuang, China, dlmuwanggang@163.com
Abstract
MATLAB ○R /Simulink ○R simulation platform with man-in-the-loop was designed to realize virtual
reality flight based on rigid-body kinematics. Joystick orders input by operators can be transformed
into control signals. Position translation, rotation quaternion and other parameters calculated by
flight motion module were sent to 3D virtual reality space. 3D virtual reality space based on VRML
was designed by V-realm BuilderTM. Joystick inputs, flight motion module and 3D virtual reality space
were combined into a whole simulation system by Simulink 3D Animation BlocksetTM. Results of
simulation demonstrated that the simulation platform can realize piloting task under the control of
multiple operators with good maneuverability and verisimilar visual human-computer interface when
3D flight trajectory was drawing in the 3D space at the same time.
Keywords: Flight Simulation, Virtual Reality, Rigid-Body Kinematics, Quaternion, VRML,
Human-Computer Interface
1. Introduction
Virtual reality (VR) is a rapidly developing computer interface that strives to immerse the user
completely within an experimental simulation [1], thereby greatly enhancing the overall impact and
providing a much more intuitive link between the computer and the human participants. Virtual reality
has been applied successfully to hundreds if not thousands of scenarios in diverse areas including rapid
prototyping, manufacturing, scientific visualization, engineering, and education [2-4].
Flight simulation is widely used in plane design, performance test, pilot training and other aviation
researches [5]. Numbers of papers have discussed the flight simulation model. Flight simulation based on
OpenGL was designed in [6]. 3D flight simulation based on Virtools was realized in [7]. Direct 3D was
applied in flight simulation in [8]. Reference [9] presents a route planning method. Precise parameters
simulations were researched in [10]. UAS flight simulation with hardware-in-the-loop was developed in
[11]. There are limits in offline simulation without man-in-the-loop. Simulations in pilot training,
formation flight or air fight assessment require function of controllability. Simulation without visual
scene is unable to give ocular demonstration. Moreover, bespoke trajectory used in real-time track-totrack association and fusion is not convictive enough because of the game theory.
The paper build a MATLAB/Simulink simulation platform with man-in-the-loop to realize virtual
reality flight based on rigid-body kinematics. Joystick order input by operator can be transformed into
control signal. Position translation, rotation quaternion and other parameters calculated by flight motion
module were sent to 3D virtual reality space. Joystick input, flight motion module and 3D virtual reality
space were combined into a whole simulation system by Simulink 3D Animation Blockset. Operator can
use joystick to control virtual plane in visual scene. It is convenient to create simulation model with the
help of great calculation function and intensive module in MATLAB/Simulink. MATLAB development
environment provides convenience for function expanding. 3D virtual reality space based on VRML
requires moderate storage space. It is also suitable for transmission via network.
This paper introduced the architecture of the flight simulation platform in Section II. The approach
and technique to design joystick module, flight motion module based on rigid-body kinematics and
virtual reality scene are introduced in detail. The function of simulation platform is tested and verified
by simulation in Section III. Section IV concludes the paper and makes clear of future work.
2. Architecture of the flight simulation platform
International Journal of Digital Content Technology and its Applications(JDCTA)
Volume7,Number7,April 2013
doi:10.4156/jdcta.vol7.issue7.112
949
The Design of Virtual Reality Flight Simulation Platform Based on Rigidbody Kinematics
Gang WANG, Wei MEI, Heng LIU, Ying XIAO
The MATLAB/Simulink virtual reality flight simulation platform based on rigid-body kinematics
consists of three modules with man-in-the-loop as shown in Figure 1. Operators use joysticks input
orders which can manoeuvre the aircrafts in virtual reality world. Position translation, rotation
quaternion and other parameters can be calculated by flight motion module when orders input. Humancomputer interface displays vivid 3D scene of flight.
Figure 1. Total scheme of virtual reality flight simulation platform
2.1 Design of joystick module
An input device should be convenient and natural for a user to transmit information to a computer,
and should be designed from an understanding of the task to be performed and the interrelationship
between the task and the device from the perspective of the user. The design of an input device should
match the perceptual needs of the user. As a result, the integration of input devices should follow a user
needs analysis in order to map their expectations onto the attributes of the overall virtual reality system
[12]
. The Joystick Input block in Simulink 3D Animation Blockset provides a convenient connector
between a Simulink model and the real world joystick. The Joystick Input block uses axes, buttons, and
the point-of-view selector, if present. Its output ports reflect the status of the joystick controls for axes
and buttons. There are twelve control press-buttons on a real joystick generally as shown in Figure 2.
Four of the press-buttons called “Axes” and eight of them called “Buttons”. Each of “Axes” can output
the value of 0 (released), -1/ +1(pressed), and each of “Button” can output Boolean 0 (released), 1
(pressed).
Figure 2. Press-button function distribution
Because of the existence of error, Simulink blocks Dead Zone and Sign were used to ensure the
output value is 0, -1 or +1 exactly. The Data Type Conversion block converts the Boolean type input
signal to the data type of double. Selector block selects joystick input signal to control different
variable. As shown in Figure 2, axes 1-4 were chosen to control the plane to thrust, yaw right, brake or
yaw left. Buttons 1-4 were chosen to control the plane pitch up, roll right, pitch down or roll left.
2.2 Design of flight motion module
The system reference frames are introduced and shown in Figure 3. Earth-based inertial frame is
(North, East, and Earth core). V-Realm Builder frame will be referred in Part C.
950
The Design of Virtual Reality Flight Simulation Platform Based on Rigidbody Kinematics
Gang WANG, Wei MEI, Heng LIU, Ying XIAO
Figure 3. Earth-based inertial frame (Xe, Ye, Ze), aircraft body frame (Xf, Yf, Zf) and V-Realm Builder
frame (Xv, Yv, Zv)
System assumptions are elaborated as follows.
Assumption 1: The aircraft is assumed to be a rigid body. This implies that the distance between two
points of aircraft is time-invariant when measured with respect to the aircraft body frame. This
assumption is realistic for small-size or low-weight aircraft such as general aviation and fighter aircraft
typically.
Assumption 2: Flight control module is designed according to kinematic model. This assumption
might seem unrealistic considering the aerodynamics. However, kinematic model is the foundation of
aircraft dynamic model simulation. And it is acceptable to a certain extent when the platform is used
for formation flight training, airfight scenario demonstration, etc.
Positive direction linear acceleration (ax, ay, az) and angular velocity (P, Q, R) of body frame axes
are given from joystick module.
The positive direction linear velocity (U, V, W) in the body axes can be obtained by integration over
acceleration. Acceleration a and velocity Vplane can be calculated by formula:
a
ax 2  a y 2  a z 2
2
2
V plane  U  V  W
(1)
2
(2)
The Kinematic Equation in a matrix format is given by:
    1 sin  tan  cos  tan    P 
     0
 sin    Q 
cos 
 
 
    0 sin  sec  cos  sec    R 
(3)
where (ψ, Θ, Φ) is the Euler angle of aircraft.
The relationship between the components of the aircraft velocity in the body axes and the Earth
inertial frame is given by the following equation:
0 U 
 X  cos   sin  0 cos  0 sin 1 0
Y   sin  cos  0 0 1 0 0 cos   sin V 
  


 
0
1 sin  0 cos 0 sin  cos  W 
Z   0
(4)
Flight parameters can be obtained by following the process in Figure 4.
 ,  , 
 , , 
 X 

Y 
 Z 
 
V plane
X 
Y 
 
Z 
..D
Figure 4. Process of calculating flight parameters
951
The Design of Virtual Reality Flight Simulation Platform Based on Rigidbody Kinematics
Gang WANG, Wei MEI, Heng LIU, Ying XIAO
Rotation quaternion can be used to represent attitude angle [13]. An attitude or unit quaternion, q,
describes a single-axis rotation about the Euler-axis for the rotation angle α, the basic definition is:
   q1  



e
sin

  q2  
2
q  q1  q2 i  q3 j  q4 k  
    
 cos    q3  

2   q 
 4 
(5)
where [q1, q2, q3]T is its vector element, q4 is its scalar element and e is an unit vector ( e1= [0 0 1]T,
e2= [0 1 0] T , e3= [1 0 0] T ). A normalized quaternion obeys the unit length constraint:
(6)
q1  q2  q3  q4  1
Suppose “*” is the multiplication operator of two quaternions. The multiplication rule of
quaternions is given by the following:
t  qr
(7)
 ( q0  iq1  jq2  kq3 )  ( r0  ir1  jr2  kr3 )
 t0  it1  jt2  kt3
where
2
2
2
2
t0  (r0 q0  r1q1  r2 q2  r3q3 )
 t  (r q  r q  r q  r q )
1
0 1
1 0
2 3
3 2

t

r
q

r
q

r
q

r
(
0 2
1 3
2 0
3 q1 )
2
t3  (r0 q3  r1q2  r2 q1  r3q0 )
Suppose p1, p2, p3 are quaternions representing angle of yaw, pitch and roll. Thus the expression of
quaternion for representing attitude angles is given by the following:

0
0
  
   sin( / 2)  

  
  

0
0
    sin( / 2)    

Q  p1  p2  p3  
  

 sin( / 2)   
0
0




 
 

 cos( / 2)   cos( / 2)   cos( / 2) 






 
  sin( 2 ) sin( 2 ) cos( 2 )  cos( 2 ) cos( 2 ) sin( 2 )  


 cos(  ) sin(  ) cos(  )  sin(  ) cos(  ) sin(  )  

2
2
2
2
2
2 
 

 sin(  ) cos(  ) cos(  )  cos(  ) sin(  ) sin(  )  

2
2
2
2
2
2 






 
 cos( ) cos( ) cos( )  sin( ) sin( ) sin( ) 

2
2
2
2
2
2 
(8)
The aircraft will come across difficulty because of lack of object for reference when it is cruising.
Navigation function is required in the system. As shown in Figure 5, the coordinate of aircraft and
target in Earth-based frame is (XA, YA, ZA), (XT, YT, ZT) while the yaw angle and pitch angle is ψ, Θ at
the moment. Then the bias angles α, β between flight direction and target direction could be calculated
by following expression.
  
T

(9)
    

T
where
952
The Design of Virtual Reality Flight Simulation Platform Based on Rigidbody Kinematics
Gang WANG, Wei MEI, Heng LIU, Ying XIAO

Y Y
 T arctan A T
X A  XT



ZA
 T arctan
2
2
(
)
Y

Y

A
T ( X A  X T )
(10)
The distance between the aircraft and target can be obtained by the expression:
D  ( X A  X T )2  (YA  YT )2  (Z A  ZT )2
(11)


( X A,YA, Z A )


T
T
( X T ,YT , ZT )
Figure 5. Bias angles in flight navigation
2.3 Design of 3D virtual reality space
V-Realm Builder is a software that is used to design virtual worlds and draw/import 3D virtual
objects. Users can manipulate the objects and the virtual scene through MATLAB commands and
Simulink models to animate the scene after building the virtual world. Virtual 3D objects’ properties
such as translation, rotation, scale, and color can be changed through MATLAB commands and
Simulink models [14]. 3D virtual reality space based on VRML requires moderate storage space. It is
also suitable for transmission via network.
Node is the elementary unit of VRML which is used to describe objects and their attribute in virtual
reality space. Nodes can be organized in a structure of sequential hierarchy tree. Figure 6 shows the
development environment of building virtual reality world using V-Realm Builder. Nodes tree on the
left side of V-Realm builder contains editers of virtual 3D objects’ properties. Node “translation” is
used to adjust the objects’ position. “Rotation” sets the rotation angle in quaternion form. “Scale” is
used to adjust the size of objects. Subsidiary objects will be placed under “children” nodes. Light,
sound, fog and many other effects can be integrated into virtual reality world with the help of V-Realm
Builder.
Figure 6. Using V-Realm Builder to design virtual reality world
Attention need to pay on the coordinate system in V-Realm Builder because it is different from the
953
The Design of Virtual Reality Flight Simulation Platform Based on Rigidbody Kinematics
Gang WANG, Wei MEI, Heng LIU, Ying XIAO
Earth-based inertial frame as shown in Figure 2. Transformation expression is given by the following.
 XV  X e

YV   Z e
Z  Y
e
 V
(12)
VR Sink block of Simulink connects the flight motion module to the virtual reality world. VR
browser inside provides human-computer interface where operators can have a view of 3D space scene.
VR Tracer is used trace the trajectory of aircraft and draw 3D trajectory in the associated virtual scene.
Color of trajectory can be chosen by a vector. VR Text Output provides a method for editing HUD
(Head Up Display) text which will display on the top of screen interface. This helps a lot to inform
pilots the fight information as assistance. Gauges Blockset™ software is a collection of blocks that
provides graphical displays for monitoring signals in Simulink models. Using this product, the platform
can set up realistic-looking icons that are custom-designed for Simulink model and visually
representative of the flight parameters.
3. Simulation results
The simulation aims to test the performance of flight simulation platform. Operator should complete
the simulation setting before running the system. Plane model and landform should be chosen in the
setting.
The simulation is conducted in a PC with a Pentium 4, 3.00 GHz Dual Core processor, an ATI
Radeon HD 6570 graphics board and 4G RAM. The output device is a monitor with pixel of 1280×
1024.
Simulation results indicate that the scene view is verisimilar and virtual plane can complete flight
tasks under the control of operator. Flight scene and information display in the human-computer
interface as shown in Figure 7 and Figure 8. The top right corner of Figure 8 is navigation map which
provides guiding help for pilots. Figure 9 and Figure 10 portray the scenes of formation flight and
pursuit flight.
Figure 7. Scene of overlook viewpoint
954
The Design of Virtual Reality Flight Simulation Platform Based on Rigidbody Kinematics
Gang WANG, Wei MEI, Heng LIU, Ying XIAO
Figure 8. Cockpit scene of flight
Scene of form
mation flight
Figure 9. S
Figure 100. Scene of purrsuit flight
S
Single
plane fllight and form
mation flight exxperiments weere conducted tto test the function of trajecctory
draw
wing. Figure 11 shows thee 3D flight traajectory draw
wn in the virtuual reality woorld. Platform can
outpput accurate flight parameteers to MATLA
AB workspace for further ressearch. Figuree 12 is a singlee 3D
trajeectory plotted by MATLAB
B command. T
Two trajectoriees which are uused in the exxperiment of rrealtimee track-to-tracck association and fusion w
were plotted inn Figure 13. F
Furthermore, gauges displaay in
simuulation system
m work well ass depicted in F
Figure 14.
955
The Design of Virtual Reality Flight Simulation Platform Based on Rigidbody Kinematics
Gang WANG, Wei MEI, Heng LIU, Ying XIAO
Figure 11. 3D trajectory in virtual reality world
Figure 12. 3D trajectory of single plane plotted by MATLAB command
Figure 13. 3D trajectories of two planes plotted by MATLAB command
Figure 14. Gauges display
956
The Design of Virtual Reality Flight Simulation Platform Based on Rigidbody Kinematics
Gang WANG, Wei MEI, Heng LIU, Ying XIAO
4. Conclusions and future work
The paper has investigated the simulation of virtual reality flight and the platform based on rigidbody kinematics was designed. The simulation system has the advantages of convenience of
programming, scalability, and maneuverability. Results of simulated flight task demonstrated that the
simulation platform can realize piloting task under the control of multiple operators with good
maneuverability and verisimilar visual human-computer interface when 3D flight trajectory was
drawing in the 3D space at the same time. The virtual reality flight simulation platform can provide
support as a “virtual testbed” for formation training, air fight assessment, real-time track-to-track
association and fusion, maneuvering targets tracking, etc.
Future work is to develop more functionality and increase system immersion. Experiments
supported by this platform will be conducted.
Acknowledgment
This work is co-supported by Chinese NSFC under Grant 61141009 and the National Defense
Technology Fund 40405070102.
References
[1] Yingcheng Xu, Li Wang, Guoping Xia, “Modeling and Visualization of Dam Construction Process
Based on Virtual Reality”, AISS: Advances in Information Sciences and Service Sciences, Vol. 3,
No. 4, pp. 76-88, 2011.
[2] Abhishek Seth, Judy M. Vance, James H. Oliver, “Virtual reality for assembly methods
prototyping: a review”, Virtual Reality, Vol 5, No. 1, pp. 5-20, 2011.
[3] Yan Zhang, Xun Zhang, Ying Chen, “The Similarity Degree Comparison Study of Travel Routes
between the VR Digital Models and the Reality Environment in Urban Planning”, JCIT: Journal of
Convergence Information Technology, Vol. 7, No. 15, pp. 116-123, 2012.
[4] Xiang LIU, Jinhai SUN, Yaping HE, Yimin LIU,Li CAO, “Overview of Virtual Reality Apply to
Sports”, JCIT: Journal of Convergence Information Technology, Vol. 6, No. 12, pp. 1-7, 2011.
[5] Marcello R.Napolitano, “Aircraft Dynamics: From Modeling to Simulation”, Hoboken: John
Wiley & Sons, In c., USA, 2012.
[6] Yao Hongge, “The Realization of Flight Simulation System Based on OpenGL”, 2010 Second
International Conference on Information Technology and Computer Science, pp. 198-201, 2010.
[7] Ying Xiao, Wei Mei and Xiaowei Zhang, “3D flight simulation based on Virtools”, 3rd
International Asia Conference on Informatics in Control, Automation and Robotics, pp. 343-349,
2011.
[8] Xu Hailiang, Li Junyang and Fei Shumin, “Design and implementation of digital flight simulation
platform”, Journal of Southeast University (Natural Science Edition), Vol. 41, No. 1, pp. 113-117,
2011.
[9] Ji Bing, Shan Ganlin and Chen Hai, “Visual Simulation Method Based on VRML and Target
Trajectory”, Journal of System Simulation, Vol.23, No.9, pp. 1900-1904, 2011.
[10] Florian Holzapfe and Stephan Theil, “Advances in Aerospace Guidance, Navigation and Control”,
Berlin: Springer-Verlag Berlin Heidelberg, Germany, 2011.
[11] Jeffery Saunders, Randal Beard, “UAS Flight Simulation with Hardware-in-the-loop Testing and
Vision Generation”, J Intell Robot Syst, Vol. 57, No. 1-4, pp. 407-415, 2010.
[12] Alex W.Stedmonn, HarshadaPatel, SarahC.Sharples, JohnR.Wilson, “Developing speech input for
virtual reality applications: Arealitybased interaction approach”, Int. J. Human-Computer Studies,
vol 69, pp. 3-8, 2011.
[13] Xu Yan-ke, Liang Xiao-geng, Jia Xiao-hong, “The Design of High Attack-angle Flying Missile
Control System Based on Feedback Linearization”, JCIT: Journal of Convergence Information
Technology, Vol. 7, No. 3, pp. 155-163, 2012.
[14] Nassim Khaled, “Virtual Reality and Animation for MATLAB and Simulink Users: Visualization
of Dynamic Models and Control Simulations”, Springer-Verlag Berlin Heidelberg, USA, 2012.
957