Augmented Reality Based Factory Model Comparison Method
Wei-wei Sun 1, Jian-feng Lu 1, De-zhong Li 2
1
CIMS Research Center, Tongji University, Shanghai, China
2
CDHAW, Tongji University, Shanghai, China
(sunweiweid@126.com, lujianfeng@tongji.edu.cn, dezhonglee@163.com)
Abstract - Through factory digital mock-up, Digital
Factory (DF) technology can save enormous time and cost in
factory planning. A problem of the digital factory mock-up
maintenance is checking the digital models with the real
factory. This paper introduces a method using Augmented
Reality (AR) technology to compare the 3D models with the
real object in real time. Compared to other measures, this
method have the benefit of the cost saving. An experiment
demonstrates the proposed method is given at the end of the
paper.
Keyword - digital factory mock-up, model comparison,
Augmented Reality, ARToolKit
I. INTRODUCTION
The factory digital mock-up creates a visual
simulation platform for product design and processes
planning which has been the key point to optimize
processes and offer optimal production scheme [1]. It
works as the foundation of Digital Factory (DF)
technology which is widely used in many fields such as
aviation, automobile manufacturing, chemical industry
and electronic products. Model calibration is a significant
issue in the application of factory digital mock-up.
Previous methods of model comparison perform
inefficient and high-cost, which result in a deep impact on
the development of DF. With the expectation of seeking a
cost-effective way, Augmented Reality (AR) is proposed
to be used in factory digital model comparison [2][3].
II. FACTORY DIGITAL MOCK-UP
Factory digital mock-up is a complex of digital
archive for the whole life-cycle of a factory. It includes
not only 3D model of the factory, but also all the design
documents, construction documents, and maintenance
information. DF technology, the company to meet the
challenges of the 21st century an effective means [4],
which integrates techniques of computer, virtualization,
emulation, and networks, plays a significant role in
keeping competitive advantage for enterprises. It operates
in a collaborative way under 3-D visualization
environment and interactive interface. Based on the actual
data and models the planned products and production
processes can be improved using virtual models until the
processes are fully developed and extensively tested for
their use in the real factory. DF is a comprehensive
approach in factory layout planning, which consists of the
3-D model design of plant (that contains workshop
structure, equipments and facilities, material flow and
other resources for production) and processes
optimization [5]. And Factory digital mock-up acts as the
prerequisite for operative information concerned.
Factory digital mock-up shows its advantages:
engineers make assessment by optimizing the plant layout
and resolving conflicts between different parts, then avoid
loss due to irrational design, and make data and
information of equipments and process flow optimum
coordinate with the factory building [6]. When applied in
the aspect of automobile industry, it coordinates materials
resource (components and modules of automobile),
equipment (machine tools and facilities), workshop (area),
and process flow (automobile manufacturing processes)
into an IT system. While in the field of pharmaceutical
and chemical industry, factory digital mock-up makes it
possible to increase product innovation and flexibility.
III. METHODOLOGY
A. Method of factory digital mock-up comparison
One of the most basic problems currently limiting
factory digital mock-up applications is updating. Regular
calibrating factory models taking real factory for
reference is necessary. There are several model
comparison methods prevailed: laser scanning, laser
ranging, photograph-visual inspection comparison.
Laser scanner firstly gets the outline integrated data of
the object rapidly by Omni-directional scanning, and then
generates point cloud records after precisely construction,
editing, and modification by computer. Accurate as the
data is, the method cannot be widely used because of the
high cost. Furthermore, the instrument is unable to display
data instantly.
Another approach for updating factory digital mockup is laser ranging. People can easily obtain elevation and
other information of the object, relative position, for
example, using the hand-held laser distance meter.
However, this kind of method is not appropriate for
objects which are precise and complicated.
Photograph-visual inspection means to make a
comparison between real factory and the photograph of
the factory. It is feasible, however, not accurate.
These methods above are commonly used at present.
However, more efforts should be paid to explore new
resort which is inexpensive and accurate. Therefore,
Augmented Reality based factory model comparison
method has been proposed. AR system can present a view
of blended scene of real factory environment and digital
mock-up. Then the information files can be easily
changed to correct the model without complicated manual
operation.
seamlessly with a real factory environment in 3-D. The
mainly work flow is shown in Figure 1.
B. Augmented Reality and ARToolKit
Augmented Reality (AR) technology can enhance
user’s perception of the real world by providing
information from computer system. It has the following
three characteristics: combine real and virtual, interactive
in real time, and registered in 3-D [7][8]. AR system has
been applied in medical, manufacturing, visualization,
path planning, entertainment, military and many other
aspect [9].
ARToolKit is an AR application development tool kit
based on C/C++ language. It has successfully developed
indoor registration technology with fiducial mark pattern
tracking system. On the condition of controlled
environment, it has achieved a fine tracking result. The kit
includes camera calibration and mark making tool which
can compound Direct3D, OpenGL graphics and VRML
scenes into the video stream and support a variety of
display devices.
C. Augmented Reality based factory model comparison
method
Train mark
Identify
mark
Load model
Fig.2. The real object scanning
Take the case of a section of a cross fire fighting
sprinkler and a pipe support in a classroom (as Figure 2
shows) which is too high to measure, just meet the
requirements of the experiment object selection.
The operating system is Windows XP with Microsoft
Visual studio 2008 development environment. An
ordinary CMOS camera (pixels 320×240) with 2.0 USB
interface and a printed fiducial mark is enough.
Besides provided mark patterns, other patterns also
can be designed and trained according with the instruction
in ARToolKit. There are some limitations to purely
computer vision based AR systems. The larger the
physical pattern, the further away the pattern can be
detected. What’s more, the simple pattern is better.
Taking the height of experimental subject into account, a
proper coordinate axis offset should be set to get a clearly
vision. The mark was fixed in the bottom right-hand
corner of the object which is also shown in Figure 2.
DAT files contain the name, size display, rotation and
other information of the models. Some new models can be
matched with modifying the data as well as model
updating. When multiple patterns tracked associate with
different 3D object needed, DAT files also can be easily
fixed to load more than one pattern.
Modify
Blend
N
Y
End
Fig.1. AR based model comparison method work process
Fig.3. The initial vision of blending
Augmented Reality based factory model comparison
method is put forward on the foundation of model-loaded
procedure in ARToolKit to blend virtual objects
The virtual model of the classroom was created by
Microstation in real proportion and converted into WRL
by 3DSMAX. The mark shown in Figure 2 has been
trained to match the digital model. The initial vision of
the blending scene was rendered as soon as the camera
identified the mark, which is shown in Figure 3.
The model and the vision of real object can be rotated
synchronously when the camera moved. While immersed
in a view of real world, some subtle changes in DAT files
can make the model get closer to the object. Then the
position can be determined with acceptable precision and
accuracy in real time. The modified model was shown in
Figure 4.
Fig.4. The modified vision of blending
V. CONCLUSION AND FUTURE WORK
From the conclusions drawn from experiment above,
Augmented Reality based factory model comparison
method has the following advantages over the existing
methods:
(1). Flexibility. The method operated in a direct way
which has avoided laser scanning, ranging, or other
manual work.
(2). Low-cost. It can do the work well without
professional instrument and also result in time-saving.
(3). Accuracy. The real object was blended with the
virtual model seamlessly, that lead to errorless result.
(4). Real-time. The real world and the digital model
were rendered and combined in real time, and it is
possible to reduce time with the use of the method by
working more efficiently.
Certainly, there are also some limitations. The
method cannot work without document database of the
object, while Laser scanning is better. And Laser ranging
does well in presenting data like elevation and relative
position for some big facilities. It is more efficiently to
make a proper combination of these approaches in the real
factory.
Since visual interface has advanced the performance,
more effort needed to perfect the AR system, which
includes tracking visible natural features without prepared
marks [10], model loaded automatically, project of the
controller interface with CAD and so on. The author will
do further study in this area to support the factory digital
modeling techniques in future.
REFERENCES
[1] U. Bracht, T.Masurat. The digital factory between vision
and reality [J]. Computers in Industry, 2005,56(4): 325-333.
[2] AZUMA R T. A Survey of Augmented Reality [J].
Presence;Teleperators
and
Virtual
Environments,
1997,6(4):355-385.
[3] AZUMA R T, BAILLOT Y,BEHRINGER R,et al,Recent
advances in augment reality [J].IEEE Computer Graphics
and Application, 2001. 21(6);34-47
[4] Liu Duangui. About Digital Factory [DB/OL].
2009.www.ai.vogel.com.cn
[5] Zhang Hao, Fan Liuqun, Ma Yumin. The technology and
application of Digital Factory (Chinese) [M]. Beijing:
China Machine Press, 2006: 28-29
[6] Yu Chenglong, Guo Gang, Liu Jiabao. “Digital Mock-up
(DMU) technology in the development of product
innovation”. Sichuan Ordnance Journal, 2003,pp. 45-48
[7] Ronald T. Azuma. A Survey of Augmented Reality [J].
Presence: Teleoperators and Virtual Environments , 1997,
6(4): 355–385.
[8] Ronald Azuma, Yohan Baillot, Reinhold Behringer, Steven
Feiner, Simon Julier, and Blair MacIntyre. Recent advances
in Augmented Reality [J]. IEEE Computer Graphics and
Applications, 2001, 21(6): 34~47.
[9] Quan Hongyan, Wang Changbo, Ling junjun.“Survey of
vision-based augmented reality technologies”. Robot, 2008,
pp.379-384.
[10] U. Neumann and S. You, “Natural feature tracking for
Augmented Reality,” IEEE Trans. Multimedia, vol. 1, no.
1, Mar. 1999, pp. 53-64.