Assembly Simulation on Collaborative Haptic
Virtual Environments
Rosa Iglesias, Elisa Prada
Sara Casado, Teresa Gutierrez
Ainhoa Uribe, Alejandro Garcia-Alonso
Fundacion Labein
Universidad del Pais Vasco (UPV-EHU)
Outline
•
Objectives
•
Haptic devices, haptic rendering
•
Haptic Assembly Simulation
•
Collaborative Haptic Assembly Simulation
•
Network Topologies
•
Client/Server (two implementations)
•
P2P
•
Conclusions
Collaborative Haptic V Environment for Assembly Tasks
GOAL: Create a Distributed Virtual Environment for collaboration,
whereby users can carry out assembly and maintenance tasks
• VE Assembly + simultaneously + haptic devices
• to design and evaluate computer generated mock-ups prior to
building any physical prototype.
Haptic devices
Haptic rendering
• “Haptic rendering is the process of computing
and generating forces in response to user
interactions with virtual objects” (Salisbury,
1995).
• “Haptic rendering allows users to “feel” virtual
objects in a simulated environment” (Salisbury,
2004)
– To touch objects, move them,…
– To create different haptic effects: texture perception,
deformable elements or collision impacts.
• Haptic devices require an update frequency of
1000 Hz.
Haptic rendering
• An update frequency of 1 KHz means:
1 kHz
Touching a sphere
Touch:
Normal
Vector
-check if P is inside
User’s point
Sphere Section
Get position P
-if inside, compute the
contact point on the
surface
-compute force
Send force
Haptic
HapticThread
loop
Haptic assembly simulation (HAS)
• Description
video
Introduction: HAS
• Examples of assembly constraints:
– Assembly along an axis
Assembly
Line
Hole
Hole
Pin
Pin
– Assembly along a common plane
Assembly Plane
Haptics require new architectures
• Classical simulation loop :
– Process input events (move objects)
– Simulation
– Image rendering
– Haptic rendering ???  it is not possible
• Due to high frequency of haptic rendering:
required two independent processes (threads).
HAS : two processes
Simulation (20-60Hz)
Haptic (1000 Hz)
•
Get last position
•
•
Simulation (collisions,
constraints, …)
Image rendering
•
•
•
•
•
Send “haptic requirements”
•
Read last position (from
haptic device)
Send last position
“haptic requirements”
received ?
Compute force (haptic
rendering)
Send force to haptic device
Problems : (1) haptic cycles without simulation updates,
(2) updates mean brusque changes (haptic feedback instability)
Objectives & Challenges
-
Broad
objective:
Several
users
can
work
simultaneously to achieve a common goal using
haptic devices in assembly operations.
-
Challenges:
•
Consistency (virtual scene synchronization) must be
guaranteed, because users simultaneously interact
with the same scene
•
Effective and stable haptic feedback when users collide
or assemble their grasped objects.
•
Scalability: number of users, virtual objects and so on.
Objectives & Challenges
•
KEY FACTOR: Network conditions
– Delay
– Variation of delay (jitter)
– Message Loss,….
•
They affect: VE consistency and user perception
– Visually:
• Sluggish scene update of the remote object
– Haptically
• Unstable haptic feedback
Objectives & Challenges in CHAS
•
GOAL: Collaborative Haptic Assembly Simulator
(CHAS), whereby two users can simultaneously
carry out assembly tasks using haptic devices.
•
Two types of interaction:
– INDEPENDENT INTERACTION:
interacting with static objects (not grasped).
– DEPENDENT INTERACTION:
depends on other user’s action.
Objectives & Challenges in CHAS
•
Two types of dependent interaction:
– Dependent collision:
when both grasped objects collide
– Remote assembly:
when a grasped object is being assembled into other grasped
object
CHALLENGES in CHAS
In the case of a remote assembly + dep. coll.:
• consistency must be guaranteed.
• each user must receive adequate haptic feedback
Network topology
To build a distributed VE:
Client/Server (C/S)
Peer-to-Peer (P2P)
C/S
Mixture of them
P2P
Multiple
Servers
Client/Server (A1)
•
PROS: Consistency is automatically satisfied,
since data are validated centrally,
and then distributed to clients.
•
Simulation
CONS: computations for simulation areCentral DB Simulation
Simulation
Administration
made sequentially
Server
DB
Administration
Network
Simulation or validation:
check if new object
position is colliding or
computes the assembly
constraints
Visualization
Interaction
Local DB
Local
DB
Interaction
Visualisation
Client 2
Client 1
A1
Sound Sight Touch
Client n
Client/Server (A3)
Observing its disadvantage: sequential computation.
A new C/S architecture was designed.
• PROS: well-balanced distribution of the workload
and parallel computation.
•
Simulation
CONS: efficiency comes from the worst
computer conditions
Consistency: server checks
if there is any inter-object
penetration. In that case,
synchronise user’s scenes
Consistency
Central DB
Server
Consistency
Administration
DB
Administration
Network
Visualization
Interaction
Local DB
Local
DB Visualisation
Interaction
Client 2
Client n
Simulation
Simulation
Client 1
A2
Sound Sight Touch
Network topology: Analysis using C/S
RESULTS using C/S architectures:
•
With A1 and A3:
several users can simultaneously interact using keyboard/mouse.
•
A3 seemed to have better performance when > 2 users
•
A user can haptically interact while others watch with A3.
•
After experiments, it was shown that simulation/validation must be
placed at each user for avoiding unstable haptic feedback, even
with independent interaction.
•
A1 and A3 are very sensitive to the network delay.
Network topology: Analysis using C/S
•
They address:
– collaboration taking turns (any delay)
– cooperative manipulation = a carrying task
• Two users grasping the same object.
• Only low-delay interaction between both users, because
haptic interaction is affected.
•
Two goals still not achieved :
– simultaneous assembly tasks
– high delay
Network topology: Analysis C/S vs P2P
Comparing C/S and P2P:
• C/S: Consistency  is more easily achieved.
Local scene update after a round-trip delay to
the server.
• P2P: Consistency  challenge.
Update delay is only one-way 
- Updates are less dependent on network conditions.
- Permit a better performance with haptic interaction.
- Potential to scale to a larger number of users.
Then, a P2P architecture is chosen
Consistency-maintenance scheme
• In our research, to build the Collaborative Haptic
Assembly Simulator (CHAS), whereby two users
can simultaneously carry out assembly tasks using
haptic devices.
• A new consistency-maintenance scheme has been
designed and tested.
Consistency-maintenance scheme
Main features:
• Gives priority to the validation of interactions between
objects grasped by users for fast haptic response.
• Small messages size: < 200 bytes
• Not need to store and manage a history of movements.
• Not specific messages to deal with inconsistency because
each user manages consistency locally.
Experimental set-up
• Many experiments were carried out varying end-to-
end delays: less than 1 ms, 50 ms, 100 ms, 200 ms
and with jitter (0 ms – 100 ms).
• The experimental platform:
NetDisturb
Switch 1
Switch 2
Experimental set-up
• The experimental platform:
– Experimental platform: 3 computers via a
100Mbps links connected by two high-speed
switches.
– Two computers are each connected to an OMNI
device.
– The other computer to simulate the different
values of delay and jitter.
NetDisturb
Switch 1
Switch 2
Results between Labein & QUB: VE & tasks
Virtual scene: Electrical box for an aircraft engine
• User Labein grasps the handle
• User Queen’s grasps
the screw
• Perform collisions
• Assemble both grasped
objects
CAD design: Electrical
box for an aircraft
engine by SENER
USER QUEEN’S
USER LABEIN
Results between Labein & QUB: Consistency
• Consistency after a dependent collision
SCREW X-POSITION (LABEIN & QUEEN'S) ( First 1200 ms)
mm
time(ms)
501
1001
-10
-30
-50
screw at User Queen's
screw at user Labein
Results between Labein & QUB: Forces
• Collision force magnitude
after a dependent collision
User Queen's (screw): collision force magnitude
(Dependent Collision)
averaged force
Newton
1,0
0,8
0,6
0,4
0,2
0,0
1
11
21
31
41
51
61
71
81
91
time (ms)
Results between Labein & QUB: Forces
• Force magnitudes
during a remote assembly
User Queen's (screw): assembly force magnitude
Newton
insertion
0,9
0,6
0,3
0
1
401
801
1201
1601
2001
2401
2801
3201
time (ms)
Results between Labein & QUB: Conclusions
• With the consistency-maintenance scheme in CHAS, users
had a realistic sensation of the VE and the performance was
satisfactory.
• CHAS provided an adequate haptic interaction when both
users perform remote assemblies or dependent collisions.
• Such haptic feedback showed that the sense of co-presence
between the users continue to improve, with regard to only
visual feedback.
THE END
QUESTIONS ?
THANK YOU VERY MUCH !
MUCHAS GRACIAS !
ESKERRIK ASKO !
DONOSTIA-SAN SEBASTIAN