Joint Advanced Students School 2005
Replication and synchronization techniques in the context of
mobile applications
By Alexander Stage
Software Engineering betrieblicher Informationssysteme (sebis)
Ernst Denert-Stiftungslehrstuhl
Lehrstuhl für Informatik 19
Institut für Informatik
TU München
wwwmatthes.in.tum.de
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 1
Agenda
 Motivation
 Mobile Computing
 Synchronization Techniques
 Replication Techniques in Databases
 Oracle Lite 10g – A Mobile Database Solution
 Peer to Peer Mobile Computing
 Future Applications and Developments
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 2
Motivation – A Scenario
The disconnected sales-person problem: a typical day
Mobile Device
Local DB
Company
Mobile Device
Local DB
Central DB
travelling path
no communication link
Mobile Device
Local DB
Customer A
JASS 05 Synchronization and Replication in the Context of Mobile Applications
Customer B
Customer C
© Alexander Stage 3
Agenda
 Motivation
 Mobile Computing
 Synchronization Techniques
 Replication Techniques in Databases
 Oracle Lite 10g – A Mobile Database Solution
 Peer to Peer Mobile Computing
 Future Applications and Developments
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 4
Mobile Computing - Characteristics
 Mobile devices:
 Laptops
 Palmtops
 Smart cell phones
 Requirements:
 Data access:
- Anywhere
- Anytime
 Nomadic users
 Constraints:
 Limited ressources
 Variable connectivty:
- Performance
- Reliability
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 5
Mobile Computing – Communication Models
Client Server Model
Client
Client
Peer to Peer Model
Client/
Server
Client/
Server
Client/
Server
Client/
Server
Server
Client
Client
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 6
Two Types of Synchronization
Process synchronization
Process A
Process B
modification
Time
Data item 1
Synchronization
Data synchronization
System A
System B
Process
Process
modification
Data Item 1
modification
Data Item 1
Process A
modification
Time
correspondance
Data item 1
Process B
Main System
modification
Data item 1
JASS 05 Synchronization and Replication in the Context of Mobile Applications
Data Item 1
© Alexander Stage 7
Aspects of Synchronization
 Amount of participating system:
 1:1
 1:n
 Amount of exchanged data:
 Incremental
- Usage of timestamps, attached to data items
- Comparison of sub data structures and exchange
 Full
 Conflict resolution mechanisms
 Questions: who wins ?
 Merging
 System time synchronization in distributed systems
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 8
Merging Data During Synchronization: The Product Sales Example
System A
System B
Order 50 units
of product A
Order 25 units
of product A
Stock Product A: 100
Stock Product A: 100
System A
System B
Order Product A: 50
Order Product A: 25
Stock Product A: 50
Stock Product A: 75
No communiation
Apply updates
Main System
Main System
Stock Product A: 100
Stock Product A: 25
Reduce stock,
Insert order
Reduce stock,
Insert order
No conflict, minimum stock is preserved
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 9
Synchronization solutions
 File system oriented:
 Unix tool rsync:
- Incremental exchange of files
 Coda file system
 Version control systems:
 CVS
 WEBDav protocol:
- HTTP based file transfer protocol
- Locking and Versioning included
 Mobile Application oriented:
 Microsoft Active Sync:
- Synchronization between Desktop and Windows CE mobile devices
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 10
Standardization Efforts in Data Synchronization:
SyncML
 SyncML defines client server based protocol for synchronization
 SyncML is XML based, main elements:
 SyncMessage
 Sync Commands:
- Modification commands: add, copy, delete, put and replace
- Query commands: read and search
- Response commands: status and results
- Execution specific commands: sequence and atomic
 Synchronization models:
 Two way sync
 Slow sync
 One way sync
 Refresh sync
 Server-alerted sync
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 11
SyncML Architecture
Source: SyncML Representation Protocol, version 1.0.1, http://www.syncml.org/docs/syncml_represent_v101_20010615.pdf
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 12
Agenda
 Motivation
 Mobile Computing
 Synchronization Techniques
 Replication Techniques in Databases
 Oracle Lite 10g – A Mobile Database Solution
 Peer to Peer Mobile Computing
 Future Applications and Developments
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 13
What is Replication ?
 Usage of multiple copies of server system (replicas)
 Goals:
 Higher availability
 Better performance
 Areas of usage: Distributed Systems and Databases
Replicated Server Only
........
Server 1
Server n
Replicated Ressource and Server
Server 1 ........
Server n
Ressource
Ressource 1 ........ Ressource n
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 14
Replication in Databases
 Replication of the resource
replicating the database
 Elimination of single point of failure
 Single server behaviour
 Correctness criterion: one-copy serializability
 Synchronization in databases: replication
 Constant trade-off between consistency and efficiency
 Update anywhere, anytime, any way replication scheme
 Simple case:
- Only a few replica nodes
- Low transaction rates
- Small amount of deadlocks and reconciliation
 System scale up:
- Problem complexity grows drastically at cubic rates
- Inconsistent replicas and system delusion (no way to repair the inconsistencies)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 15
Replication in Databases – A Classification Scheme
 Classification dimensions:
 Where
- Dedicated replica:
- Single Master (primary master): primary and secondaries
- Every replica:
- Multi Master (update everywhere)
 When
- Synchronous (eager replication)
- Asynchronous (lazy replication)
 Other (not considered here)
- Interaction activities (network traffic)
- Constant or linear growth
- Transaction termination:
- Voting or Non-Voting techniques
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 16
Abstract Functional Replication Model
 Distinction between different replication solutions:
 Phase ordering
 Phase skipping
Request
Phase:
Server
Coordination
Phase:
Execution
Phase:
Agreement
Coordination
Phase:
Client
Response
Phase:
Client sends a
request to
one or
all replicas
Replicas
coordinate
their work
with each other
(ordering of
Execution
steps)
Replicas
execute
operation
Replicas ensure
that they all
have done
the same
thing (2pc)
Client receives
a response
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 17
Synchronous Single Master Replication I
 Characteristics
 Keeps all replicas synchronous in consistent state
 No update anomalies
 Reduces performance
 Increase response times
 Possible solution
 Trigger on selected tables
 Updates remotely all replicas
 Fails if one replica fails
 Usage of 2pc
 Implication:
- No control over when replication should take place
- Only databases with 2pc support
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 18
Synchronous Single Master Replication II
(primary)
(secondary)
(secondary)
Source: Wiesmann, Pedone, Schiper, Kemme, Alonso: Understanding Replication in Databases and Distributed Systems,
Proceedings of 20th International Conference on Distributed Computing Systems (ICDCS'2000)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 19
Asynchronous Single Master Replication I
 Characteristics
 Response after primary processed request:
- Primary takes care about transaction ordering
 Drawback: stale data items versions
 Performance is main goal
 Possible solutions
 Database system log based propagation:
- Committed updates only
- Restriction: Wait for current transactions
- Further improvement: only send data fields
 Log table:
- Trigger records update and insert actions
- Periodically recreate log table
- Send log table to replicas (preserving oder)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 20
Asynchronous Single Master Replication II
(primary)
(secondary)
(secondary)
Source: Wiesmann, Pedone, Schiper, Kemme, Alonso: Understanding Replication in Databases and Distributed Systems,
Proceedings of 20th International Conference on Distributed Computing Systems (ICDCS'2000)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 21
Handling Failures
 Two types of failure:
 Replica fails:
- Secondary
 System not affected
 Secondary catch up
- Primary
 Re-election of primary
 Most up-to-date replica: common system state
- Possible data loss
 Network partition
- Problem: Which partition is allowed to process further requests ?
 Majority consensus algorithms
 Quorum consensus algorithms
 Danger: Dramatic system performance loss
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 22
Synchronous Multi Master Replication I
 Similar to single master synchronous replication
 Updates allowed at every replica
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 23
Synchronous Multi Master Replication II
Source: Wiesmann, Pedone, Schiper, Kemme, Alonso: Understanding Replication in Databases and Distributed Systems,
Proceedings of 20th International Conference on Distributed Computing Systems (ICDCS'2000)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 24
Asynchronous Multi Master Replication I
 Characteristics
 Multi version concurrency control mechanisms
 Conflicts
 Reconciliation
- Automatic
- Manually
 Possible solution
 Unique timestamps
- Concatenation of local replica system clock to replica identifier
- Thomas Write Rule:
 Update applied if timestamp numerical greater than local timestamp
- Also known as optimistic replication
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 25
Asynchronous Multi Master Replication II
(primary)
(primary)
(primary)
Source: Wiesmann, Pedone, Schiper, Kemme, Alonso: Understanding Replication in Databases and Distributed Systems,
Proceedings of 20th International Conference on Distributed Computing Systems (ICDCS'2000)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 26
Replication Requirements in Mobile Context
 Disconnected operations require
 Asynchronous Multi Master Replication
 More frequent reconciliation because of
- Lazy replication
- Message delays
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 27
Replication Techniques - Summary
 Replication classification scheme
Update location
Update propagation
Eager / synchronous
Lazy / asynchronous
primary server
primary server
Eager / synchronous
Lazy / asynchromnous
Update everywhere
Update everywhere
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 28
Agenda
 Motivation
 Mobile Computing
 Synchronization techniques
 Replication techniques in databases
 Oracle Lite 10g – A Mobile Database Solution
 Peer to Peer Mobile Computing
 Future Applications and Developments
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 29
Oracle Lite 10g – A Mobile Database Solution
 Supports disconnected mobile database applications
 Local database (snapshot) on mobile device
 Content: subset of enterprise database
 Synchronization with enterprise database
 Central component: Mobile Server
 Application Model
 Publication
- Deployed on Mobile Server
- Distribution of application code
- Distribution of SQL defined snapshots (controllable by variable)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 30
Oracle Lite 10g Architecture
Source: Oracle® Database Lite Developer's Guide 10g (10.0.0) Part No. B13788-0
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 31
Oracle Lite 10g Synchronization Process I
 Snapshot
 Keeps track of local modifications
 Three types of publication items (application depended)
 Complete refresh
- All data items
 Fast refresh
- Incremental (updates and inserts)
 Queue based refresh
- No refresh (e.g. data collection: newly created data items)
 Asynchronous synchronization
 Central component Mobile Server
 Via „In queues“ and „Out queues“
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 32
Oracle Lite 10g Synchronization Process II
Source: Oracle® Database Lite Developer's Guide 10g (10.0.0) Part No. B13788-0
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 33
Oracle Lite 10g Evaluation
 Well suited for „build from scratch“
 Enterprise database directly accessible from Mobile Server
 „Mobile only“ or „Designed for mobility“ applications
 Existing data access layers not supported (EJB, CORBA, other)
 Duplication of mission-critical data access logic
 Possible code portation efforts
 Maintenance efforts increase
 Benefits reduced or outweighted
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 34
Agenda
 Motivation
 Mobile Computing
 Synchronization techniques
 Replication techniques in databases
 Oracle Lite 10g – A Mobile Database Solution
 Peer to Peer Mobile Computing
 Future Applications and Developments
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 35
Peer to Peer Mobile Computing
 Shortcommings of client server model
 No true mobility
 No communication between mobile devices
 Requirements
 Any to any communication
- All devices equal
 Large replication factors
 Careful data replication
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 36
The Ward Model I
Ward 2
Ward 3
Ward member
Ward 1
Ward master
Overlapped ward member
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 37
The Ward Model II
 Dynamically reconfigurable synchronization topology
 All units equal mobile peers:
 Intra ward mobility
 Inter ward mobility
- Ward changing (heavyweight)
- Ward overlapping (lightweight)
 Ward master:
 Re-electable
 Knows ward members
 No storage of ward member data
 Manages data import / export for ward
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 38
The Ward Model III
 Second, top-level ward: all ward master
 Ward set: data of all ward members within one ward
 Changes over time
 Optimistically replicated within ward
 Selective replication: intra- and inter-ward
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 39
Two Tier Replication
 Characteristics
 Comparable with ward model
 Static backbone of stationary replicated database servers
 Mobile replicas connect eventually at all stationary replicas
 Benefit
 Compromise between efficiency and true mobility
 Possible improvement:
 Combination with ward model (selective replication)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 40
Agenda
 Motivation
 Mobile Computing
 Synchronization techniques
 Replication techniques in databases
 Oracle Lite 10g – A Mobile Database Solution
 Peer to Peer Mobile Computing
 Future Applications and Developments
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 41
Future Applications and Developments
 Car to car communication (http://www.car-to-car.org)
 Mobile network of communicating cars
 Security services
 Development of replication schemes, based on ward model ?
 Moving wards
 Multi level ward hierachy
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 42
Thanks for your attention !
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 43
Discussion
 Try to describe other real world usage scenarios of mobile applications, where
synchronization and replication techniques need to be used
 Which technologies could be helpful for application level data replication and
integration. Think of technologies in the field of Enterprise Application Integration
(EAI). Which advantages and disadvantages can be identified with respect to
requirements in mobile applications
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 44
Appendix I: SyncML Example
Source: SyncML Representation Protocol, version 1.0.1, http://www.syncml.org/docs/syncml_represent_v101_20010615.pdf
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 45
Appendix II: Majority Consensus Algorithm I
Secondary 1
Secondary 1
Secondary 2
Network
Partition
Secondary 2
Primary
Primary
Secondary 3
Secondary 4
Secondary 4
Secondary 3
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 46
Appendix II: Majority Consensus Algorithm II
Secondary 2
Secondary 1
2,5 <
Inactive
Partition
2
3
Working
partition
Old Primary
New Primary
Secondary 4
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 47
Appendix III: Quorum Consensus Algorithm I
Weight: 1
Secondary 1
Weight: 2
Primary
Weight: 1
Secondary 1
Primary
Weight: 2
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 48
Appendix III: Quorum Consensus Algorithm II
Partition weight = 2
Working
partition
Weight: 2
Primary
JASS 05 Synchronization and Replication in the Context of Mobile Applications
>
Partition weight = 1
Inactive
partition
Weight: 1
Secondary 1
© Alexander Stage 49