

Store
Store home
Devices
o
o
o
o
o
o
o
Microsoft Surface
PCs & tablets
Xbox
Virtual reality
Accessories
Windows phone
Microsoft Band
Software
o
o
o
Office
Windows
Additional software
Apps
o
o
o
All apps
Windows apps
Windows phone apps
Games
o
o
o
o
o
Xbox One games
Xbox 360 games
PC games
Windows games
Windows phone games
Entertainment
o
o
o
All Entertainment
Movies & TV
Music
Business & Education
o
o
Business Store
Education Store
o
Developer
Sale
o
o
Back-to-school essentials
Sale
Find a store
Gift cards

Products
Software & services
o
o
o
o
o
o
o
o
o
o
o
o
o
Windows
Office
Free downloads & security
Internet Explorer
Microsoft Edge
Skype
OneNote
OneDrive
Microsoft Health
MSN
Bing
Microsoft Groove
Microsoft Movies & TV
Devices & Xbox
o
o
o
o
o
o
o
o
o
All Microsoft devices
Microsoft Surface
All Windows PCs & tablets
PC accessories
Xbox & games
Microsoft Band
Microsoft Lumia
All Windows phones
Microsoft HoloLens
For business
o
o
o
o
Cloud Platform
Microsoft Azure
Microsoft Dynamics
Windows for business
o
o
o
o
o
o
o
Office for business
Skype for business
Surface for business
Enterprise solutions
Small business solutions
Find a solutions provider
Volume Licensing
For developers & IT pros
o
o
o
o
o
Develop Windows apps
Microsoft Azure
MSDN
TechNet
Visual Studio
For students & educators
o
o
o
o

Office for students
OneNote in classroom
Shop PCs & tablets perfect for students
Microsoft in Education
Support
Sign in

Search Microsoft Search

Research Research
o Research Home
o Research areas
 Algorithms
 Artificial intelligence and machine learning
 Computer systems and networking
 Computer vision
 Data visualization, analytics, and platform
 Ecology and environment
 Economics
 Graphics and multimedia
 Hardware, devices, and quantum computing
 Human-centered computing
 Mathematics
 Medical, health, and genomics






o
o
o
o
o



Natural language processing and speech
Programming languages and software engineering
Search and information retrieval
Security, privacy, and cryptography
Social Sciences
Technology for emerging markets
Products & Downloads
Programs & Events
 Academic Programs
 Events & Conferences
People
Careers
About
 About
 Microsoft Research blog
 Asia Lab
 Cambridge Lab
 India Lab
 New England Lab
 New York City Lab
 Redmond Lab
 Applied Sciences Lab
Research areas
o Algorithms
o Artificial intelligence and machine learning
o Computer systems and networking
o Computer vision
o Data visualization, analytics, and platform
o Ecology and environment
o Economics
o Graphics and multimedia
o Hardware, devices, and quantum computing
o Human-centered computing
o Mathematics
o Medical, health, and genomics
o Natural language processing and speech
o Programming languages and software engineering
o Search and information retrieval
o Security, privacy, and cryptography
o Social Sciences
o Technology for emerging markets
Products & Downloads
Programs & Events
o Academic Programs
o Events & Conferences



People
Careers
About
o About
o Microsoft Research blog
o Asia Lab
o Cambridge Lab
o India Lab
o New England Lab
o New York City Lab
o Redmond Lab
o Applied Sciences Lab
Optimizing File Replication over LimitedBandwidth Networks using Remote
Differential Compression
November 1, 2006


Download PDF
BibTex
Authors





Dan Teodosiu
Nikolaj Bjorner
Yuri Gurevich
Mark Manasse
Joe Porkka
Publication Type
TechReport
Pages
16
Number
MSR-TR-2006-157


Abstract
Related Info
Abstract
Remote Differential Compression (RDC) protocols can efficiently update files over a limitedbandwidth network when two sites have roughly similar files; no site needs to know the content
of another’s files a priori. We present a heuristic approach to identify and transfer the file
differences that is based on finding similar files, subdividing the files into chunks, and
comparing chunk signatures. Our work significantly improves upon previous protocols such as
LBFS and RSYNC in three ways. Firstly, we present a novel algorithm to efficiently find the
client files that are the most similar to a given server file. Our algorithm requires 96 bits of metadata per file, independent of file size, and thus allows us to keep the metadata in memory and
eliminate the need for expensive disk seeks. Secondly, we show that RDC can be applied
recursively to signatures to reduce the transfer cost for large files. Thirdly, we describe new ways
to subdivide files into chunks that identify file differences more accurately. We have
implemented our approach in DFSR, a state-based multi-master file replication service shipping
as part of Windows Server 2003 R2. Our experimental results show that similarity detection
produces results comparable to LBFS while incurring a much smaller overhead for maintaining
the metadata. Recursive signature transfer further increases replication efficiency by up to
several orders of magnitude.
Related Info
Related Files
 tr-2006-157.doc
Research Areas
 Computer systems and networking
Research Labs
 Microsoft Research Lab - Redmond
Follow Microsoft Research



Follow @MSFTResearch
Share this page



Tweet
Learn






Windows
Office
Skype
Outlook
OneDrive
MSN
Devices






Microsoft Surface
Xbox
PC and laptops
Microsoft Lumia
Microsoft Band
Microsoft HoloLens
Microsoft Store





View account
Order tracking
Retail store locations
Returns
Sales & support
Downloads






Download Center
Windows downloads
Windows 10 Apps
Office Apps
Microsoft Lumia Apps
Internet Explorer
Values





Diversity and inclusion
Accessibility
Environment
Microsoft Philanthropies
Corporate Social Responsibility

Privacy at Microsoft
Company
​






Careers
About Microsoft
Company news
Investors
Research
Site map

English (United States)






Contact us
Privacy & cookies
Terms of use
Trademarks
About our ads
© 2016 Microsoft