Rsync on HP-UX
Brief Presentation By Unix/Linux Apprentice
with 26 Years of Experience
Dusan Baljevic
Sydney, Australia
2011
Why This Document?
•
As a classical electronics/telecommunications engineer, I
believe in proper planning process – measure three times
before cutting:
http://www.circlingcycle.com.au/dusan.html
•
It is based on my 26-year practical experiences in
Unix/Linux.
•
My way of Operations Acceptance Testing and other tools:
http://www.circlingcycle.com.au/Unix-sources/
August 18, 2011
Rsync on HP-UX Webinar
2
Rsync History
•
rsync is an open source utility that provides fast
incremental file transfer.
•
rsync is freely available under the GNU General Public
License.
•
It has been available for more than 10 years now.
•
Available for Unix, Linux, and Windows.
•
Official site: http://rsync.samba.org/
August 18, 2011
Rsync on HP-UX Webinar
3
Rsync Features…
•
File based.
•
Incremental writes.
•
File delta.
•
Full restore.
•
Can use RSH, SSH or direct sockets as the transport.
•
Transmission security via SSH.
•
Internal pipelining reduces latency for multiple files.
•
File security via Encrypted File System (EvFS).
•
Cannot handle open-files (skips transferring them).
•
Cannot handle raw volumes.
•
Does not detect file renames and conflicts.
•
Scheduling done through O/S tools (cron, at, batch).
August 18, 2011
Rsync on HP-UX Webinar
4
Rsync Alternatives and Add-ons…
•
There are over 90 projects that deal with rsync in some
way:
http://unix.freshmeat.net/search/?Go.x=1&Go.y=1&q=rsync&section=projects
•
Other tools for file/directory synchronization:
http://en.wikipedia.org/wiki/Comparison_of_file_synchronization_software
August 18, 2011
Rsync on HP-UX Webinar
5
Rsync Algorithm
•
The rsync utility uses an algorithm (invented by Australian computer
programmer Andrew Tridgell) for efficiently transmitting a structure (such
as a file) across a communications link when the receiving computer
already has a different version of the same structure.
•
The recipient splits its copy of the file into fixed-size non-overlapping
chunks, of size S, and computes two checksums for each chunk: the
MD4 hash, and a weaker “rolling checksum”. It sends these checksums
to the sender.
•
The sender then compares its rolling checksums with the set sent by the
recipient to determine if any matches exist. If they do, it verifies the match
by computing the MD4/MD5* checksum for the matching block and by
comparing it with the MD4/MD5 checksum sent by the recipient.
•
The sender then sends the recipient those parts of its file that didn't
match any of the recipient's blocks, along with assembly instructions on
how to merge these blocks into the recipient's version to create a file
identical to the sender's copy.
•
If the sender's and recipient's versions of the file have many sections in
common, the utility needs to transfer relatively little data to synchronize
the files.
August 18, 2011
Rsync on HP-UX Webinar
6
Current Rsync Checksum Weaknesses – Part 1*
•
As mentioned on the previous slide, the recipient splits its copy of the file
into fixed-size non-overlapping chunks and computes two checksums for
each chunk: the MD4 hash, and a weaker 'rolling checksum' (version 30
of the protocol, released with rsync version 3.0.0, now uses MD5 hashes
rather than MD4). It sends these checksums to the sender.
•
The sender computes the rolling checksum for every chunk of size S in
its own version of the file, even overlapping chunks. This can be
calculated efficiently because of a special property of the rolling
checksum: if the rolling checksum of bytes n through n + S − 1 is R, the
rolling checksum of bytes n + 1 through n + S can be computed from R,
byte n, and byte n + S without having to examine the intervening bytes.
Thus, if one had already calculated the rolling checksum of bytes 1–25,
one could calculate the rolling checksum of bytes 2–26 solely from the
previous checksum, and from bytes 1 and 26.
•
The rolling checksum used in rsync is based on Mark Adler's adler-32
checksum, which is used in zlib, and is itself based on Fletcher's
checksum.
August 18, 2011
Rsync on HP-UX Webinar
7
Current Rsync Checksum Weaknesses – Part
2
•
The sender then compares its rolling checksums with the set sent by the
recipient to determine if any matches exist. If they do, it verifies the match
by computing the hash for the matching block and by comparing it with
the hash for that block sent by the recipient.
•
The sender then sends the recipient those parts of its file that did not
match the recipient's blocks, along with information on where to merge
these blocks into the recipient's version. This makes the copies identical.
However, there is a small probability that differences between chunks in
the sender and recipient are not detected, and thus remains uncorrected.
This requires a simultaneous hash collision in MD5 and the rolling
checksum. It is possible to generate MD5 collisions, and the rolling
checksum is not cryptographically strong, but the chance for this to occur
by accident is nevertheless extremely remote. With 128 bits from MD5
plus 32 bits from the rolling checksum, and assuming maximum entropy
in these bits, the probability of a hash collision with this combined
checksum is 2exp(−(128+32)) = 2exp(−160). The actual probability is a
few times higher, since good checksums approach maximum output
entropy but very rarely achieve it.
August 18, 2011
Rsync on HP-UX Webinar
8
Current Rsync Checksum Weaknesses – Part
3
•
If the sender's and recipient's versions of the file have many
sections in common, the utility needs to transfer relatively
little data to synchronize the files.
•
The MD5 Message-Digest Algorithm is a widely used
cryptographic hash function that produces a 128-bit (16-byte)
hash value. Specified in RFC 1321, MD5 has been employed
in a wide variety of security applications, and is also
commonly used to check data integrity. However, it has been
shown that MD5 is not collision resistant;[3] as such, MD5 is
not suitable for applications like SSL certificates or digital
signatures that rely on this property. An MD5 hash is typically
expressed as a 32-digit hexadecimal number.
August 18, 2011
Rsync on HP-UX Webinar
9
Current Rsync Checksum Weaknesses – Part
4
•
MD5 was designed by Ron Rivest in 1991 to replace an earlier hash
function, MD4. In 1996, a flaw was found with the design of MD5. While it
was not a clearly fatal weakness, cryptographers began recommending
the use of other algorithms, such as SHA-1 (which has since been found
also to be vulnerable). In 2004, more serious flaws were discovered,
making further use of the algorithm for security purposes questionable;
specifically, a group of researchers described how to create a pair of files
that share the same MD5 checksum. Further advances were made in
breaking MD5 in 2005, 2006, and 2007. In an attack on MD5 published in
December 2008, a group of researchers used this technique to fake SSL
certificate validity.
•
•
US-CERT says MD5 "should be considered cryptographically broken and
unsuitable for further use“.
•
Most U.S. government applications now require the SHA-2 family of hash
functions.
August 18, 2011
Rsync on HP-UX Webinar
10
Rsync – Memory Utilization
•
At one time, we used rsync version 2.x at a large private
hospital group in Australia (53 hospitals). It had one major
issue when we used it extensively: huge directory trees
were taking lot of RAM.
•
However, if you use rsync version 3.x, it does not keep the
whole tree in memory if it is big – it uses an incrementalrecursion algorithm.
August 18, 2011
Rsync on HP-UX Webinar
11
Rsync on HP-UX
•
For HP-UX 11iv2 and 11iv3, it is delivered on HP-UX Internet Express *
media, or can be obtained as free download at:
http://www.software.hp.com/portal/swdepot/displayProductInfo.do?productNumber=HPUXIEXP1131
•
For all currently supported HP-UX releases (11iv1 to 11iv3), there is
also The Porting and Archiving Centre for HP-UX:
http://hpux.connect.org.uk/hppd/hpux/Networking/Admin/rsync-3.0.8/
Make sure when installing depots from this site to also install ALL runtime dependencies (next slide).
•
Compile from sources.
August 18, 2011
Rsync on HP-UX Webinar
12
Rsync at The Porting and Archiving Centre
for HP-UX
Description: Rsync uses an algorithm which provides a very fast method for bringing
remote files into sync. It does this by sending just the differences in the files,
optionally
with compression, across the link, without requiring that both sets of files are present
at one of the ends of the link beforehand.
Author: Andrew Tridgell, Paul Mackerras tridge@samba.org
Home URL: http://rsync.samba.org/
License: GNU General Public License v3
Installation Tree: /usr/local
Languages used: C
Build-time dependencies: gettext libiconv make popt
Run-time dependencies: gettext libiconv popt
Documentation: Installation README Man Page
August 18, 2011
Rsync on HP-UX Webinar
13
Rsync Typical Scenario – Part 1
•
Day 1 – the first backup. No data at the remote location, so
a complete file transfer is required:
Local File
4GB
Remote File
Data transferred:
Rsync ~2GB (2:1 compression)
FTP/rcp/scp ~ 4GB
August 18, 2011
Rsync on HP-UX Webinar
14
Rsync Typical Scenario – Part 2
•
Day 2 – 0.5GB is added at the start of the file and the rest of
the file is left intact (red blocks of data) *:
Local File
4.5GB
Remote File
Data transferred:
Rsync ~0.25GB (2:1 compression)
FTP/rcp/scp ~ 4.5GB
August 18, 2011
Rsync on HP-UX Webinar
15
Rsync Typical Scenario – Part 3
•
Day 3 – The green (0.4GB) and yellow (0.2GB) blocks of
data are moved around (total of 0.6GB – the file contents are
the same, just block moved around):
Local File
4.5GB
Remote File
Data transferred:
Rsync ~0 (a small overhead)
FTP/rcp/scp ~ 4.5GB
August 18, 2011
Rsync on HP-UX Webinar
16
Rsync Command-line Options
# /usr/local/bin/rsync -v
August 18, 2011
Rsync on HP-UX Webinar
17
Rsync from Internet Express
It is important to pass the full path of the rsync command if it is not in the
PATH:
srvA# /opt/iexpress/rsync/bin/rsync -az -H -v --stats \
--rsync-path=/opt/iexpress/rsync/bin/rsync \
testfile.gz userB@srvB:/somedir
August 18, 2011
Rsync on HP-UX Webinar
18
Rsync from Porting and Archiving for HPUX
srvA# /usr/local/bin/rsync -a -r -v -t -z --stats \
--progress --rsh=/usr/bin/ssh \
--rsync-path=/usr/local/bin/rsync /labs srvB:
August 18, 2011
Rsync on HP-UX Webinar
19
Rsync Example when Previous Run
Interrupted or Failed
•
The flag “–partial” keeps the partially downloaded files on the
target.
•
This option is useful if the data transfer process gets interrupted by
some error of malfunction.
•
For example, if the rsync command terminated before all the data was
transported we could launch the same command again:
srvA# rsync -P myproject.gz userB@srvB:/somedir
rsync would use partial check sums to test the validity of that part of the
file that was already transported previously and start the actual data
transport only for the missing parts of the myproject.gz file.
August 18, 2011
Rsync on HP-UX Webinar
20
Rsync and SSH Key Exchange
userA@srvA# ssh-keygen -t rsa
Do not enter a passphrase, or use ssh-agent is passphrase is important.
userA@srvA# ssh userB@serverB mkdir -p .ssh
userA@srvA# cat .ssh/id_rsa.pub | ssh userB@srvB \
cat >> .ssh/authorized_keys
userA@srvA# ssh userB@srvB chmod 0700 .ssh/
userA@srvA# ssh userB@srvB chmod 0600 .ssh/authorized_keys
userA@srvA# rsync -a -r -v -t -z --stats --progress \
-e ssh /sourcedir/ userB@srvB:/targetdir/
August 18, 2011
Rsync on HP-UX Webinar
21
Rsync – SSH Chains and Different Port
•
rsync files between hosts that can not talk directly – use ssh
chain):
# rsync
-av --rsh="ssh -TA userA@srvA ssh -TA -l userB" \
/mydir/ srvB:/somedir/
•
rsync on different port and forcing IP protocol:
# rsync --progress --partial --rsh="ssh -p 8322" \
--bwlimit=300 --ipv4 remuser@remsrv:~/myfile.tgz .
August 18, 2011
Rsync on HP-UX Webinar
22
Rsync Example with Full Copy
srvA# /usr/local/bin/rsync -a -r -v -t -z --stats \
--progress --rsh=/usr/bin/ssh \
--rsync-path=/usr/local/bin/rsync /labs srvB:
...
Number of files: 118
Number of files transferred: 109
Total file size: 483583369 bytes
Total transferred file size: 483583369 bytes
Literal data: 483583369 bytes
Matched data: 0 bytes
File list size: 2002
File list generation time: 0.016 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 393465005
Total bytes received: 2119
August 18, 2011
Rsync on HP-UX Webinar
23
Rsync Example with Sparse Files * – Part 1
srvA# prealloc sparsefile1 1024000
srvA# /usr/local/bin/rsync -a -A -r -v -t -z \
--stats --progress \
--r sync-path=/usr/local/bin/rsync /src srvB:
srvB# ll sparsefile1
2000 -rw-r----sparsefile3
August 18, 2011
1 root sys
1024000 Aug 18 14:50
Rsync on HP-UX Webinar
24
Rsync Example with Sparse Files * – Part 2
srvA# prealloc sparsefile2 1024000
srvA# /usr/local/bin/rsync -a -A -S -r -v -t -z \
--stats --progress
\
--rsync-path=/usr/local/bin/rsync /src srvB:
sending incremental file list
src/sparsefile2
1024000 100%
24.88MB/s 0:00:00 (xfer#1, to-check=1/8)
Total transferred file size: 1024000 bytes
Literal data: 1024000 bytes
...
Total bytes sent: 1329
Total bytes received: 35
sent 1329 bytes
received 35 bytes
total size is 28825484
August 18, 2011
2728.00 bytes/sec
speedup is 21133.05
Rsync on HP-UX Webinar
25
Rsync Example with Sparse Files – Part 3
srvA# ll sparsefile*
2000 -rw-r-----
1 root
sys
1024000 Aug 18 14:45 sparsefile1
2000 -rw-r-----
1 root
sys
1024000 Aug 18 14:50 sparsefile2
srvB# ll sparsefile*
2000 -rw-r-----
1 root
sys
1024000 Aug 18 14:45 sparsefile1
0 -rw-r-----
1 root
sys
1024000 Aug 18 14:50 sparsefile2
August 18, 2011
Rsync on HP-UX Webinar
26
Rsync Example with ACLs * – Part 1
srvA# getacl vxdump
# file: vxdump
# owner: root
# group: sys
user::r-x
group::r-x
class:r-x
other:--srvA# setacl -m u:dusan:rx vxdump
srvA# pwget -n dusan
dusan:*:111:1::/home/dusan:/usr/bin/sh
(User “dusan” has UID 111)
# getacl vxdump
# file: vxdump
# owner: root
# group: sys
user::r-x
user:dusan:r-x
group::r-x
class:r-x
other:--August 18, 2011
Rsync on HP-UX Webinar
27
Rsync Example with ACLs * – Part 2
srvA# /usr/local/bin/rsync -a -r -v -t -z --stats \
--progress --rsync-path=/usr/local/bin/rsync \
/src srvB:
Check the file on the target srvB (ACL is missing):
srvB# getacl /src/vxdump
# file: /src/vxdump
# owner: root
# group: sys
user::r-x
group::r-x
class:r-x
other:--August 18, 2011
Rsync on HP-UX Webinar
28
Rsync Example with ACLs * – Part 3
srvA# /usr/local/bin/rsync -a –A -r -v -t -z --stats \
--progress --rsync-path=/usr/local/bin/rsync \
/src srvB:
Check the file on the target srvB (note that the ACL is owned by user “111” because
“dusan” is obviously not in the password database, but the ACL is copied!).
srvB# getacl /src/vxdump
# file: /src/vxdump
# owner: root
# group: sys
user::r-x
user:111:r-x
group::r-x
class:r-x
other:--August 18, 2011
Rsync on HP-UX Webinar
29
Rsync with Dry Run
•
When doing something other than non-trivial copies or using
features of rsync that you have never used before, add the
“-n” or “--dry-run” switch to make it a dry run.
# rsync -avhn /mydir /otherdir/
# rsync -nbrvvhn --del /mydir /otherdir/
# rsync -rn --size-only --exclude=*.iso \
/mydir/ /otherdir/
August 18, 2011
Rsync on HP-UX Webinar
30
Rsync Anonymous Server from Porting and
Archiving for HP-UX – Part 1
•
Install the depot.
•
Add into /etc/group
nobody::-2:
•
Add into /etc/services
rsync
•
873/tcp
Add into /etc/inetd.conf
rsync stream tcp nowait root /usr/local/bin/rsync rsyncd –daemon
... And reload the daemon (inetd –c)
August 18, 2011
Rsync on HP-UX Webinar
31
Rsync Anonymous Server from Porting and
Archiving for HP-UX – Part 2
•
Create /etc/rsyncd.conf
uid = nobody
gid = nobody
use chroot = yes
read only=yes
max connections = 5
log file = /var/adm/syslog/rsyncd.log
[ftp] path = /src
comment = HP-UX Source export area
•
Check Bastille or disable it.
August 18, 2011
Rsync on HP-UX Webinar
32
Rsync Anonymous Server from Porting and
Archiving for HP-UX – Part 3
To check what is available on the rsync server:
# /usr/local/bin/rsync -avlH rsync://myhost
src
August 18, 2011
HP-UX Source export area
Rsync on HP-UX Webinar
33
Mirror By Rsync
If a remote server runs anonymous rsync server, mirroring is
achieved in the following manner:
# /usr/local/bin/rsync
-avvlH --rsync-path=/usr/local/bin/rsync \
myhost:/src/ /src
opening connection using: ssh myhost /usr/local/bin/rsync --server --sender vvlHogDtpre.isf . /src/
Password:
receiving incremental file list
created directory /src
delta-transmission enabled
rsync-3.0.8-ia64-11.31.depot
sparsefile2
sparsefile3
total: matches=0
hash_hits=0
•
sent 166 bytes
•
total size is 29849484
August 18, 2011
false_alarms=0 data=29849484
received 29853794 bytes
5427992.73 bytes/sec
speedup is 1.00
Rsync on HP-UX Webinar
34
Rsync and LVM Snapshots – Part 1
•
There is no official way to copy data from a snapshot back to the original
volume. Here is how I normally accomplish this task, although it
depends on what kind of files I am trying to copy, any special file
attributes, and the amount of data
•
If the amount of data is not massive, and there are not many files in the
file system), use rsync to re-synchronize data between a snapshot and
the original. Be very careful about the command line, because if you use
the wrong order of the volumes, the copy of the data goes in the wrong
direction!
# rsync -aHv /snap/ /orig
Rsync is known to be VERY slow for big file transfers and when there
are lot of files.
August 18, 2011
Rsync on HP-UX Webinar
35
Rsync and LVM Snapshots – Part 2
•
Since file systems are VxFS, vxdump(1M) and vxrestore(1M) are very
convenient. Much faster:
# fsck /snap
# vxdump 0f - /snap | (cd /orig && vxrestore xf -)
•
Use native HP-UX tools, like tar(1M), pax(1M), fbackup(1M), and
cpio(1M). Here are some simple examples:
# cd /snap && tar cpf - * | (cd /orig && tar xvpf -)
# cd /snap && fbackup -i . -f - | ( cd /orig && frecover -Xsrf - )
# cd /snap && pax -w . | ( cd /orig && pax -r -pe )
August 18, 2011
Rsync on HP-UX Webinar
36
Rsync and Expanded Remote File List
Pass the listing of remote command as list of files to rsync.
The following line will run the find command on the remote
machine in the /remdir directory and rsync all “*.conf"
files it finds to the local machine in the /somedir directory.
# rsync -avR remsrv:'`find /remdir -name "*.[conf]"`‘ \
/somedir/
August 18, 2011
Rsync on HP-UX Webinar
37
Rsync and Relative Directory
•
If you want the target directory structure to be relative
(chroot in a way) you can add flag"-R".
•
The directory structure /mydir/data would then look like
/BACKUP/mydir/data/ as the sync path name starts
from / on the source machine:
# rsync -Ravx --timeout=30 --delete-excluded \
user@remsrv:/mydir/data/ /BACKUP/
August 18, 2011
Rsync on HP-UX Webinar
38
Rsync and “Snapshots”
•
Use “--link-dest” option to create space-efficient
snapshot-based backups. It appears to have multiple
complete copies of the backed up data (one for each
backup run) but files that do not change between runs are
hard linked instead of creating new copies, thus saving
space.
•
The major disadvantage of this technique is that if a file is
corrupted due to disk error it is just as corrupt in all
snapshots that link to that file. Having offline backups would
protect against this possibility.
August 18, 2011
Rsync on HP-UX Webinar
39
Rsync - File Size and Bandwidth Limits
If you need to update a site over a slow link, run two passes
of rsync.
•
Transfer the small files firstly:
# rsync -a --max-size=150K /mydir/ remsrv:/somedir/
•
Then do the same for the large files and limit bandwidth to
100 KBytes per second:
# rsync -a --min-size=150K --bwlimit=100 \
/mydir/ remsrv:/somedir/
August 18, 2011
Rsync on HP-UX Webinar
40
Rsync – Exclude Files
An example how to exclude files and directories from rsync
transfer.
# rsync -azhve ssh --stats --progress \
--exclude-from '/somedir/EXCLUDE.txt' \
--delete-excluded /mydir remsrv:/somedir/
August 18, 2011
Rsync on HP-UX Webinar
41
Rsync – Avoid Checksum For Large Files
Before Transfer
•
It is a common error to use option “-c” when huge files are
transferred. rsync is going to have to read/checksum the
entire file, and reading them is going to take a long time,
unless the file is stored on SSDs or some very fast storage.
•
Instead, try:
# rsync -vhz --partial --inplace …
“-c” means that it checksums the entire file BEFORE doing
any transfers, rather than using the timestamp to see if it
has changed, which means reading the whole file twice.
August 18, 2011
Rsync on HP-UX Webinar
42
Rsync and “Clean Shell”
•
The "is your shell clean" message and the "protocol mismatch"
message are usually caused by having some program or command
running in Shell’s profiles (.cshrc, .profile, .kshrc, .bashrc or equivalent)
every time when using a remote-shell program (such as ssh or rsh).
Data written in this way corrupts the rsync data stream. rsync detects
this at startup and produces those error messages. However, if you are
using rsync-daemon syntax (host::path or rsync://) without
using a remote-shell program (no --rsh or -e option), there is not
remote-shell program involved, and the problem is probably caused by
an error on the daemon side (so check the daemon logs). To test it:
# ssh user@myhost.domain.dom /bin/true
The above command should not output anything at all (except an ssh
password prompt, if applicable). If there is any output on stdout, disable
whatever is generating that output so that rsync does not get “garbage”
trying to talk to the remote rsync.
August 18, 2011
Rsync on HP-UX Webinar
43
Rsync Common Problems
Several common causes for a remote rsync process going away:
•
The destination disk is full (at least the size of the largest file that
needs to be updated available must be in free disk space for the
transfer to succeed).
•
An idle connection caused a router or remote-shell server to
close the connection.
•
A network error caused the connection to be dropped.
•
The remote rsync executable was not found.
•
Remote Shell setup is not working right or is not "clean" (it is
sending spurious text to rsync).
•
If the problem might be an idle connection getting closed, use a
“--timeout” option (newer rsync versions send keep-alive
messages during periods of no activity).
August 18, 2011
Rsync on HP-UX Webinar
44
Rsync Performance Analysis and Tuning
There is no silver bullet!
Test everything and rsync will certainly bring
benefits.
August 18, 2011
Rsync on HP-UX Webinar
45
Thank You!
Dusan Baljevic
Sydney, Australia
2011
Appendix
Dusan Baljevic
Sydney, Australia
2011
Rsync Command-line Options – Part 1
Usage: rsync [OPTION]... SRC [SRC]... DEST
or rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST
or rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST
or rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST
or rsync [OPTION]... [USER@]HOST:SRC [DEST]
or rsync [OPTION]... [USER@]HOST::SRC [DEST]
or rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]
The ':' usages connect via remote shell, while '::' & 'rsync://' usages connect
to an rsync daemon, and require SRC or DEST to start with a module name.
August 18, 2011
Appendix- Rsync on HP-UX Webinar
48
Rsync Command-line Options – Part 2
Options
-v, --verbose
increase verbosity
-q, --quiet
suppress non-error messages
--no-motd
suppress daemon-mode MOTD (see manpage caveat)
-c, --checksum
skip based on checksum, not mod-time & size
-a, --archive
archive mode; equals -rlptgoD (no -H,-A,-X)
--no-OPTION
turn off an implied OPTION (e.g. --no-D)
-r, --recursive
recurse into directories
-R, --relative
use relative path names
--no-implied-dirs
-b, --backup
don't send implied dirs with --relative
make backups (see --suffix & --backup-dir)
--backup-dir=DIR
make backups into hierarchy based in DIR
--suffix=SUFFIX
set backup suffix (default ~ w/o --backup-dir)
-u, --update
skip files that are newer on the receiver
--inplace
update destination files in-place (SEE MAN PAGE)
--append
append data onto shorter files
--append-verify
like --append, but with old data in file checksum
August 18, 2011
Appendix- Rsync on HP-UX Webinar
49
Rsync Command-line Options – Part 3
Options
-d, --dirs
transfer directories without recursing
-l, --links
copy symlinks as symlinks
-L, --copy-links
transform symlink into referent file/dir
--copy-unsafe-links
only "unsafe" symlinks are transformed
--safe-links
ignore symlinks that point outside the source tree
-k, --copy-dirlinks
transform symlink to a dir into referent dir
-K, --keep-dirlinks
treat symlinked dir on receiver as dir
-H, --hard-links
preserve hard links
-p, --perms
preserve permissions
-E, --executability
preserve the file's executability
--chmod=CHMOD
affect file and/or directory permissions
-A, --acls
preserve ACLs (implies --perms)
-o, --owner
preserve owner (super-user only)
-g, --group
preserve group
--devices
preserve device files (super-user only)
--specials
preserve special files
August 18, 2011
Appendix- Rsync on HP-UX Webinar
50
Rsync Command-line Options – Part 4
Options
-D
same as --devices --specials
-t, --times
preserve modification times
-O, --omit-dir-times
omit directories from --times
--super
receiver attempts super-user activities
-S, --sparse
handle sparse files efficiently
-n, --dry-run
perform a trial run with no changes made
-W, --whole-file
copy files whole (without delta-xfer algorithm)
-x, --one-file-system
don't cross filesystem boundaries
-B, --block-size=SIZE
force a fixed checksum block-size
-e, --rsh=COMMAND
specify the remote shell to use
--rsync-path=PROGRAM
specify the rsync to run on the remote machine
--existing
skip creating new files on receiver
--ignore-existing
skip updating files that already exist on receiver
--remove-source-files
sender removes synchronized files (non-dirs)
--del
an alias for --delete-during
--delete
delete extraneous files from destination dirs
--delete-before
receiver deletes before transfer, not during
--delete-during
receiver deletes during transfer (default)
--delete-delay
find deletions during, delete after
--delete-after
receiver deletes after transfer, not during
--delete-excluded
also delete excluded files from destination dirs
--ignore-errors
delete even if there are I/O errors
--force
force deletion of directories even if not empty
--max-delete=NUM
don't delete more than NUM files
--max-size=SIZE
don't transfer any file larger than SIZE
--min-size=SIZE
don't transfer any file smaller than SIZE
--partial
keep partially transferred files
--partial-dir=DIR
put a partially transferred file into DIR
--delay-updates
put all updated files into place at transfer's end
August 18, 2011
Appendix- Rsync on HP-UX Webinar
51
Rsync Command-line Options – Part 5
Options
-m, --prune-empty-dirs
prune empty directory chains from the file-list
--numeric-ids
don't map uid/gid values by user/group name
--timeout=SECONDS
set I/O timeout in seconds
--contimeout=SECONDS
set daemon connection timeout in seconds
-I, --ignore-times
don't skip files that match in size and mod-time
--size-only
skip files that match in size
--modify-window=NUM
compare mod-times with reduced accuracy
-T, --temp-dir=DIR
create temporary files in directory DIR
-y, --fuzzy
find similar file for basis if no dest file
--compare-dest=DIR
also compare destination files relative to DIR
--copy-dest=DIR
... and include copies of unchanged files
--link-dest=DIR
hardlink to files in DIR when unchanged
-z, --compress
compress file data during the transfer
--compress-level=NUM
explicitly set compression level
--skip-compress=LIST
skip compressing files with a suffix in LIST
-C, --cvs-exclude
August 18, 2011
auto-ignore files the same way CVS does
Appendix- Rsync on HP-UX Webinar
52
Rsync Command-line Options – Part 6
Options
-f, --filter=RULE
add a file-filtering RULE
-F
same as --filter='dir-merge /.rsync-filter'
repeated: --filter='- .rsync-filter'
--exclude=PATTERN
exclude files matching PATTERN
--exclude-from=FILE
read exclude patterns from FILE
--include=PATTERN
don't exclude files matching PATTERN
--include-from=FILE
read include patterns from FILE
--files-from=FILE
read list of source-file names from FILE
-0, --from0
all *-from/filter files are delimited by 0s
-s, --protect-args
no space-splitting; only wildcard special-chars
--address=ADDRESS
bind address for outgoing socket to daemon
--port=PORT
specify double-colon alternate port number
--sockopts=OPTIONS
specify custom TCP options
--blocking-io
use blocking I/O for the remote shell
--stats
give some file-transfer stats
-8, --8-bit-output
leave high-bit chars unescaped in output
-h, --human-readable
output numbers in a human-readable format
--progress
August 18, 2011
show progress during transfer
Appendix- Rsync on HP-UX Webinar
53
Rsync Command-line Options – Part 7
Options
-P
same as --partial --progress
-i, --itemize-changes
output a change-summary for all updates
--out-format=FORMAT
output updates using the specified FORMAT
--log-file=FILE
log what we're doing to the specified FILE
--log-file-format=FMT
log updates using the specified FMT
--password-file=FILE
read daemon-access password from FILE
--list-only
list the files instead of copying them
--bwlimit=KBPS
limit I/O bandwidth; KBytes per second
--write-batch=FILE
write a batched update to FILE
--only-write-batch=FILE like --write-batch but w/o updating destination
--read-batch=FILE
read a batched update from FILE
--protocol=NUM
force an older protocol version to be used
--iconv=CONVERT_SPEC
request charset conversion of filenames
-4, --ipv4
prefer IPv4
-6, --ipv6
prefer IPv6
--version
(-h) --help
August 18, 2011
print version number
show this help (-h works with no other options)
Appendix- Rsync on HP-UX Webinar
54