HOWTOlabs rsync
Configuration tips

Elsewhere [ edit ]

rsync allows transfering file sets efficiently between directories on the same or different hosts.   It does this primarily by identifying and transfering only only the portions of files that have changed. 

$ rsync -azvP -e "ssh -i public.pem" user@aws.server.com:/efs/archive /Volumes/ext_disk/backup/ 

  [ example providing a certificate key to authenticate ]

# cat /etc/xinetd.d/rsync

  # default: off
  # description: The rsync server is a good addition to an ftp server, as it \
  #       allows crc checksumming etc.
  service rsync
  {
  #       disable = yes
          socket_type     = stream
          wait            = no
          user            = root
          server          = /usr/bin/rsync
          server_args     = --daemon
          log_on_failure  += USERID
  }
For backups dumps using rsync deamon, will need to specify UID = root. Otherwise any files not readable by world will generate a insufficient privledges error (backup will continue, files with non-world readble privledges will be skipped). 
# cat /etc/rsyncd.conf

  max connections = 2
  log file = /var/log/rsync.log
  timeout = 300

  [backups]
          comment = Backup patrol
          path = /public/backups
          read only = yes
          list = yes
  #       uid = nobody
  #       gid = nobody
  hosts allow = 72.18.242.128/29 216.240.133.161/32


# rsync rsync://cp2.scalix.com/backups/

  drwxr-xr-x        4096 2006/08/10 02:06:10 .
  -rw-r--r--     4795076 2006/08/07 20:39:01 bugzilla.scalix.com-Mon.bugzilla.tar.gz
  -rw-r--r--   210649648 2006/08/07 20:39:41 bugzilla.scalix.com-Mon.mysql.tar.gz
  -rw-r--r--     1297534 2006/08/07 20:39:43 bugzilla.scalix.com-Mon.system.tar.gz
  -rw-r--r--     4795077 2006/08/10 02:05:30 bugzilla.scalix.com-Thu.bugzilla.tar.gz
  -rw-r--r--   210649466 2006/08/10 02:06:08 bugzilla.scalix.com-Thu.mysql.tar.gz
  -rw-r--r--     1299041 2006/08/10 02:06:10 bugzilla.scalix.com-Thu.system.tar.gz
  -rw-r--r--     4795075 2006/08/08 02:05:23 bugzilla.scalix.com-Tue.bugzilla.tar.gz
  -rw-r--r--   210649565 2006/08/08 02:05:58 bugzilla.scalix.com-Tue.mysql.tar.gz
  -rw-r--r--     1298355 2006/08/08 02:05:59 bugzilla.scalix.com-Tue.system.tar.gz
  -rw-r--r--     4795076 2006/08/09 02:05:28 bugzilla.scalix.com-Wed.bugzilla.tar.gz
  -rw-r--r--   210649724 2006/08/09 02:06:07 bugzilla.scalix.com-Wed.mysql.tar.gz
  -rw-r--r--     1298692 2006/08/09 02:06:09 bugzilla.scalix.com-Wed.system.tar.gz

# rsync rsync://cp2.scalix.com/

  backups         Backup patrol
The following script demonstrates some common options to use with rsync. -v lists filenames as they are processed. -r recurse directories on the source host and transfer files to destination host. Note: files and folders on destination host not on the source host file be left alone. --bwlimit=64 transfer rate (KBytes/sec) not to exceed. --delete files on source host that are no longer present will be removed from destination host (must be used with -r). --exclude=old files on source host to ignore. 
# cat rsync_foobar.sh

  SRC=remote.foobar.com/backups
  DST=/archive/foobar

  date

  # T1 link is 1544 kbps = 160 kBps
  # ync -v --progress --bwlimit=64 rsync://$SRC/* $DST
  rsync -v -r --bwlimit=64 --delete --exclude=old --exclude=RCS --exclude=active rsync://$SRC/* $DST

# sh rsync_foobar.sh

  Fri Aug 18 11:22:57 PDT 2006
  receiving file list ... done
  forumsback.sh
  rsync-web.sh
  wikiback.sh
  www-20060815-daily2.tar.gz
  www-20060816-daily3.log
  www-20060816-daily3.tar.gz
  www-20060817-daily4.log
  www-20060817-daily4.tar.gz
  www-back.sh
  www-ignore
 
  sent 488099 bytes  received 282712 bytes  4270.42 bytes/sec
  total size is 1423946362  speedup is 1847.34

Quick Reference

Elsewhere [ edit ] 
$ rsync -azvP -e "ssh -i public.pem" user@aws.server.com:/efs/archive /Volumes/ext_disk/backup/ 

  [ example providing a certificate key to authenticate ]