#rsync FAQ

Maintained by Kevin Korb (BasketCase)

This is a document that I have written in an attempt to answer some of the common questions that I have encountered in the #rsync support channel on irc.freenode.net. Some of these may be covered in the official FAQ and all of them are probably covered in the man page but people ask anyway so here we are.


Index

  1. How do I sync both directions?
  2. How do I rsync a database?
  3. Rsync seems to want to update file(s) even though they are all the same
  4. Why doesn't --delete delete anything?
  5. How do I connect to an sshd on a non-standard port number or use other ssh parameters?
  6. Do I need to setup the rsync service or rsyncd.conf file?
  7. How do I rsync on Windows?
  8. I can't get includes/excludes to work
  9. How do I get rsync to transfer a file as soon as it is modified?
  10. Do I need rsync installed on both systems?
  11. Why does rsync re-copy the entire file when most of it is already there?
  12. I am afraid of allowing root to ssh in to the system. What are my options?
  13. I ran rsync but the source and the target are not the same size?
  14. I am using --checksum and it is really slow. Sometimes so slow that it times out.
  15. Why can't I access the files I just copied with rsync on Windows 7 or Vista?
  16. What are some cool command line switches to read about and try with rsync?
  17. What are some alternatives to rsync?

  1. Q: How do I sync both directions?
  2. Q: How do I rsync a database?
  3. Q: Rsync seems to want to update file(s) even though they are all the same
  4. Q: Why doesn't --delete delete anything?
  5. Q: How do I connect to an sshd on a non-standard port number or use other ssh parameters?
  6. Q: Do I need to setup the rsync service or rsyncd.conf file?
  7. Q: How do I rsync on Windows?
    Note: I do not recommend any of these for backing up Windows itself.
  8. Q: I can't get includes/excludes to work
  9. Q: How do I get rsync to transfer a file as soon as it is modified?
  10. Q: Do I need rsync installed on both systems?
  11. Q: Why does rsync re-copy the entire file when most of it is already there?
  12. Q: I am afraid of allowing root to ssh in to the system. What are my options?
  13. Q: I ran rsync but the source and the target are not the same size?
    1. The first thing to consider would be excludes. If you excluded stuff then obviously the sizes will never be the same.
    2. If your target is slightly smaller than your source the likely cause is a difference in directory sizes. This is simply due to how directories allocate disk space and can't really be helped. I have devised a quick shell command to add up all of the file sizes in the current directory without including the directory sizes:
      echo `find . -type f -ls | awk '{print $7 "+"}'`0 | bc
      This can be used to confirm that the files themselves are the same sizes even if the directories are not.
    3. If your target is 1-10% larger than your source the most likely cause is hard links. Rsync by default or even with -a/--archive does not preserve hard links so if you rsync 2 hard links they will end up as duplicate files on the target taking up twice the disk space. If you want to preserve hard links add the -H/--hard-links option.
    4. If your target is >10% larger than your source the most likely cause is sparse files. Rsync by default does not write any files as sparse files even if they are on the source (it can't actually tell). If you have sparse files (most commonly used as virtual machine images and incomplete p2p downloads) then you will want to use the --sparse option. Note that this can turn things around and make the target smaller than the source as rsync with --sparse will not allocate disk space for any long string of null characters possibly making files on the target sparse when they were not on the source.
    5. There are also differences in filesystem types, block sizes, file slack overhead, etc. that can cause the outcome to be different.
    6. If you are comparing sizes using -h or --human-readable you should be aware that all of the GNU fileutilities use binary units (power of 2) while rsync uses decimal units (power of 1000). If you use --human-readable twice on rsync that difference will go away.
    7. If you have checked all of this and you are still bugged by an unexplained size difference then I would like to point out that simple sizes are not a very useful check for completeness or accuracy on a data copying operation. It doesn't check the contents of the files at all and it is subject to variations I explained above. I would suggest using an actual file verification utility such as cfv to verify your files using real cryptographic hashes. The cfv utility is very similar to the simple md5sum utility except that it is recursive, faster, and has a %completion bar.
  14. I am using --checksum and it is really slow. Sometimes so slow that it times out.
  15. Q: Why can't I access the files I just copied with rsync on Windows 7 or Vista?
  16. Q: What are some cool command line switches to read about and try with rsync?
  17. Q: What are some alternatives to rsync?