Saturday, January 26, 2013

Fastest Remote Directory rsync over ssh CentOS /Fedora/RHEL/Ubuntu

rsync -aHAXxv --numeric-ids --delete --progress -e "ssh -T -c arcfour -o Compression=no -x" user@<source>:<source_dir> <dest_dir>

The fastest remote directory rsync over ssh archival

This creates an archive that does the following:

rsync:: (Everyone seems to like -z, but it is much slower for me)
-a: archive mode - rescursive, preserves owner, preserves permissions, preserves modification times, preserves group, copies symlinks as symlinks, preserves device files.
-H: preserves hard-links
-A: preserves ACLs
-X: preserves extended attributes
-x: don't cross file-system boundaries
-v: increase verbosity
--numeric-ds: don't map uid/gid values by user/group name
--delete: delete extraneous files from dest dirs (differential clean-up during sync)
--progress: show progress during transfer

-T: turn off pseudo-tty to decrease cpu load on destination.
-c arcfour: use the weakest but fastest SSH encryption. Must specify "Ciphers arcfour" in sshd_config on destination.
-o Compression=no: Turn off SSH compression.
-x: turn off X forwarding if it is on by default.
Flip: rsync -aHAXxv --numeric-ids --delete --progress -e "ssh -T -c arcfour -o Compression=no -x" [source_dir] [dest_host:/dest_dir]

Thursday, January 24, 2013

Add a new LUN to live Multipath environment CentOS/RHEL

  • Rescan SCSI bus:
    # /usr/bin/
  • Lookup the names of newly available disks
    # lsblk
  • Lookup WWID for each newly discovered disk:
    # scsi_id --page=0x83 --whitelisted --device=/dev/<disk>
  • Add those WWIDs to blacklist_exceptions and multipaths in /etc/multipath.conf.
  • Reload multipathd and check the disks are properly configured:
    # /etc/init.d/multipathd reload
    # lsblk