User:Az990tony/backup
Jump to navigation
Jump to search
Overview
In my test environment, I have two machines.
+------------+ +------------+ +-------------------+ | Internet |--------- | XF (squid) |--------| XS (schoolserver) | | Modem | | | | LAMP+Moodle | +------------+ +------------+ +-------------------+
In this backup scheme:
- XF will backup its own files and files from XS
- XS will backup its own files and files from XF
This backup scheme is based on rsnapshot. See http://rsnapshot.org for details.
Under /library/.snapshots/ there will be the following directories
- hourly.0 # most recent backup
- hourly.1 # previous backup
- hourly.2 # backup before that
- daily.0 # yesterday's hourly.0
- daily.1 # hourly.0 of the day before that
- daily.2
- daily.3
- daily.4
- weekly.0 # hourly.0 of last Monday's backup
- weekly.1 # hourly.0 of previous Monday
- weekly.2
- weekly.3
- monthly.0 # first backup of this month
- monthly.1 # first backup of last month
- monthly.2 # first backup of month before that
The backups use hard links, so if not much changes, the total amount of space is actually quite small.
XF (squid/filter/cache)
Here are the files and directories involved on XF
- /root/.ssh/xfkey # this is a key generated on XF with no passphrase
- /root/.ssh/xskey.pub # this is a key generated on XS with no passphrase
- /root/bin/remote-xs.sh # this script will invoke backup-xs.sh script on XS
- /root/bin/backup-xf.sh # XS remote-xf.sh will invoke this script
- /etc/rsnapshot.conf # rsnapshot performs rsync over ssh connection
- /library/.snapshots/ # all of the backups will be kept here
- /library/.snapshots/hourly.0/local-xf/ # backups of local XF taken here
- /library/.snapshots/hourly.0/remote-xs/ # backups of remote XS taken here over SSH
rsnapshot.conf
This is the /etc/rsnapshot.conf I used.
################################################# # rsnapshot.conf - rsnapshot configuration file # ################################################# # # # PLEASE BE AWARE OF THE FOLLOWING RULES: # # # # This file requires tabs between elements # # # # Directories require a trailing slash: # # right: /home/ # # wrong: /home # # # # updated by Tony Pearson 02/26/2008 xf # ################################################# ####################### # CONFIG FILE VERSION # ####################### config_version 1.2 ########################### # SNAPSHOT ROOT DIRECTORY # ########################### # All snapshots will be stored under this root directory. # # /library is LVM with most space # snapshot_root /library/.snapshots/ # If no_create_root is enabled, rsnapshot will not automatically create the # snapshot_root directory. This is particularly useful if you are backing # up to removable media, such as a FireWire or USB drive. # #no_create_root 1 ################################# # EXTERNAL PROGRAM DEPENDENCIES # ################################# # LINUX USERS: Be sure to uncomment "cmd_cp". This gives you extra features. # EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility. # # See the README file or the man page for more details. # cmd_cp /bin/cp # uncomment this to use the rm program instead of the built-in perl routine. # cmd_rm /bin/rm # rsync must be enabled for anything to work. This is the only command that # must be enabled. # cmd_rsync /usr/bin/rsync # Uncomment this to enable remote ssh backups over rsync. # cmd_ssh /usr/bin/ssh # Comment this out to disable syslog support. # cmd_logger /usr/bin/logger # Uncomment this to specify the path to "du" for disk usage checks. # If you have an older version of "du", you may also want to check the # "du_args" parameter below. # cmd_du /usr/bin/du # Uncomment this to specify the path to rsnapshot-diff. # # yum install rsnapshot put everything under /usr/bin # cmd_rsnapshot_diff /usr/bin/rsnapshot-diff # Specify the path to a script (and any optional arguments) to run right # before rsnapshot syncs files # #cmd_preexec /path/to/preexec/script # Specify the path to a script (and any optional arguments) to run right # after rsnapshot syncs files # #cmd_postexec /path/to/postexec/script ######################################### # BACKUP INTERVALS # # Must be unique and in ascending order # # i.e. hourly, daily, weekly, etc. # ######################################### interval hourly 3 interval daily 5 interval weekly 4 interval monthly 3 ############################################ # GLOBAL OPTIONS # # All are optional, with sensible defaults # ############################################ # Verbose level, 1 through 5. # 1 Quiet Print fatal errors only # 2 Default Print errors and warnings only # 3 Verbose Show equivalent shell commands being executed # 4 Extra Verbose Show extra verbose information # 5 Debug mode Everything # verbose 2 # Same as "verbose" above, but controls the amount of data sent to the # logfile, if one is being used. The default is 3. # loglevel 3 # If you enable this, data will be written to the file you specify. The # amount of data written is controlled by the "loglevel" parameter. # logfile /var/log/rsnapshot # If enabled, rsnapshot will write a lockfile to prevent two instances # from running simultaneously (and messing up the snapshot_root). # If you enable this, make sure the lockfile directory is not world # writable. Otherwise anyone can prevent the program from running. # lockfile /var/run/rsnapshot.pid # Default rsync args. All rsync commands have at least these options set. # # -a archive mode # -r recursive # -x do not span LVM boundaries # --delete delete files that don't exist on source # --relative relative directory names # --numeric-ids don't map uid/gid from user/group names # rsync_short_args -arx rsync_long_args --delete --numeric-ids --relative # ssh has no args passed by default, but you can specify some here. # # xfkey must have been generated on this system with no passphrase ssh_args -i /root/.ssh/xfkey # Default arguments for the "du" program (for disk space reporting). # The GNU version of "du" is preferred. See the man page for more details. # If your version of "du" doesn't support the -h flag, try -k flag instead. # #du_args -csh # If this is enabled, rsync won't span filesystem partitions within a # backup point. This essentially passes the -x option to rsync. # The default is 0 (off). # #one_fs 0 # The include and exclude parameters, if enabled, simply get passed directly # to rsync. If you have multiple include/exclude patterns, put each one on a # separate line. Please look up the --include and --exclude options in the # rsync man page for more details on how to specify file name patterns. # #include ??? #include ??? #exclude ??? #exclude ??? # The include_file and exclude_file parameters, if enabled, simply get # passed directly to rsync. Please look up the --include-from and # --exclude-from options in the rsync man page for more details. # #include_file /path/to/include/file #exclude_file /path/to/exclude/file # If your version of rsync supports --link-dest, consider enable this. # This is the best way to support special files (FIFOs, etc) cross-platform. # The default is 0 (off). # link_dest 1 # When sync_first is enabled, it changes the default behaviour of rsnapshot. # Normally, when rsnapshot is called with its lowest interval # (i.e.: "rsnapshot hourly"), it will sync files AND rotate the lowest # intervals. With sync_first enabled, "rsnapshot sync" handles the file sync, # and all interval calls simply rotate files. See the man page for more # details. The default is 0 (off). # #sync_first 0 # If enabled, rsnapshot will move the oldest directory for each interval # to [interval_name].delete, then it will remove the lockfile and delete # that directory just before it exits. The default is 0 (off). # #use_lazy_deletes 0 ############################### ### BACKUP POINTS / SCRIPTS ### ############################### # LOCALHOST backup_script /root/bin/backup-xf.sh unused0 backup /home/ local-xf/ backup /root/ local-xf/ backup /etc/ local-xf/ #backup /usr/ local-xf/ backup /var/log/ local-xf/ # SCHOOLSERVER backup_script /bin/date "+ backup of schoolserver started at %c" unused1 backup_script /root/bin/remote-xs.sh unused2 backup root@schoolserver:/home/ remote-xs/ backup root@schoolserver:/root/ remote-xs/ backup root@schoolserver:/etc/ remote-xs/ backup root@schoolserver:/var/log remote-xs/ backup_script /bin/date "+ backup of schoolserver ended at %c" unused3
Here is my remote-xs.sh script (MIT license)
#!/bin/sh # # Copyright (c) 2008 Tony Pearson. # # Licensed under the MIT license for contribution to the # One Laptop per Child (OLPC) foundation. # # Permission is hereby granted, free of charge, to any person # obtaining a copy of this software and associated documentation # files (the "Software"), to deal in the Software without # restriction, including without limitation the rights to use, # copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the # Software is furnished to do so, subject to the following # conditions: # # The above copyright notice and this permission notice shall be # included in all copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES # OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT # HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, # WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR # OTHER DEALINGS IN THE SOFTWARE. # # For more information see the Open Source Initiative: # http://www.opensource.org/licenses/mit-license.php # KEY="/root/.ssh/xfkey" DEST="root@schoolserver" ssh -i $KEY $DEST "/root/bin/backup-xs.sh"