User:Az990tony/backup
Jump to navigation
Jump to search
Overview
In my test environment, I have two machines.
+------------+ +------------+ +-------------------+ | Internet |--------- | XF (squid) |--------| XS (schoolserver) | | Modem | | | | LAMP+Moodle | +------------+ +------------+ +-------------------+
In this backup scheme:
- XF will backup its own files and files from XS
- XS will backup its own files and files from XF
This backup scheme is based on rsnapshot. See http://rsnapshot.org for details.
Under /library/.snapshots/ there will be the following directories
- hourly.0 # most recent backup
- hourly.1 # previous backup
- hourly.2 # backup before that
- daily.0 # yesterday's hourly.0
- daily.1 # hourly.0 of the day before that
- daily.2
- daily.3
- daily.4
- weekly.0 # hourly.0 of last Monday's backup
- weekly.1 # hourly.0 of previous Monday
- weekly.2
- weekly.3
- monthly.0 # first backup of this month
- monthly.1 # first backup of last month
- monthly.2 # first backup of month before that
The backups use hard links, so if not much changes, the total amount of space is actually quite small.
To take a fresh backup, the following command can be used:
[root@schoolserver ~] rsnapshot hourly
This will delete hourly.2, move all hourly.1 files to hourly.2, move all files from hourly.0 to hourly.1 and then do an rsync to determine what has changed since the last backup.
Backups can be fully automated using a crontab schedule
XF (squid/filter/cache)
Here are the files and directories involved on XF
- /root/.ssh/xfkey # this is a key generated on XF with no passphrase
- /root/.ssh/xskey.pub # this is a key generated on XS with no passphrase
- /root/bin/remote-xs.sh # this script will invoke backup-xs.sh script on XS
- /root/bin/backup-xf.sh # XS remote-xf.sh will invoke this script
- /etc/rsnapshot.conf # rsnapshot performs rsync over ssh connection
- /library/.snapshots/ # all of the backups will be kept here
- /library/.snapshots/hourly.0/local-xf/ # backups of local XF taken here
- /library/.snapshots/hourly.0/remote-xs/ # backups of remote XS taken here over SSH
rsnapshot.conf
This is the /etc/rsnapshot.conf I used.
################################################# # rsnapshot.conf - rsnapshot configuration file # ################################################# # # # PLEASE BE AWARE OF THE FOLLOWING RULES: # # # # This file requires tabs between elements # # # # Directories require a trailing slash: # # right: /home/ # # wrong: /home # # # # updated by Tony Pearson 02/26/2008 xf # ################################################# ####################### # CONFIG FILE VERSION # ####################### config_version 1.2 ########################### # SNAPSHOT ROOT DIRECTORY # ########################### # All snapshots will be stored under this root directory. # # /library is LVM with most space # snapshot_root /library/.snapshots/ # If no_create_root is enabled, rsnapshot will not automatically create the # snapshot_root directory. This is particularly useful if you are backing # up to removable media, such as a FireWire or USB drive. # #no_create_root 1 ################################# # EXTERNAL PROGRAM DEPENDENCIES # ################################# # LINUX USERS: Be sure to uncomment "cmd_cp". This gives you extra features. # EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility. # # See the README file or the man page for more details. # cmd_cp /bin/cp # uncomment this to use the rm program instead of the built-in perl routine. # cmd_rm /bin/rm # rsync must be enabled for anything to work. This is the only command that # must be enabled. # cmd_rsync /usr/bin/rsync # Uncomment this to enable remote ssh backups over rsync. # cmd_ssh /usr/bin/ssh # Comment this out to disable syslog support. # cmd_logger /usr/bin/logger # Uncomment this to specify the path to "du" for disk usage checks. # If you have an older version of "du", you may also want to check the # "du_args" parameter below. # cmd_du /usr/bin/du # Uncomment this to specify the path to rsnapshot-diff. # # yum install rsnapshot put everything under /usr/bin # cmd_rsnapshot_diff /usr/bin/rsnapshot-diff # Specify the path to a script (and any optional arguments) to run right # before rsnapshot syncs files # #cmd_preexec /path/to/preexec/script # Specify the path to a script (and any optional arguments) to run right # after rsnapshot syncs files # #cmd_postexec /path/to/postexec/script ######################################### # BACKUP INTERVALS # # Must be unique and in ascending order # # i.e. hourly, daily, weekly, etc. # ######################################### interval hourly 3 interval daily 5 interval weekly 4 interval monthly 3 ############################################ # GLOBAL OPTIONS # # All are optional, with sensible defaults # ############################################ # Verbose level, 1 through 5. # 1 Quiet Print fatal errors only # 2 Default Print errors and warnings only # 3 Verbose Show equivalent shell commands being executed # 4 Extra Verbose Show extra verbose information # 5 Debug mode Everything # verbose 2 # Same as "verbose" above, but controls the amount of data sent to the # logfile, if one is being used. The default is 3. # loglevel 3 # If you enable this, data will be written to the file you specify. The # amount of data written is controlled by the "loglevel" parameter. # logfile /var/log/rsnapshot # If enabled, rsnapshot will write a lockfile to prevent two instances # from running simultaneously (and messing up the snapshot_root). # If you enable this, make sure the lockfile directory is not world # writable. Otherwise anyone can prevent the program from running. # lockfile /var/run/rsnapshot.pid # Default rsync args. All rsync commands have at least these options set. # # -a archive mode # -r recursive # -x do not span LVM boundaries # --delete delete files that don't exist on source # --relative relative directory names # --numeric-ids don't map uid/gid from user/group names # rsync_short_args -arx rsync_long_args --delete --numeric-ids --relative # ssh has no args passed by default, but you can specify some here. # # xfkey must have been generated on this system with no passphrase ssh_args -i /root/.ssh/xfkey # Default arguments for the "du" program (for disk space reporting). # The GNU version of "du" is preferred. See the man page for more details. # If your version of "du" doesn't support the -h flag, try -k flag instead. # #du_args -csh # If this is enabled, rsync won't span filesystem partitions within a # backup point. This essentially passes the -x option to rsync. # The default is 0 (off). # #one_fs 0 # The include and exclude parameters, if enabled, simply get passed directly # to rsync. If you have multiple include/exclude patterns, put each one on a # separate line. Please look up the --include and --exclude options in the # rsync man page for more details on how to specify file name patterns. # #include ??? #include ??? #exclude ??? #exclude ??? # The include_file and exclude_file parameters, if enabled, simply get # passed directly to rsync. Please look up the --include-from and # --exclude-from options in the rsync man page for more details. # #include_file /path/to/include/file #exclude_file /path/to/exclude/file # If your version of rsync supports --link-dest, consider enable this. # This is the best way to support special files (FIFOs, etc) cross-platform. # The default is 0 (off). # link_dest 1 # When sync_first is enabled, it changes the default behaviour of rsnapshot. # Normally, when rsnapshot is called with its lowest interval # (i.e.: "rsnapshot hourly"), it will sync files AND rotate the lowest # intervals. With sync_first enabled, "rsnapshot sync" handles the file sync, # and all interval calls simply rotate files. See the man page for more # details. The default is 0 (off). # #sync_first 0 # If enabled, rsnapshot will move the oldest directory for each interval # to [interval_name].delete, then it will remove the lockfile and delete # that directory just before it exits. The default is 0 (off). # #use_lazy_deletes 0 ############################### ### BACKUP POINTS / SCRIPTS ### ############################### # LOCALHOST backup_script /root/bin/backup-xf.sh unused0 backup /home/ local-xf/ backup /root/ local-xf/ backup /etc/ local-xf/ #backup /usr/ local-xf/ backup /var/log/ local-xf/ # SCHOOLSERVER backup_script /bin/date "+ backup of schoolserver started at %c" unused1 backup_script /root/bin/remote-xs.sh unused2 backup root@schoolserver:/home/ remote-xs/ backup root@schoolserver:/root/ remote-xs/ backup root@schoolserver:/etc/ remote-xs/ backup root@schoolserver:/var/log remote-xs/ backup_script /bin/date "+ backup of schoolserver ended at %c" unused3
Here is my remote-xs.sh script (MIT license)
#!/bin/sh # # Copyright (c) 2008 Tony Pearson. # # Licensed under the MIT license for contribution to the # One Laptop per Child (OLPC) foundation. # # Permission is hereby granted, free of charge, to any person # obtaining a copy of this software and associated documentation # files (the "Software"), to deal in the Software without # restriction, including without limitation the rights to use, # copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the # Software is furnished to do so, subject to the following # conditions: # # The above copyright notice and this permission notice shall be # included in all copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES # OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT # HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, # WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR # OTHER DEALINGS IN THE SOFTWARE. # # For more information see the Open Source Initiative: # http://www.opensource.org/licenses/mit-license.php # KEY="/root/.ssh/xfkey" DEST="root@schoolserver" ssh -i $KEY $DEST "/root/bin/backup-xs.sh"