User:Az990tony/backup

From OLPC
Jump to: navigation, search

Overview

In my test environment, I have two machines. 

+------------+          +------------+        +-------------------+
| Internet   |--------- | XF (squid) |--------| XS (schoolserver) |
| Modem      |          |            |        | LAMP+Moodle       |
+------------+          +------------+        +-------------------+

In this backup scheme:

  • XF will backup its own files and files from XS
  • XS will backup its own files and files from XF

This backup scheme is based on rsnapshot. See http://rsnapshot.org for details.

Under /library/.snapshots/ there will be the following directories

  • hourly.0 # most recent backup
  • hourly.1 # previous backup
  • hourly.2 # backup before that
  • daily.0 # yesterday's hourly.0
  • daily.1 # hourly.0 of the day before that
  • daily.2
  • daily.3
  • daily.4
  • weekly.0 # hourly.0 of last Monday's backup
  • weekly.1 # hourly.0 of previous Monday
  • weekly.2
  • weekly.3
  • monthly.0 # first backup of this month
  • monthly.1 # first backup of last month
  • monthly.2 # first backup of month before that

The backups use hard links, so if not much changes, the total amount of space is actually quite small.

XF (squid/filter/cache)

Here are the files and directories involved on XF

  • /root/.ssh/xfkey # this is a key generated on XF with no passphrase
  • /root/.ssh/xskey.pub # this is a key generated on XS with no passphrase
  • /root/bin/remote-xs.sh # this script will invoke backup-xs.sh script on XS
  • /root/bin/backup-xf.sh # XS remote-xf.sh will invoke this script
  • /etc/rsnapshot.conf # rsnapshot performs rsync over ssh connection
  • /library/.snapshots/ # all of the backups will be kept here
  • /library/.snapshots/hourly.0/local-xf/ # backups of local XF taken here
  • /library/.snapshots/hourly.0/remote-xs/ # backups of remote XS taken here over SSH

rsnapshot.conf

This is the /etc/rsnapshot.conf I used. 

#################################################
# rsnapshot.conf - rsnapshot configuration file #
#################################################
#                                               #
# PLEASE BE AWARE OF THE FOLLOWING RULES:       #
#                                               #
# This file requires tabs between elements      #
#                                               #
# Directories require a trailing slash:         #
#   right: /home/                               #
#   wrong: /home                                #
#                                               #
# updated by Tony Pearson 02/26/2008 xf         #
#################################################

#######################
# CONFIG FILE VERSION #
#######################

config_version	1.2

###########################
# SNAPSHOT ROOT DIRECTORY #
###########################

# All snapshots will be stored under this root directory.
#
# /library is LVM with most space 
#
snapshot_root	/library/.snapshots/

# If no_create_root is enabled, rsnapshot will not automatically create the
# snapshot_root directory. This is particularly useful if you are backing
# up to removable media, such as a FireWire or USB drive.
#
#no_create_root	1

#################################
# EXTERNAL PROGRAM DEPENDENCIES #
#################################

# LINUX USERS:   Be sure to uncomment "cmd_cp". This gives you extra features.
# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
#
# See the README file or the man page for more details.
#
cmd_cp		/bin/cp

# uncomment this to use the rm program instead of the built-in perl routine.
#
cmd_rm		/bin/rm

# rsync must be enabled for anything to work. This is the only command that
# must be enabled.
#
cmd_rsync	/usr/bin/rsync

# Uncomment this to enable remote ssh backups over rsync.
#
cmd_ssh		/usr/bin/ssh

# Comment this out to disable syslog support.
#
cmd_logger	/usr/bin/logger

# Uncomment this to specify the path to "du" for disk usage checks.
# If you have an older version of "du", you may also want to check the
# "du_args" parameter below.
#
cmd_du		/usr/bin/du

# Uncomment this to specify the path to rsnapshot-diff.
#
# yum install rsnapshot put everything under /usr/bin
#
cmd_rsnapshot_diff	/usr/bin/rsnapshot-diff

# Specify the path to a script (and any optional arguments) to run right
# before rsnapshot syncs files
#
#cmd_preexec	/path/to/preexec/script

# Specify the path to a script (and any optional arguments) to run right
# after rsnapshot syncs files
#
#cmd_postexec	/path/to/postexec/script

#########################################
#           BACKUP INTERVALS            #
# Must be unique and in ascending order #
# i.e. hourly, daily, weekly, etc.      #
#########################################

interval	hourly	3
interval	daily	5
interval	weekly	4
interval	monthly	3

############################################
#              GLOBAL OPTIONS              #
# All are optional, with sensible defaults #
############################################

# Verbose level, 1 through 5.
# 1     Quiet           Print fatal errors only
# 2     Default         Print errors and warnings only
# 3     Verbose         Show equivalent shell commands being executed
# 4     Extra Verbose   Show extra verbose information
# 5     Debug mode      Everything
#
verbose		2

# Same as "verbose" above, but controls the amount of data sent to the
# logfile, if one is being used. The default is 3.
#
loglevel	3

# If you enable this, data will be written to the file you specify. The
# amount of data written is controlled by the "loglevel" parameter.
#
logfile	/var/log/rsnapshot

# If enabled, rsnapshot will write a lockfile to prevent two instances
# from running simultaneously (and messing up the snapshot_root).
# If you enable this, make sure the lockfile directory is not world
# writable. Otherwise anyone can prevent the program from running.
#
lockfile	/var/run/rsnapshot.pid

# Default rsync args. All rsync commands have at least these options set.
#
# -a 		archive mode
# -r 		recursive
# -x 		do not span LVM boundaries
# --delete      delete files that don't exist on source
# --relative    relative directory names
# --numeric-ids don't map uid/gid from user/group names
#
rsync_short_args	-arx
rsync_long_args	--delete --numeric-ids --relative 

# ssh has no args passed by default, but you can specify some here.
#
# xfkey must have been generated on this system with no passphrase
ssh_args	-i /root/.ssh/xfkey

# Default arguments for the "du" program (for disk space reporting).
# The GNU version of "du" is preferred. See the man page for more details.
# If your version of "du" doesn't support the -h flag, try -k flag instead.
#
#du_args	-csh

# If this is enabled, rsync won't span filesystem partitions within a
# backup point. This essentially passes the -x option to rsync.
# The default is 0 (off).
#
#one_fs		0

# The include and exclude parameters, if enabled, simply get passed directly
# to rsync. If you have multiple include/exclude patterns, put each one on a
# separate line. Please look up the --include and --exclude options in the
# rsync man page for more details on how to specify file name patterns. 
# 
#include	???
#include	???
#exclude	???
#exclude	???

# The include_file and exclude_file parameters, if enabled, simply get
# passed directly to rsync. Please look up the --include-from and
# --exclude-from options in the rsync man page for more details.
#
#include_file	/path/to/include/file
#exclude_file	/path/to/exclude/file

# If your version of rsync supports --link-dest, consider enable this.
# This is the best way to support special files (FIFOs, etc) cross-platform.
# The default is 0 (off).
#
link_dest	1	

# When sync_first is enabled, it changes the default behaviour of rsnapshot.
# Normally, when rsnapshot is called with its lowest interval
# (i.e.: "rsnapshot hourly"), it will sync files AND rotate the lowest
# intervals. With sync_first enabled, "rsnapshot sync" handles the file sync,
# and all interval calls simply rotate files. See the man page for more
# details. The default is 0 (off).
#
#sync_first	0

# If enabled, rsnapshot will move the oldest directory for each interval
# to [interval_name].delete, then it will remove the lockfile and delete
# that directory just before it exits. The default is 0 (off).
#
#use_lazy_deletes	0

###############################
### BACKUP POINTS / SCRIPTS ###
###############################

# LOCALHOST
backup_script		/root/bin/backup-xf.sh	unused0
backup	/home/		local-xf/
backup	/root/		local-xf/
backup	/etc/		local-xf/
#backup	/usr/		local-xf/
backup	/var/log/	local-xf/

# SCHOOLSERVER
backup_script	/bin/date "+ backup of schoolserver started at %c"	unused1
backup_script	/root/bin/remote-xs.sh		unused2
backup	root@schoolserver:/home/	remote-xs/
backup	root@schoolserver:/root/	remote-xs/
backup	root@schoolserver:/etc/		remote-xs/
backup	root@schoolserver:/var/log	remote-xs/
backup_script	/bin/date "+ backup of schoolserver ended  at %c"	unused3

Here is my remote-xs.sh script (MIT license) 

#!/bin/sh
#
# Copyright (c) 2008 Tony Pearson.
#
# Licensed under the MIT license for contribution to the 
# One Laptop per Child (OLPC) foundation.
# 
# Permission is hereby granted, free of charge, to any person 
# obtaining a copy of this software and associated documentation 
# files (the "Software"), to deal in the Software without 
# restriction, including without limitation the rights to use, 
# copy, modify, merge, publish, distribute, sublicense, and/or sell 
# copies of the Software, and to permit persons to whom the 
# Software is furnished to do so, subject to the following 
# conditions: 
# 
# The above copyright notice and this permission notice shall be 
# included in all copies or substantial portions of the Software. 
# 
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES 
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT 
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 
# OTHER DEALINGS IN THE SOFTWARE. 
# 
# For more information see the Open Source Initiative:
# http://www.opensource.org/licenses/mit-license.php
#
KEY="/root/.ssh/xfkey"
DEST="root@schoolserver"
ssh -i $KEY $DEST "/root/bin/backup-xs.sh"
Retrieved from "http://wiki.laptop.org/mediawiki/index.php?title=User:Az990tony/backup&oldid=112981"