commit 705ff92e0304b71d704145cd4299695a5e33dabf Author: Janek Bevendorff Date: Tue Apr 2 20:25:38 2013 +0200 Initial commit diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..029b94f --- /dev/null +++ b/.gitignore @@ -0,0 +1,11 @@ +*.tmp +*.swp +.directory +Thumbs.db +desktop.ini +Desktop.ini +.DS_Store + +server/usr/local/etc/rs-backup/server-config +client/usr/local/etc/rs-backup/client-config +client/usr/local/etc/rs-backup/include-files diff --git a/README.md b/README.md new file mode 100644 index 0000000..ac1f536 --- /dev/null +++ b/README.md @@ -0,0 +1,70 @@ +# rs-backup-suite + +rs-backup-suite is a set of shell scripts for setting up a custom NAS on a computer in the network. It uses [rsync](http://rsync.samba.org/) and [rsnapshot](http://www.rsnapshot.org/). + +## How it works +rs-backup-suite is designed for push backups, which means the client pushes its files to the server. This is ideal for computers which are not always on such as most desktop PCs. + +It is also a user-centric backup system. That means each user creates his own backup on the NAS instead of root backing up the whole machine at once (although this is possible). That also means that each user has a UNIX account on the NAS. The NAS username is usually - (e.g. mymachine-johndoe). + +On the client machine(s) each user can create a file called `.rs-backup-include` (name is configurable) inside his home directory which includes the list of files to that should be considered by the backup. Additionally root can maintain a similar file located at `/usr/local/etc/rs-backup/include-files` for the system files. + +## Setup (please read this carefully before performing any actions!) +rs-backup-suite is split into two parts: a client part for pushing the backup to the NAS and a server part which runs on the NAS itself. + +### Server +For the server part simply copy the contents of the `server` directory to your root directory and all the necessary files will be in place. Make sure that all files that are copied to `/usr/local/bin` and `/usr/local/sbin` are executable. Furthermore make sure that `/usr/local/bin` and `/usr/local/sbin` are in your `$PATH` environment variable as root. Finally rename the file `/usr/local/etc/server-config.example` to `/usr/local/etc/server-config`. + +#### Adding a backup user +A backup user can be created by running + + rs-add-user hostname username [ssh-public-key-file] + +where `hostname` is the name of the client host and `username` is the name of the user on that machine for whom this account is made. Of course you can use any other names for `hostname` and `username` as well, but it's generally a good idea to stick to this naming convention. The resulting UNIX username will be the combination of both. + +The optional third parameter specifies the path to the SSH public key file which the user will use to log into the NAS. If you don't specify it, the user won't be able to log in at all. But you can add one later at any time by running + + rs-add-ssh-key hostname username ssh-public-key-file + +`hostname` and `username` are the same as above and mandatory for identifying the user that should get the new key. + +**TIP:** If you don't remember the parameters for all these commands, simply run them without any and you'll get simple usage instructions. + +#### Making the chroot work +rs-backup-suite can chroot backup users into the backup home base directory. For this to work you need to add the following to your `/etc/fstab` and run `mount -a` afterwards: + + # Chroot + /bin /bkp/bin none bind 0 0 + /lib /bkp/lib none bind 0 0 + /usr/bin /bkp/usr/bin none bind 0 0 + /usr/lib /bkp/usr/lib none bind 0 0 + /usr/local/bin /bkp/usr/local/bin none bind 0 0 + /usr/share/perl5 /bkp/usr/share/perl5 none bind 0 0 + /dev /bkp/dev none bind 0 0 + +Then add this to the end of your `/etc/ssh/sshd_config`: + + Match Group backup + ChrootDirectory /bkp/ + +Then restart OpenSSH. Your backup users are now chrooted into `/bkp`. + +**NOTE:** When using a chroot environment and you change anything in your user configuration (e.g. the username) you need to run `rs-update-passwd` or your user might not be able to log in anymore. + +#### Tweaking the configuration file +The configuration file is `/usr/local/etc/server-config`. There you can configure the following directives: + +* `BACKUP_ROOT`: The directory under which the home directories of the backup users are stored. The default is `/bkp` +* `FILES_DIR`: The directory under which the actual backups are kept (relative to the backup user's home directory). The default is `files`. +* `SET_QUOTA`: Whether to set disk quota for the users or not (for Ext3/4 file systems). Default is `false`. +* `QUOTA_SOFT_LIMIT`, `QUOTA_HARD_LIMIT`, `QUOTA_INODE_SOFT_LIMIT`, `QUOTA_INODE_HARD_LIMIT`: The individual limits for disk quota. Ignored, if `SET_QUOTA` is `false`. + +**WARNING:** Adjust these settings *before* you create backup users, because they won't be re-applied for already existing users! + +### Client +On the client machines the script `/usr/local/bin/rs-backup-run` is used for performing the backups. This script can either be run as root or as an unprivileged user. The behavior differs in both cases: + +* If run as root, all files and folder specified in `/usr/local/etc/rs-backup/include-files` will be backed up. The backup user used for logging into the NAS is `hostname-root` by default (where `hostname` is the hostname of the current machine). Additionally the home directories of all users will be scanned. If a home directory contains a file called `.rs-backup-include` all files and folders specified inside that file will be backed up under this user's privileges. The username user for logging into the NAS is `hostname-username` by default (where `hostname` is again substituted by the hostname of the current machine and `username` is substituted by the user whose home directory is being backed up). +* If run as a normal user, only the files that are specified in the `.rs-backup-include` file inside the own home directory will be backed up. + +`rs-backup-run` takes several command line arguments. To get a description for all of them run `rs-backup-run --help`. \ No newline at end of file diff --git a/client/usr/local/bin/rs-backup-run b/client/usr/local/bin/rs-backup-run new file mode 100755 index 0000000..4b58998 --- /dev/null +++ b/client/usr/local/bin/rs-backup-run @@ -0,0 +1,495 @@ +#!/bin/bash +## +# rs-backup-run +# Copyright (C) 2013 by Janek Bevendorff +# Website: http://www.refining-linux.org/ +# +# Script to push backups to a remote rsync backup server. +# Read the source code or use with --help parameter for +# further usage information. +## + + +############################################################################### +# Source configs +############################################################################### + +. /usr/local/etc/rs-backup/client-config + + +############################################################################### +# Additional internal config +############################################################################### + +_VERSION=0.1 +_GLOBAL_INCLUSION_PATTERN_FILE="/usr/local/etc/rs-backup/include-files" +_FORCED_INCLUSION_PATTERN_FILE="" +_SKIP_HOME_DIRS=false +_FORCED_LOG_FILE="" +_QUIET_MODE=false +_VERBOSE_MODE=false +_DRY_RUN=false +_FORCE_RUN=false +_ERROR_COUNT=0 + + +############################################################################### +# Function declarations +############################################################################### + +# Print usage instructions to the screen +# +# Usage: print_help +print_help() { + cat << HELP +rs-backup-run version ${_VERSION} +Copyright (C) 2013 by Janek Bevendorff +Web site: http://www.refining-linux.org/ + +Push backup to rsync backup server over SSH. + +If run as root and with no parameters, a full backup of all files and folders +specified in '${_GLOBAL_INCLUSION_PATTERN_FILE}' and all home directories +containing a proper backup config file will be pushed. + +If invoked without root privileges, only the home directory of the current +user will be backed up. + +Usage: $(basename $0) [OPTION]... + +Options: + -r, --remote-host=HOST The remote host to connect to + --remote-user=NAME The username to use for logging into the remote server + (%h will be replaced with the host name of this + machine and %u with your username) + --push-module=NAME The remote rsync server module + --ssh-options=OPTS Additional SSH options (will be merged with the default + options set in the rs-backup client-config file) + -o, --rsync-options=OPTS Additional options for rsync + -n, --dry-run Perform a test run (same as the --dry-run option for + rsync). Enable --verbose mode for useful control output + -s, --no-home-dirs Don't back up home dirs, only perform global system + backup (root only) + -i, --include-from=FILE Specify an alternate inclusion pattern file + This will override the default setting. If the script + is run as root, only the system backup will be + performed, no additional home directories will be + backed up + -l, --log-level=NUM Set log level to NUM (between 0 and 4) + --log-file=FILE Set a different log file location + -f, --force-run Force rs-backup to run, even if a lock file exists + -q, --quiet Don't print any error messages or warnings to the + screen (only write to log file) + -v, --verbose Print all messages of the current debug level + -h, --help Print this help and exit +HELP +} + +# Write log messages to screen and/or log file +# +# Usage: write_log +# +write_log() { + local log_msg + local log_date + local log_dest + + if [ $1 -gt 0 ] && [ $1 -le $LOG_LEVEL ]; then + case $1 in + 1) log_msg="ERROR: ${2}" ;; + 2) log_msg="WARNING: ${2}" ;; + 3) log_msg="INFO: ${2}" ;; + *) log_msg="DEBUG: ${2}" ;; + esac + + log_date="[$(date)]" + full_log_msg="${log_date} ${log_msg}" + + if [ "${_FORCED_LOG_FILE}" != "" ]; then + log_dest=${_FORCED_LOG_FILE} + elif [ $(id -u) -eq 0 ]; then + log_dest=${LOG_FILE} + elif [ "${HOME}" != "" ] && [ "${USER_LOG_FILE}" != "" ]; then + log_dest=${HOME}/${USER_LOG_FILE} + else + echo "WARNING: Couldn't determine valid log file location, using '/var/tmp'..." >&2 + log_dest="/var/tmp/${LOG_FILE}" + fi + + if ! test_file_perms "w" "${log_dest}"; then + echo "ERROR: Couldn't open log file for writing, redirecting to STDOUT!" >&2 + echo "${log_date} ${log_msg}" >&1 + else + echo "${log_date} ${log_msg}" >> "${log_dest}" + fi + + if ! $_QUIET_MODE && [ $1 -eq 1 ]; then + $_VERBOSE_MODE || $PRINT_ERRORS && echo "${log_msg}" >&2 + elif ! $_QUIET_MODE && [ $1 -le 2 ]; then + $_VERBOSE_MODE || $PRINT_WARNINGS && echo "${log_msg}" >&2 + elif ! $_QUIET_MODE && [ $1 -gt 2 ]; then + $_VERBOSE_MODE && echo "${log_msg}" >&1 + fi + + fi + + # Increase error count + if [ $1 -eq 1 ]; then + _ERROR_COUNT=$(($_ERROR_COUNT + 1)) + fi +} + + +# Test if a file is readable and/or writeable +# +# Usage: test_file_perms +# +test_file_perms() { + local perms=$1 + local filename=$2 + local result + + if [ "${perms}" == "r" ]; then + [ -f "${filename}" ] && [ -r "${filename}" ] + elif [ "${perms}" == "w" ]; then + [ -f "${filename}" -a -w "${filename}" ] || [ ! -e "${2}" -a -w "$(dirname ${2})" ] + elif [ "${perms}" == "rw" ]; then + [ -f "${filename}" ] && [ -w "${filename}" ] && [ -r "${filename}" ] + fi + + return $? +} + +# Generate and return remote user name for SSH login +# +# Usage: get_remote_username [] [] +# +get_remote_username() { + local username=$1 + local hostname=$2 + + if [ "${username}" == "" ]; then + username=$(id -un) + fi + + if [ "${hostname}" == "" ]; then + hostname=$(hostname) + fi + + echo $(echo ${REMOTE_USER} | sed "s/%h/${hostname}/" | sed "s/%u/${username}/") +} + +# Get a stripped down version of the /etc/passwd file with all non-system users, +# their UIDs, GIDs, realpath'd home directories and shells +# Users whose home directories don't actually exist are skipped +# +# Usage: get_processed_passwd_file +# +get_processed_passwd_file() { + local line + local username + local uid + local gid + local home_dir + local shell + + cat /etc/passwd | while read line; do + uid=$(echo -n "${line}" | cut -d ":" -f 3) + + if [ $uid -lt 1000 ]; then + continue + fi + + home_dir=$(echo -n "${line}" | cut -d ":" -f 6) + home_dir=$(realpath "${home_dir}") + + if [ ! -d "${home_dir}" ]; then + continue + fi + + username=$(echo -n "${line}" | cut -d ":" -f 1) + gid=$(echo -n "${line}" | cut -d ":" -f 4) + shell=$(echo -n "${line}" | cut -d ":" -f 7) + + echo "${username}:x:${uid}:${gid}::${home_dir}:${shell}" + done +} + +# Back up a directory. +# The third parameter is optional and specifys the user under whose +# privileges the backup will be performed +# +# Usage: perform_backup [] +# +perform_backup() { + local inclusion_pattern_file=$1 + local destination=$2 + local username=$3 + local msg + local backup_cmd + local ssh_cmd + local rsync_opts + local exit_code + local tee_device="/dev/null" + + if $_DRY_RUN; then + rsync_opts="${RSYNC_OPTIONS} --dry-run" + fi + + if $_VERBOSE_MODE; then + tee_device="/dev/tty" + rsync_opts="${RSYNC_OPTIONS} --verbose" + else + rsync_opts="${RSYNC_OPTIONS} --quiet" + fi + + if [ "${username}" == "" ]; then + username=$(id -un) + fi + + if [ "${_FORCED_INCLUSION_PATTERN_FILE}" != "" ]; then + inclusion_pattern_file=${_FORCED_INCLUSION_PATTERN_FILE} + fi + + if ! test_file_perms "r" "${inclusion_pattern_file}"; then + write_log 1 "Pattern file '${inclusion_pattern_file}' does not exist or is not readable!" + return 1 + fi + + if [ "${SSH_OPTIONS}" != "" ]; then + ssh_cmd="ssh"$(echo -n "${SSH_OPTIONS}" | sed 's/\\/\\\\/g' | sed 's/"/\\"/g') + else + ssh_cmd="ssh" + fi + + backup_cmd="rsync \ + --rsh=\"${ssh_cmd}\" \ + --archive \ + --acls \ + --delete \ + --delete-excluded \ + --include-from=\"${inclusion_pattern_file}\" \ + --exclude=\"*\" \ + ${rsync_opts} \ + / \ + \"${destination}\"" + + write_log 4 "This is the command about to be run: $(echo ${backup_cmd})" + + set -o pipefail + + if [ $(id -u) -eq 0 ] && [ "${username}" != "$(id -un)" ]; then + write_log 4 "Running backup with privileges of user '${username}' (UID: $(id -u ${username}))..." + msg=$(su - "${username}" -c "${backup_cmd}" 2>&1 | tee "${tee_device}") + elif [ $(id -u) -ne 0 ] && [ "${username}" != "$(id -un)" ]; then + write_log 1 "Cannot run run backup as user '${username}' (UID: $(id -u ${username}), missing root privileges!" + return 1 + else + msg=$(sh -c "${backup_cmd}" 2>&1 | tee "${tee_device}") + fi + + exit_code=$? + + if [ ${exit_code} -ne 0 ]; then + write_log 1 "Backup failed! Error message: ${msg}" + return ${exit_code} + else + write_log 3 "Backup finished." + fi +} + +# Back up selected system files +# +# Usage: back_up_system +# +back_up_system() { + if [ $(id -u) -ne 0 ]; then + write_log 1 "Cannot perform system backup unless run as root!" + return 1 + fi + + write_log 4 "Entering directory '/root'..." + cd /root + + if $_DRY_RUN; then + write_log 3 "Starting global system backup (DRY RUN)..." + else + write_log 3 "Starting global system backup..." + fi + perform_backup "${_GLOBAL_INCLUSION_PATTERN_FILE}" "$(get_remote_username root)@${REMOTE_HOST}::${PUSH_MODULE}" +} + +# Back up single home directory +# +# Usage: back_up_single_home_dir +# +back_up_single_home_dir() { + local home_dir=$(realpath "${1}") + local username=$2 + + # Don't back up home directory if no files are marked for backup + if [ ! -f "${home_dir}/${INCLUSION_PATTERN_FILE}" ]; then + write_log 4 "Skipping '${home_dir}', because '${INCLUSION_PATTERN_FILE}' does not exist." + return + fi + + # Also don't create a backup if no SSH key exists and no custom SSH options were specified + if [ ! -f "${home_dir}/.ssh/id_rsa" ] && [ ! -f "${home_dir}/.ssh/config" ] && [ "${SSH_OPTIONS}" == "" ]; then + write_log 4 "Skipping '${home_dir}', because no proper SSH key could be found." + return + fi + + write_log 4 "Entering directory '${home_dir}'..." + cd "${home_dir}" + + if $_DRY_RUN; then + write_log 3 "Starting backup of '${home_dir}' (DRY RUN)..." + else + write_log 3 "Starting backup of '${home_dir}'..." + fi + perform_backup "${home_dir}/${INCLUSION_PATTERN_FILE}" "$(get_remote_username ${username})@${REMOTE_HOST}::${PUSH_MODULE}" "${username}" +} + +# Back up all home dirs +# +# Usage: back_up_home_dirs +# +back_up_home_dirs() { + write_log 3 "Starting backup of all home directories..." + + get_processed_passwd_file | while read line; do + back_up_single_home_dir "$(echo -n ${line} | cut -d ':' -f 6)" "$(echo -n ${line} | cut -d ':' -f 1)" + done +} + +# Prase command line args +# +# Usage: parse_cmd_args +# +parse_cmd_args() { + local args + local name=$(basename $0) + + if [ "$1" == "" ]; then + return + fi + + getopt -T > /dev/null + if [ $? -ne 4 ]; then + write_log 1 "Need GNU getopt for command line parameter parsing!" + exit 1; + fi + + args=$(getopt \ + -s sh \ + -o "r:o:nsi:l:fqvh" \ + -l "remote-host:,remote-user:,push-module:,ssh-options:,rsync-options:,dry-run,no-home-dirs,include-from:,log-level:,log-file:,force-run,quiet,verbose,help" \ + -n "${name}" \ + -- "${@}") + + if [ $? -ne 0 ]; then + exit 1 + fi + + eval set -- "${args}" + + while true; do + case "$1" in + "-r"|"--remote-host") + REMOTE_HOST=$2 + shift 2 ;; + "--remote-user") + REMOTE_USER=$2 + shift 2 ;; + "--push-module") + PUSH_MODULE=$2 + shift 2 ;; + "--ssh-options") + SSH_OPTIONS="${SSH_OPTIONS} ${2}" + shift 2 ;; + "-o"|"--rsync-options") + RSYNC_OPTIONS="${RSYNC_OPTIONS} $2" + shift 2 ;; + "-n"|"--dry-run") + _DRY_RUN=true + shift ;; + "-s"|"--no-home-dirs") + _SKIP_HOME_DIRS=true + shift ;; + "-i"|"--include-from") + # File must exist and be readable + ! test_file_perms "r" "${2}" && echo "$name: '${2}' does not exist or is not readable!" >&2 && exit 1 + _FORCED_INCLUSION_PATTERN_FILE=$2 + _SKIP_HOME_DIRS=true + shift 2 ;; + "-l"|"--log-level") + LOG_LEVEL=$2; + shift 2 ;; + "--log-file") + # Test if file is writeable + ! test_file_perms "w" "${2}" && echo "$name: '${2}' is not writeable!" >&2 && exit 1 + _FORCED_LOG_FILE=$2 + shift 2 ;; + "-f"|"--force-run") + _FORCE_RUN=true + shift ;; + "-q"|"--quiet") + _QUIET_MODE=true + shift ;; + "-v"|"--verbose") + ! $_QUIET_MODE && _VERBOSE_MODE=true + shift ;; + "-h"|"--help") + print_help + exit ;; + *) + shift + break ;; + esac + done +} + +############################################################################### +# Intitialize the actual backup +############################################################################### +parse_cmd_args "$@" + +# Check if a backup is already running +if [ -f /tmp/rs-backup.lock ] && ! $_FORCE_RUN; then + write_log 1 "Backup lock file exists. Either a backup is already running or it didn't shut down properly last time." + write_log 1 "If you're sure no backup is running right now, remove the lock file '/tmp/rs-backup.lock' or use the '--force-run' parameter." + exit 1 +fi + +write_log 4 "No other backup running, ready to start." + +# Create lock file +write_log 4 "Creating lock file..." +touch /tmp/rs-backup.lock + +# Check if script has been invoked as root +if [ $(id -u) -eq 0 ]; then + write_log 4 "Running as root, performing global system backup..." + back_up_system + if ! $_SKIP_HOME_DIRS; then + back_up_home_dirs + else + write_log 3 "Skipping home directory backup as requested." + fi +else + write_log 3 "Running without root privileges, only backing up user home directory..." + if [ "${HOME}" != "" ]; then + back_up_single_home_dir "$(realpath ${HOME})" "$(id -nu)" + else + write_log 2 "Current user has no home directory, skipping." + fi +fi + +write_log 4 "Removing lock file..." +rm /tmp/rs-backup.lock + +write_log 4 "Done." + +if [ $_ERROR_COUNT -gt 0 ]; then + exit 1 +fi diff --git a/client/usr/local/etc/rs-backup/client-config.example b/client/usr/local/etc/rs-backup/client-config.example new file mode 100644 index 0000000..fdc4686 --- /dev/null +++ b/client/usr/local/etc/rs-backup/client-config.example @@ -0,0 +1,55 @@ +## +# Global config file for rs-backup-run +# This is an example file. Copy it over without the .example extension +# and modify it to your needs. +# Most config options can also be passed directly as command line parameters. +## + +# Remote host to push the files to +# The remote host must have a working rsync server running which allows +# passwordless (public key) login over SSH +REMOTE_HOST="host" + +# The rsync module on the remote server +PUSH_MODULE="push" + +# Username to use for logging into the remote server. +# You can use the variables %h which will be replaced with the hostname +# of this machine and %u which will be replaced with your local username. +# If a global system backup is performed, %u will be 'root' for the global +# backup and the corresponding user for the individual home directories +REMOTE_USER="%h-%u" + +# Additional SSH options +#SSH_OPTIONS="-C -i .ssh/id_rsa" + +# Additional options for rsync +#RSYNC_OPTIONS="" + +# Global log file to use when running as root +LOG_FILE="/var/log/rs-backup.log" + +# Log filename (only basename) of the log file to use when running as a +# normal user. The file will be placed inside the user's home directory. +# Leave empty if you don't want any per-user log file. +# The user log file will only be written when the user originally invoked +# the script, not if his home directory is backed up during a full system +# backup run by root +USER_LOG_FILE="rs-backup.user.log" + +# Name of the file inside the users' home directories +# containing the patterns for matching files to include or exclude. +# The format is the same as the global 'include-files' config file +# and described in the FILTER RULES section of the rsync(1) man page. +# If no such file is found inside a home directory, it won't be backup up +INCLUSION_PATTERN_FILE=".rs-backup-include" + +# Log verbosity +# (0 = quiet, 1 = errors only, 2 = errors and warnings, 3 = info, 4 = debug) +LOG_LEVEL=3 + +# Send error messages to STDERR +PRINT_ERRORS=true + +# Send warnings to STDERR (implies PRINT_ERRORS=true) +PRINT_WARNINGS=true diff --git a/client/usr/local/etc/rs-backup/include-files.example b/client/usr/local/etc/rs-backup/include-files.example new file mode 100644 index 0000000..a38d5fb --- /dev/null +++ b/client/usr/local/etc/rs-backup/include-files.example @@ -0,0 +1,26 @@ +# List of files and folders to include in a system-wide backup. +# +# NOTE: you should only specify files that are part of the +# system and/or owned by root. Home directories should be backed +# up by the users themselves as this will keep file access rights +# separate on the backup server. +# +# For help with the syntax of this file consult the FILTER RULES +# section of the rsync(1) man page. + +- .directory +- Thumbs.db +- desktop.ini +- Desktop.ini +- .DS_Store +- *.tmp +- *.swp +- lost+found/*** + +/etc/*** +/usr +/usr/etc/*** +/usr/share/*** +/usr/local +/usr/local/etc/*** +/usr/local/share/*** diff --git a/server/bkp/bkp b/server/bkp/bkp new file mode 120000 index 0000000..945c9b4 --- /dev/null +++ b/server/bkp/bkp @@ -0,0 +1 @@ +. \ No newline at end of file diff --git a/server/bkp/etc/rsnapshot.global.conf b/server/bkp/etc/rsnapshot.global.conf new file mode 100644 index 0000000..625ed97 --- /dev/null +++ b/server/bkp/etc/rsnapshot.global.conf @@ -0,0 +1,16 @@ +config_version 1.2 + +cmd_cp /usr/bin/cp +cmd_rm /usr/bin/rm +cmd_rsync /usr/bin/rsync +cmd_logger /usr/bin/logger + +retain push 2 +retain daily 7 +retain weekly 4 +retain monthly 2 + +verbose 2 +loglevel 3 +one_fs 1 +sync_first 1 diff --git a/server/etc/cron.daily/rs-backup-rotate b/server/etc/cron.daily/rs-backup-rotate new file mode 100755 index 0000000..7f779a0 --- /dev/null +++ b/server/etc/cron.daily/rs-backup-rotate @@ -0,0 +1,2 @@ +#!/bin/sh +/usr/local/sbin/rs-rotate-cron daily diff --git a/server/etc/cron.monthly/rs-backup-rotate b/server/etc/cron.monthly/rs-backup-rotate new file mode 100755 index 0000000..85f53c0 --- /dev/null +++ b/server/etc/cron.monthly/rs-backup-rotate @@ -0,0 +1,2 @@ +#!/bin/sh +/usr/local/sbin/rs-rotate-cron monthly diff --git a/server/etc/cron.weekly/rs-backup-rotate b/server/etc/cron.weekly/rs-backup-rotate new file mode 100755 index 0000000..34de77d --- /dev/null +++ b/server/etc/cron.weekly/rs-backup-rotate @@ -0,0 +1,2 @@ +#!/bin/sh +/usr/local/sbin/rs-rotate-cron weekly diff --git a/server/etc/rs-skel/rsync.log b/server/etc/rs-skel/rsync.log new file mode 100644 index 0000000..e69de29 diff --git a/server/usr/local/bin/rs-rotate b/server/usr/local/bin/rs-rotate new file mode 100755 index 0000000..cfe2779 --- /dev/null +++ b/server/usr/local/bin/rs-rotate @@ -0,0 +1,15 @@ +#!/bin/sh + +if [ "$1" == "" ]; then + echo "Usage: $(basename $0) " + exit +fi + +if [ "$RSYNC_EXIT_STATUS" == "" ]; then + echo "This script is intended to be run as rsync post-xfer hook." 2>&1 + exit 1 +fi + +if [ $RSYNC_EXIT_STATUS -eq 0 ]; then + rsnapshot -c "$1" push +fi diff --git a/server/usr/local/bin/rs-run-ssh-cmd b/server/usr/local/bin/rs-run-ssh-cmd new file mode 100755 index 0000000..73dba13 --- /dev/null +++ b/server/usr/local/bin/rs-run-ssh-cmd @@ -0,0 +1,13 @@ +#!/bin/sh + +home_dir=$1 + +if [ "${SSH_ORIGINAL_COMMAND}" == "internal-sftp" ] || [ "${SSH_ORIGINAL_COMMAND}" == "/usr/lib/ssh/sftp-server" ]; then + cd "${home_dir}/files" + exec /usr/lib/ssh/sftp-server -R +else + exec /usr/bin/rsync --server --daemon --config="${home_dir}/rsync.conf" . +fi + +echo "Session failed." >&2 +exit 1 diff --git a/server/usr/local/etc/rs-backup/.server-config.swp b/server/usr/local/etc/rs-backup/.server-config.swp new file mode 100644 index 0000000..b5982cd Binary files /dev/null and b/server/usr/local/etc/rs-backup/.server-config.swp differ diff --git a/server/usr/local/etc/rs-backup/rsnapshot.conf.template b/server/usr/local/etc/rs-backup/rsnapshot.conf.template new file mode 100644 index 0000000..1888e5f --- /dev/null +++ b/server/usr/local/etc/rs-backup/rsnapshot.conf.template @@ -0,0 +1,6 @@ +include_conf ${BACKUP_ROOT}/etc/rsnapshot.global.conf + +snapshot_root ${HOME_DIR}/${FILES_DIR} +logfile ${HOME_DIR}/rsnapshot.log +lockfile ${HOME_DIR}/rsnapshot.pid +backup ${HOME_DIR}/${FILES_DIR}/.sync ./ diff --git a/server/usr/local/etc/rs-backup/rsync.conf.template b/server/usr/local/etc/rs-backup/rsync.conf.template new file mode 100644 index 0000000..68ee053 --- /dev/null +++ b/server/usr/local/etc/rs-backup/rsync.conf.template @@ -0,0 +1,20 @@ +[push] +uid = ${USERNAME} +gid = ${GROUPNAME} +path = ${HOME_DIR}/${FILES_DIR}/.sync +use chroot = 0 +read only = 0 +write only = 1 +log file = ${HOME_DIR}/rsync.log +fake super = 1 +max connections = 1 +lock file = ${HOME_DIR}/rsyncd.lock +post-xfer exec = /usr/local/bin/rs-rotate "${HOME_DIR}/rsnapshot.conf" + +[pull] +uid = ${USERNAME} +gid = ${GROUPNAME} +path = ${HOME_DIR}/${FILES_DIR} +use chroot = 0 +read only = 1 +fake super = 1 diff --git a/server/usr/local/etc/rs-backup/server-config.example b/server/usr/local/etc/rs-backup/server-config.example new file mode 100644 index 0000000..6bbb40d --- /dev/null +++ b/server/usr/local/etc/rs-backup/server-config.example @@ -0,0 +1,15 @@ +# Base directory for all backups +BACKUP_ROOT="/bkp" + +# Directory containing the actual backup files (relative to BACKUP_ROOT/) +FILES_DIR="files" + +# Set quota for individual backup users +SET_QUOTA=false + +# Quota limits (use 0 for no limit) +# Numbers may also end with k, M, G or T for magnitudes of 1024 +QUOTA_SOFT_LIMIT="350G" +QUOTA_HARD_LIMIT="355G" +QUOTA_INODE_SOFT_LIMIT="3900k" +QUOTA_INODE_HARD_LIMIT="4000k" diff --git a/server/usr/local/sbin/rs-add-ssh-key b/server/usr/local/sbin/rs-add-ssh-key new file mode 100755 index 0000000..9fedec0 --- /dev/null +++ b/server/usr/local/sbin/rs-add-ssh-key @@ -0,0 +1,28 @@ +#!/bin/sh +## +# rs-add-ssh-key +# Copyright (C) 2013 by Janek Bevendorff +# Website: http://www.refining-linux.org/ +# +# Add SSH key to a backup user for passwordless login +## + +if [ "$1" == "" ] || [ "$2" == "" ] || [ "$3" == "" ]; then + echo "Usage: $(basename $0) " + exit +fi + +. /usr/local/etc/rs-backup/server-config + +home_dir="${BACKUP_ROOT}/${1}-${2}" + +mkdir -p "${home_dir}/.ssh" + +# Add public key to authorized_keys file and restrict shell commands +echo -n "command=\"/usr/local/bin/rs-run-ssh-cmd '${home_dir}'\" " >> "${home_dir}/.ssh/authorized_keys" +cat "$3" >> "${home_dir}/.ssh/authorized_keys" + +# Protect SSH config by assigning ownership to root +chown -R root:root "${home_dir}/.ssh" +chmod 0755 "${home_dir}/.ssh" +chmod 0644 "${home_dir}/.ssh/authorized_keys" diff --git a/server/usr/local/sbin/rs-add-user b/server/usr/local/sbin/rs-add-user new file mode 100755 index 0000000..045be5c --- /dev/null +++ b/server/usr/local/sbin/rs-add-user @@ -0,0 +1,53 @@ +#!/bin/sh +## +# rs-add-user +# Copyright (C) 2013 by Janek Bevendorff +# Website: http://www.refining-linux.org/ +# +# Set up a backup user +## + +if [ "$1" == "" ] || [ "$2" == "" ]; then + echo "Usage: $(basename $0) []" + exit +fi + +. /usr/local/etc/rs-backup/server-config + +remote_hostname="$1" +remote_username="$2" +ssh_key_file="$3" +local_username="${remote_hostname}-${remote_username}" + +useradd -G backup -b "${BACKUP_ROOT}" -m -k /etc/rs-skel -p '*' -s /bin/sh "${local_username}" +rs-update-passwd + +# Generate config files from templates +USERNAME="${local_username}" +GROUPNAME="${local_username}" +HOME_DIR="${BACKUP_ROOT}/${local_username}" + +rsync_conf="$(cat /usr/local/etc/rs-backup/rsync.conf.template)" +rsnapshot_conf="$(cat /usr/local/etc/rs-backup/rsnapshot.conf.template)" + +rsync_conf=$(eval "cat << EOF +${rsync_conf} +EOF") + +rsnapshot_conf=$(eval "cat << EOF +${rsnapshot_conf} +EOF") + +echo "${rsync_conf}" > "${HOME_DIR}/rsync.conf" +echo "${rsnapshot_conf}" > "${HOME_DIR}/rsnapshot.conf" + +# Protect config files +chown root:root "${HOME_DIR}/rsync.conf" +chmod 0644 "${HOME_DIR}/rsync.conf" +chown root:root "${HOME_DIR}/rsnapshot.conf" +chmod 0644 "${HOME_DIR}/rsnapshot.conf" + +# If SSH key file has been specified +if [ "${ssh_key_file}" != "" ]; then + rs-add-ssh-key "${remote_hostname}" "${remote_username}" "${ssh_key_file}" +fi diff --git a/server/usr/local/sbin/rs-rotate-cron b/server/usr/local/sbin/rs-rotate-cron new file mode 100755 index 0000000..7afd6a8 --- /dev/null +++ b/server/usr/local/sbin/rs-rotate-cron @@ -0,0 +1,37 @@ +#!/bin/sh +# Create daily, weekly or monthly snapshots from manual push backups + +if [ "$1" == "" ]; then + echo "Usage: $(basename $0) " + exit +fi + +. /usr/local/etc/rs-backup/server-config + +for home_dir in "${BACKUP_ROOT}"/*; do + if [ -d "${home_dir}/${FILES_DIR}" ] && [ -e "${home_dir}/rsnapshot.conf" ]; then + # Since we're relying on the client to push changes, there isn't + # necessarily a new increment pushed yet. + # Only rotate if there are really new increments to prevent + # successive backup deletion + config=$(cat "${BACKUP_ROOT}/etc/rsnapshot.global.conf") + + # Get number of preceding increments + config=$(echo "${config}" | grep -P '^retain\t') + config=$(echo "${config}" | grep -oPz "retain\t+(\w+)\t+(\d+)\nretain\s+${1}\t+" | sed -n 1p) + preceding_name=$(echo "${config}" | awk '{ print $2 }') + preceding_number=$(($(echo "${config}" | awk ' { print $3 }') - 1)) + + # Continue if no proper preceding increment could be found + if [ "${preceding_name}" == "" ] || + [ ! -d "${home_dir}/${FILES_DIR}/${preceding_name}.${preceding_number}" ]; then + echo "Not rotating ${1}" + continue + fi + + cd "${home_dir}/${FILES_DIR}" + owner=$(ls -ld "${home_dir}/${FILES_DIR}" | awk '{ print $3 }') + + su - "${owner}" -c "rsnapshot -c '${home_dir}/rsnapshot.conf' '$1'" + fi +done diff --git a/server/usr/local/sbin/rs-update-passwd b/server/usr/local/sbin/rs-update-passwd new file mode 100755 index 0000000..1fed63f --- /dev/null +++ b/server/usr/local/sbin/rs-update-passwd @@ -0,0 +1,11 @@ +#!/bin/sh + +. /usr/local/etc/rs-backup/server-config + +# Take precautions :-) +if [ "${BACKUP_ROOT}" == "" ] || [ "$(realpath ${BACKUP_ROOT})" == "/" ]; then + echo "Sorry, won't overwrite real '/etc/passwd'!" + exit 1 +fi + +cat /etc/passwd | grep "::${BACKUP_ROOT}/[^/:]\+:/bin/sh$" > "${BACKUP_ROOT}/etc/passwd" diff --git a/server/usr/local/sbin/spin-down-storage b/server/usr/local/sbin/spin-down-storage new file mode 100755 index 0000000..cff518c --- /dev/null +++ b/server/usr/local/sbin/spin-down-storage @@ -0,0 +1,22 @@ +#!/bin/sh +# Check if disk has been used since last check and spin it down if not + +if [ "${1}" == "" ]; then + echo "Usage: $(basename ${0}) " + exit +fi + +last_state_file="/tmp/storage-state-${1}" + +touch $last_state_file +chmod 600 $last_state_file + +new_storage_state=$(cat /proc/diskstats | grep "$1") +old_storage_state=$(cat $last_state_file) + +if [ "$new_storage_state" = "$old_storage_state" ]; then + sync + sdparm --flexible --readonly --command=stop /dev/$1 2>&1 > /dev/null +fi + +echo "$new_storage_state" > $last_state_file