Create a free snapshot style Backup Server with Linux and Rsync

Leave a reply

Overview

Create a free snapshot style Backup Server with Linux and Rsync.

Create a free snapshot style Backup Server with Linux and Rsync

Requirements

– Linux Server/Desktop backup manger (we’ll use CentOS and we’ll call it BackupServer)
– Linux Servers/Desktops to backup (we’ll use CentOS and we’ll call it Source1)
– SSH Connection
– Root Access

Establish SSH connection without password between the BackupServer and Source1

Generate id_rsa, id_rsa.pub in the BackupServer:

1

ssh-keygen (do not use a password - just hit Enter)
ssh-keygen (do not use a password - just hit Enter)

Copy (and rename) id_rsa.pub to authorized_keys on Server

1

scp -r -P 22 /root/.ssh/id_rsa.pub Source1:/root/.ssh/authorized_keys
scp -r -P 22 /root/.ssh/id_rsa.pub Source1:/root/.ssh/authorized_keys

You can connect now as root between the servers without a password.

Mount source to backup

We will give an example of NFS but you can also use CIFS (SAMBA) mounts:

For NFS mount edit ‘/etc/fstab’ and add:

1

Source1:/vol1/ /source/vol1 nfs rsize=8192,wsize=8192,timeo=14,intr
Source1:/vol1/ /source/vol1 nfs rsize=8192,wsize=8192,timeo=14,intr

Create a folder to mount:

1

mkdir /source/vol1
mkdir /source/vol1

Phase 3: create the manager script files

Create /scripts/daily.sh:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

#!/bin/sh
echo "Starting Daily Backup"
echo "Please Wait..."
 
SNAPSHOT_RW=/backup;
 
# step 1: delete the oldest snapshot, if it exists:
if [ -d $SNAPSHOT_RW/daily.5 ] ;
then
        rm -rf $SNAPSHOT_RW/daily.5 ;
fi ;
 
# step 2: shift the middle snapshots(s) back by one, if they exist
if [ -d $SNAPSHOT_RW/daily.4 ] ;
then
        mv $SNAPSHOT_RW/daily.4 $SNAPSHOT_RW/daily.5 ;
fi;
if [ -d $SNAPSHOT_RW/daily.3 ] ;
then
        mv $SNAPSHOT_RW/daily.3 $SNAPSHOT_RW/daily.4 ;
fi;
if [ -d $SNAPSHOT_RW/daily.2 ] ;
then
        mv $SNAPSHOT_RW/daily.2 $SNAPSHOT_RW/daily.3 ;
fi;
if [ -d $SNAPSHOT_RW/daily.1 ] ;
then
        mv $SNAPSHOT_RW/daily.1 $SNAPSHOT_RW/daily.2 ;
fi;
 
# step 3: make a hard-link-only (except for dirs) copy of the latest snapshot, if that exists
if [ -d $SNAPSHOT_RW/daily.0 ] ;
then
        cp -al $SNAPSHOT_RW/daily.0 $SNAPSHOT_RW/daily.1 ;
fi;
 
# step 4: rsync from the system into the latest snapshot (notice that
# rsync behaves like cp --remove-destination by default, so the destination
# is unlinked first.  If it were not so, this would copy over the other
# snapshot(s) too!
rsync -va --delete --delete-excluded --exclude-from '/backup/exclude.txt' /eng/ $SNAPSHOT_RW/daily.0 ;
 
# step 5: update the mtime of daily.0 to reflect the snapshot time
touch $SNAPSHOT_RW/daily.0 ;
#!/bin/sh
echo "Starting Daily Backup"
echo "Please Wait..."

SNAPSHOT_RW=/backup;

# step 1: delete the oldest snapshot, if it exists:
if [ -d $SNAPSHOT_RW/daily.5 ] ;
then
        rm -rf $SNAPSHOT_RW/daily.5 ;
fi ;

# step 2: shift the middle snapshots(s) back by one, if they exist
if [ -d $SNAPSHOT_RW/daily.4 ] ;
then
        mv $SNAPSHOT_RW/daily.4 $SNAPSHOT_RW/daily.5 ;
fi;
if [ -d $SNAPSHOT_RW/daily.3 ] ;
then
        mv $SNAPSHOT_RW/daily.3 $SNAPSHOT_RW/daily.4 ;
fi;
if [ -d $SNAPSHOT_RW/daily.2 ] ;
then
        mv $SNAPSHOT_RW/daily.2 $SNAPSHOT_RW/daily.3 ;
fi;
if [ -d $SNAPSHOT_RW/daily.1 ] ;
then
        mv $SNAPSHOT_RW/daily.1 $SNAPSHOT_RW/daily.2 ;
fi;

# step 3: make a hard-link-only (except for dirs) copy of the latest snapshot, if that exists
if [ -d $SNAPSHOT_RW/daily.0 ] ;
then
        cp -al $SNAPSHOT_RW/daily.0 $SNAPSHOT_RW/daily.1 ;
fi;

# step 4: rsync from the system into the latest snapshot (notice that
# rsync behaves like cp --remove-destination by default, so the destination
# is unlinked first.  If it were not so, this would copy over the other
# snapshot(s) too!
rsync -va --delete --delete-excluded --exclude-from '/backup/exclude.txt' /eng/ $SNAPSHOT_RW/daily.0 ;

# step 5: update the mtime of daily.0 to reflect the snapshot time
touch $SNAPSHOT_RW/daily.0 ;

save the file and give execute permissions:

1

chmod +x /scripts/daily.sh
chmod +x /scripts/daily.sh

Create /scripts/weekly.sh:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

#!/bin/sh
echo "Starting Weekly Backup"
echo "Please Wait..."
 
SNAPSHOT_RW=/backup;
 
# step 1: delete the oldest snapshot, if it exists:
if [ -d $SNAPSHOT_RW/weekly.3 ] ;
then
        rm -rf $SNAPSHOT_RW/weekly.3 ;
fi ;
 
# step 2: shift the middle snapshots(s) back by one, if they exist
if [ -d $SNAPSHOT_RW/weekly.2 ] ;
then
    mv $SNAPSHOT_RW/weekly.2 $SNAPSHOT_RW/weekly.3 ;
fi;
if [ -d $SNAPSHOT_RW/weekly.1 ] ;
then
    mv $SNAPSHOT_RW/weekly.1 $SNAPSHOT_RW/weekly.2 ;
fi;
if [ -d $SNAPSHOT_RW/weekly.0 ] ;
then
    mv $SNAPSHOT_RW/weekly.0 $SNAPSHOT_RW/weekly.1;
fi;
 
# step 3: make a hard-link-only (except for dirs) copy of
# daily.5, assuming that exists, into weekly.0
if [ -d $SNAPSHOT_RW/daily.5 ] ;
then
    cp -al $SNAPSHOT_RW/daily.5 $SNAPSHOT_RW/weekly.0 ;
fi;
 
# note: do *not* update the mtime of weekly.0; it will reflect
# when daily.5 was made, which should be correct.
#!/bin/sh
echo "Starting Weekly Backup"
echo "Please Wait..."

SNAPSHOT_RW=/backup;

# step 1: delete the oldest snapshot, if it exists:
if [ -d $SNAPSHOT_RW/weekly.3 ] ;
then
        rm -rf $SNAPSHOT_RW/weekly.3 ;
fi ;

# step 2: shift the middle snapshots(s) back by one, if they exist
if [ -d $SNAPSHOT_RW/weekly.2 ] ;
then
    mv $SNAPSHOT_RW/weekly.2 $SNAPSHOT_RW/weekly.3 ;
fi;
if [ -d $SNAPSHOT_RW/weekly.1 ] ;
then
    mv $SNAPSHOT_RW/weekly.1 $SNAPSHOT_RW/weekly.2 ;
fi;
if [ -d $SNAPSHOT_RW/weekly.0 ] ;
then
    mv $SNAPSHOT_RW/weekly.0 $SNAPSHOT_RW/weekly.1;
fi;

# step 3: make a hard-link-only (except for dirs) copy of
# daily.5, assuming that exists, into weekly.0
if [ -d $SNAPSHOT_RW/daily.5 ] ;
then
    cp -al $SNAPSHOT_RW/daily.5 $SNAPSHOT_RW/weekly.0 ;
fi;

# note: do *not* update the mtime of weekly.0; it will reflect
# when daily.5 was made, which should be correct.

save the file and give execute permissions:

1

chmod +x /scripts/weekly.sh
chmod +x /scripts/weekly.sh

Create /scripts/exclude.txt (optional – excludes folders you don’t need to backup):

MOVED
Moved
TrashCan
Expired

Manage automatic backups

For example run the ‘weekly.sh’ script every Saturday at 02:05 :

1
2
3

crontab -e
2 5 * * 0-5 /scripts/daily.sh
2 5 * * 6 /scripts/weekly.sh
crontab -e
2 5 * * 0-5 /scripts/daily.sh
2 5 * * 6 /scripts/weekly.sh

This is an explanation of crontab:

Minute Hour Day of Month Month Day of Week Command
(0-59) (0-23) (1-31) (1-12 or Jan-Dec) (0-6 or Sun-Sat)

Refrences:
– http://www.mikerubel.org/computers/rsync_snapshots/

Related posts:

  1. Monitor FlexLM Licenses Status using Bash scripts
  2. Linux How-to: Mount ISO file is Easy!!
  3. Backup Multiple MySQL Databases using a BASH script

Leave a Reply

Your email address will not be published. Required fields are marked *

CAPTCHA

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.