BackupDocsLinuxLaptop
Introduction
This Wiki page is intended for Linux users and it is only about backup of personal documents. By "personal documents", we mean : spreadsheets, reports, presentations, images, eBooks, etc. So, it is not about backup of codes or data, since there are specific procedures for these kinds of backup. The volume of the personal files should not exceed a few hundreds of GB per user. A storage area has been created on x4500 for such backup tasks. It is accessible through the directory /laptops_backup on the lxpub<x> machines.
On this page, you will learn how to automate, with a bash script, the backup of your personal files to a distant storage area, using SSH for the connexion on it. And you will also learn how to schedule the automatic launching of the backup script with cron.
And here are the main steps you will pass through this guide :
- Configure your SSH account to connect with a key on the backup storage (lxpub<x>)
2. Configure your laptop to avoid the problem of the passphrase on your private SSH key
3. Configure the backup bash script
4. Configure cron on your laptop to automate the backup task
Once you have gone through these steps, your personal documents will be transparently backed up every hour.
The explanations below are given for Fedora, but can be easily transposed for Debian/Ubuntu.
Don't hesitate to ask us for help if you don't feel at ease with this quite lengthy procedure !
Configure your SSH connexion to the storage with a SSH keypair
From now on, all the explanations on this page will be given considering lxpub1 as the access point to the backup storage. But you can indifferently use lxpub2, lxpub4, lxpub5 or lxpub6. They all give access to the same storage area through the directory /laptops_backup.
Before going further in this section, you must have got a personal account (<your_user_name> in the explanations below) on lxpub1. If it is not the case, ask the system administrators.
When using SSH to access a distant machine, you have the choice between two main kinds of authentication method : password and keypair. Password authentication is to be banned for security reasons. In this section, we will show you how to implement keypair authentication to gain a secure and easy access to the storage for backup.
If you already have a SSH keypair, you can skip the two first steps (a. and b.).
l. A valid keypair can be easily generated with ssh-keygen. This program will create a public and a private key. Needless to say that you should protect your private key with a strong password and that you do not share the private key with others nor use the same private key on different machines. l. From the machine you use to connect run ssh-keygen -t rsa -b 2048
- It first prompts for the location of the files. Keep the default values unless you know what you are doing.
- Then it will prompt for a password. This is the password used to encrypt your key.
- This generates 2 files ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub, of which ~/.ssh/id_rsa.pub is the public key.
l. To enable authentication with your keypair on lxpub1, you will have to copy your public key in ~/.ssh/authorized_keys on that machine. Here is the simplest way to do it :
ssh-copy-id <your_user_name>@lxpub1
l. You can now try to connect : ssh <your_user_name>@lxpub1
- It's possible that your SSH client doesn't use SSH protocol 2 by default. If not, try to connect with e.g. "ssh -2 <your_user_name>@lxpub1". If this works, you can make this the default option by adding in ~/.ssh/config the line
Protocol 2
Avoiding the problem of the passphrase
As a wise user, you probably put a passphrase on your ssh private key to protect it. But this will be problem if you use cron to launch the backup script, because cron doesn't know you passphrase ! There is a safe and clever way to avoid this issue : keychain. Thanks to keychain, you'll be asked your passphrase once per session, just after your login. All subsequent passphrase request will be fulfilled by keychain, in a transparent way for you.
To install keychain, you must first install ssh-agent (you must be root for that) :
yum install ssh-agent yum install keychain
Now, keychain must ask you your passphrase just after the login. To ensure this, edit your .bash_profile with favourite editor :
vi $HOME/.bash_profile
and put the following lines in it :
### START-Keychain ### /usr/bin/keychain $HOME/.ssh/id_rsa source $HOME/.keychain/$HOSTNAME-sh ### End-Keychain ###
If you restart your computer, right after login step, there should be a new window asking you your passphrase.
The backup script
First, log on lxpub1 and create in /laptops_backup the directory that will contain your backup, taking care of access rights :
ssh <your_user_name>@lxpub1 mkdir /laptops_backup/<your_user_name> chmod 700 /laptops_backup/<your_user_name>
Now, log off from lxpub1 and on your laptop, edit your backup script :
vi ~/backup_laptop.sh
Here is an example of backup script you can adapt to your case :
#!/bin/bash # Backup script for my Linux laptop echo "" echo "" echo "BACKUP --BEGIN--" date logger "BACKUP TO UI BEGINS" # To avoid the problem with passphrase source $HOME/.keychain/$HOSTNAME-sh ####################################### # DEFINE THE ENDPOINTS OF THE BACKUP # ####################################### # here define the source path bck_src=/home/stgerard/ # here define the destination path bck_dest=/laptops_backup/stgerard/ # things you want to exclude from the backup (relative to the source path) excludes="--exclude .gvfs --exclude Download --exclude Music" ####################################### # DEFINE SSH CONNEXION PARAMS # ####################################### # define the user name user=stgerard # define the name of the destination machine for the backup machine=lxpub1.iihe.ac.be # give the full name of your private SSH key privatekey=$HOME/.ssh/id_rsa ####################################### # DEFINE RSYNC OPTIONS # ####################################### options="-avz --numeric-ids --delete --delete-excluded" # ------------------------------------- # create the destination directory ssh -xq $user@$machine "mkdir -p $bck_dest 2>&1" rsync $options -e "ssh -i $privatekey" $bck_src $excludes $user@$machine:$bck_dest logger "BACKUP TO UI FINISHED" echo "BACKUP --END--" echo "" echo ""
After having created this script, make it executable :
chmod u+x ~/backup_laptop.sh
We strongly advise you to read the man pages of the rsync command (or any documentation you like), to fully understand the consequences of each option. Notice the way we exclude some sub-directories from the backup source, with several "--exclude" options. Take also care of the trailing slash at the end of the source path ("/home/stgerard/") : it means we want to copy the contain of the source, without creating a new directory level. Be also aware that the parent directory of the destination must exist. In our example, /laptops_backup/stgerard/ must already exist, otherwise rsync will fail to create the destination. That's why we do a mkdir just before the rsync in the bash script. And finally, you should now that the "--delete" option will remove from destination all the files which are no more present in the source. In clear, it means that if you delete a file from the source, it will also be removed from the backup. This behaviour is suitable to save space on the storage, but, in the same time, it may be a drawback if the backup task is launched after you have accidentally deleted some files on your laptop !
Automate the backup with cron
To automate the launch of the previous backup script with cron, type :
crontab -e
You are now in a vi-like editor. Here is an example you can adapt to your case :
03 * * * * /home/stgerard/backup_laptop.sh >/home/stgerard/backup_laptop.log 2>&1
In this example, the script will be launched every hour and three minutes, and its output is redirected to a log file.