|
|
| (17 intermediate revisions by 5 users not shown) |
| Line 1: |
Line 1: |
| | == Backups == |
|
| |
|
| === Backup ===
| | All backups can be found in the '''/backup''' directory on the UIs.<br> |
| | The /user, /group, /software & /ice3 are backed up every day to a secondary ceph storage cluster, in case our production cluster goes down. |
|
| |
|
| The information below is outdated: you cannot access the backups anymore. Send a mail to grid_admin AT listserv.vub.ac.be with the path of the files you want and the date.
| |
| We are thinking of changes to this backup access, and will update the twiki accordingly (06 June 2015).
| |
|
| |
|
| ==== User accessible backups ==== ==> NOT UP-TO-DATE
| | !! NOTE: '''/pnfs''' is '''''NOT''''' backed-up, as it is a massive storage !! |
|
| |
|
| The new file server allows us to make snapshots that are accessible for all the users to read. They are located at:
| | !! NOTE2: Those backups are READ-ONLY, so extract the files you want from them into your normal directory !! |
|
| |
|
| <pre>
| |
| /user/.zfs/snapshot/
| |
| </pre>
| |
| These snapshots contain all your files that have been modified or deleted since the day the snapshot was taken. <br>
| |
|
| |
|
| Under this directory you will find a number of sub-directories, ending in @weekly.<nr> and @daily.<nr>. Daily snapshots are taken on all days except Sunday. 0 is the latest one (the previous day). Weekly backups are taken on Sunday and the numbering convention is the same as for the daily ones.<br>
| | === /user, /group, /software & /ice3 === |
| All these directories are directly browsable and you can copy you old files back to you home directory. Remark that the snapshots are read only, so you should not attempt to modify anything in it.<br>
| |
| <br>
| |
| In addition, the localgrid area also has a daily snapshot. No weekly snapshots are provided here, only 7 daily snapshots as this is only meant to run on the cluster. The snapshots are to be found at:
| |
|
| |
|
| <pre>
| |
| /localgrid/.zfs/snapshot/
| |
| </pre>
| |
|
| |
|
| | Our ceph cluster allows us to do regular snapshots, they can be found in '''/backup/$DATE/{user,group,software,ice3}''': |
| | * There is one done every day at 8.30am, we keep the last seven of those (so a complete week), eg: |
| | :: <pre>scheduled-2024-02-04-08_30_00_UTC</pre> |
|
| |
|
| <br>
| | * We also keep the last 4 Sunday snapshots, equivalent to a month. |
| <br>
| |
| We however encourage you to commit your code to the CERN cvs every night, to minimize the possible losses. | |
| ==== Non accessible backups ====
| |
| | |
| Additionally to this, weekly on Tuesday, a full backup to a different machine is taken in case a serious hardware problem would occur on our system. This backup is made every Tuesday.
| |
| | |
| ==== IMPORTANT: /localgrid ====
| |
| | |
| For people relying heavily on their work on /localgrid (for local submission), it is important to notice that the files under /localgrid are not backed up at all.<br>
| |
| Therefore, we encourage you to take the necessary steps if you need so. The best thing is to back everything up on the /user disk. What follows set this up in an automated fashion so that backup is done automatically.<br>
| |
| Make a directory that will hold the backup on /user and make the initial backup:
| |
| <pre>
| |
| mkdir /user/odevroed/LocalgridBackup
| |
| rsync -a /localgrid/Odevroed /user/odevroed/LocalgridBackup
| |
| </pre>
| |
| | |
| Next, set up a cron job that will do this automatically every day:
| |
| | |
| <pre>
| |
| crontab -e
| |
| </pre>
| |
| This opens a vi editor. Type i to insert text and insert:
| |
| | |
| <pre>
| |
| 00 00 * * * rsync -a --delete /localgrid/odevroed /user/odevroed/LocalgridBackup
| |
| </pre>
| |
| | |
| The --delete is necessary, as files you deleted on /localgrid need to be deleted in the backup too. If you do not do this, the backup will continue to grow and you will reach the limit of your quota.<br>
| |
| The crontab can only be done on M3. In this way, we can make a backup of the necessary files that cron uses.
| |
| | |
| | |
| | |
| | |
| | |
| | |
| {{TracNotice|{{PAGENAME}}}}
| |
Backups
All backups can be found in the /backup directory on the UIs.
The /user, /group, /software & /ice3 are backed up every day to a secondary ceph storage cluster, in case our production cluster goes down.
!! NOTE: /pnfs is NOT backed-up, as it is a massive storage !!
!! NOTE2: Those backups are READ-ONLY, so extract the files you want from them into your normal directory !!
/user, /group, /software & /ice3
Our ceph cluster allows us to do regular snapshots, they can be found in /backup/$DATE/{user,group,software,ice3}:
- There is one done every day at 8.30am, we keep the last seven of those (so a complete week), eg:
scheduled-2024-02-04-08_30_00_UTC
- We also keep the last 4 Sunday snapshots, equivalent to a month.