MonWebServicesMigration
Goal
For the record, mon is a CentOS 46 Xen virtual machine on top of fs.wn. It is still living in the old CB5 branch of our Quattor SVN. It is mainly used as a Web server, hosting this Trac Wiki.
We want a new mon in CB6, with a more recent OS (SL 55) installed on a KVM virtual machine on top of one of our dom0x.wn virtualization hosts.
Since we will upgrade the OS, we will also have to upgrade all the Web related stuffs (Apache, Trac, MySQL,...).
Planning the migration of our Trac Wikis on a test machine
Installation of the test machine
We used Quattor with the "base" machine-type. Its OS is Scientific Linux SL release 5.5 (Boron). Here are the hardware features of the VM : 2GB RAM, 30GB hard disk, 2 CPUs.
What are the needed RPMs ?
- First, shutdown the two Quattor daemons, since we will have to change the machine manually.
- Subversion is already installed and is version : subversion-1.4.2-4.el5_3.1
- Python is already installed and is version : python-2.4.3-27.el5
- Install DAG repo :
rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS//rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm
RPM installation of trac and dependencies :
[root@laranja ~]# yum install trac Installing: trac noarch 0.12-2.el5.rf Installing for dependencies: httpd x86_64 2.2.3-43.sl5.3 mod_python x86_64 3.2.8-3.1 python-genshi noarch 0.6-1.el5.rf python-setuptools noarch 0.6c5-2.el5 python-sqlite2 x86_64 2.3.3-1.el5.rf
Trac migration of our Wikis
From trac-0.10.4-1.el4.rf on mon to trac-0.12-2.el5.rf on laranja.
- On mon, get the list of the Wikis to migrate :
[root@mon ~]# ls -al /var/www/trac/ total 28 drwxr-xr-x 7 apache apache 4096 Dec 17 2010 . drwxr-xr-x 12 root root 4096 Apr 30 2008 .. drwxr-xr-x 10 apache apache 4096 Feb 2 2007 future-exps drwxr-xr-x 11 apache apache 4096 Jan 2 2007 t2b drwxr-xr-x 10 apache apache 4096 Dec 23 2007 t2b-iihe drwxr-xr-x 10 apache apache 4096 Jun 18 2008 vsc drwxr-xr-x 10 apache apache 4096 Dec 17 2010 vsc-hydra
- On laranja, create a new Trac environment for each Wiki :
(we will only give details for t2b Wiki) [root@laranja ~]# trac-admin /var/www/trac/t2b initenv ... Project Name [My Project]> T2B ... Database connection string [sqlite:db/trac.db]> ... Project environment for 'T2B' created. You may now configure the environment by editing the file: /var/www/trac/t2b/conf/trac.ini If you'd like to take this new project environment for a test drive, try running the Trac standalone web server <tt>tracd</tt>: tracd --port 8000 /var/www/trac/t2b Then point your browser to http://localhost:8000/t2b. ...
- On mon, dump the trac database :
[root@mon ~]# sqlite3 /var/www/trac/t2b/db/trac.db ".dump" >> sqlite_trac_t2b.sql
and copy the dump of the db to laranja :
[root@mon ~]# scp sqlite_trac_t2b.sql laranja.wn.iihe.ac.be:~/
- On laranja, create a db from the dump and put it in the right place :
[root@laranja ~]# cat sqlite_trac_t2b.sql | sqlite3 trac.db [root@laranja ~]# mv trac.db /var/www/trac/t2b/db/ mv: overwrite `/var/www/trac/t2b/db/trac.db'? y
and set apache as the owner :
chown -R apache:apache /var/www/trac
- Don't forget to also migrate the attachments :
on mon : [root@mon ~]# scp -r /var/www/trac/t2b/attachments/* laranja.wn.iihe.ac.be:/var/www/trac/t2b/attachments/ on laranja : [root@laranja ~]# chown -R apache:apache /var/www/trac/t2b/attachments/*
- On laranja, upgrade the Trac projects :
[root@laranja ~]# trac-admin /var/www/trac/t2b upgrade [root@laranja ~]# trac-admin /var/www/trac/t2b wiki upgrade
- Change the trac.ini of the project... [yet to be done]
Trac authentication system
On mon, a Perl script was launch everyday to refresh the list of authorized users. This script cannot be run on a "base" machine (you need glite stuffs). I'm still looking for a solution...
Apache migration
RPMs installation
We first tried :
include { 'rpms/web_server' };
but there were too many border effects because this template also installs other RPMs not directly related to httpd (glibc, kernel, etc.). Here is what was finally added to the config/trac :
"/software/packages"=pkg_repl("distcache","1.4.5-14.1","x86_64");
"/software/packages"=pkg_repl("mod_auth_kerb","5.1-3.el5","x86_64");
"/software/packages"=pkg_repl("mod_auth_mysql","3.0.0-3.2.el5_3","x86_64");
"/software/packages"=pkg_repl("mod_auth_pgsql","2.0.3-2.3.1","x86_64");
"/software/packages"=pkg_repl("mod_authz_ldap","0.26-9.el5","x86_64");
"/software/packages"=pkg_repl("mod_nss","1.0.3-8.el5","x86_64");
"/software/packages"=pkg_repl("mod_perl","2.0.4-6.el5","x86_64");
"/software/packages"=pkg_repl("perl-BSD-Resource","1.28-1.fc6.1","x86_64");
"/software/packages"=pkg_repl("mod_python","3.2.8-3.1","x86_64");
"/software/packages"=pkg_repl("mod_ssl","2.2.3-31.sl5.4","x86_64");
"/software/packages"=pkg_repl("mod_dav_svn","1.4.2-4.el5_3.1","x86_64");
"/software/packages"=pkg_repl("php","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-cli","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-common","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-devel","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-ldap","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-mysql","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-odbc","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-pdo","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-pear","1.4.9-6.el5","noarch");
"/software/packages"=pkg_repl("php-pgsql","5.1.6-27.el5","x86_64");
Migration of the configuration
We first did a copy/paste of the /etc/httpd directory :
On master3 : service httpd stop cp -a /etc/httpd /etc/httpd_old On mon : rsync -avz -e "ssh" --delete /etc/httpd/ --exclude logs --exclude modules --exclude run master3:/etc/httpd/
But since we were moving from httpd version 2.0 to 2.2, we had to modify a few things in /etc/httpd/conf/httpd.conf, especially in the LoadModule section (some modules became deprecated). Here is a diff showing the details :
[root@mon ~]# diff old_httpd_config new_httpd_config 1,4c1,10 < LoadModule access_module modules/mod_access.so < LoadModule auth_module modules/mod_auth.so < LoadModule auth_anon_module modules/mod_auth_anon.so < LoadModule auth_dbm_module modules/mod_auth_dbm.so --- > #LoadModule access_module modules/mod_access.so > LoadModule authz_host_module modules/mod_authz_host.so > LoadModule authz_user_module modules/mod_authz_user.so > #LoadModule auth_module modules/mod_auth.so > LoadModule auth_basic_module modules/mod_auth_basic.so > #LoadModule auth_anon_module modules/mod_auth_anon.so > LoadModule authn_anon_module modules/mod_authn_anon.so > LoadModule authn_file_module modules/mod_authn_file.so > #LoadModule auth_dbm_module modules/mod_auth_dbm.so > LoadModule authn_dbm_module modules/mod_authn_dbm.so 7c13,14 < LoadModule auth_ldap_module modules/mod_auth_ldap.so --- > #LoadModule auth_ldap_module modules/mod_auth_ldap.so > LoadModule authnz_ldap_module modules/mod_authnz_ldap.so 28c35 < LoadModule imap_module modules/mod_imap.so --- > #LoadModule imap_module modules/mod_imap.so
MySQL migration
MySQL RPMs installation
Since the new mon will be machine-type "base", we need add this in its profile :
include {'common/mysql/server'};
to get the MySQL RPMs installed.
Databases migration
- Copy the /var/lib/mysql from mon to laranja :
On mon : scp -r /var/lib/mysql laranja.wn.iihe.ac.be:/var/lib/ On laranja : chown -R mysql:mysql /var/lib/mysql service mysqld start
- Use the command mysql_upgrade :
On laranja : mysqlcheck --check-upgrade --all-databases --auto-repair -p (you'll be asked the mysql password for root user...) mysql_fix_privilege_tables --password=... (replace the dots by the real password for root...)
Ganglia migration
Since there is no change of version for Ganglia, we simply have to copy/paste the data :
On mon : rsync -avz -e "ssh" --delete /var/lib/ganglia/rrds/ master3:/var/lib/ganglia/rrds/
elog
It was not installed with an RPM.
OldPnfsFiles
Simply copying the CakePHP project /var/www/html/OldPnfsFiles/ should do the trick but the new mon has a newer version of PHP. This version is more strict on syntax negligence so a few typo's had to be fixed. REMARK: to have the OldPnfsFiles database working also on the new maite the correct files had to be copied.