MonWebServicesMigration

From T2B Wiki
Jump to navigation Jump to search

Goal

For the record, mon is a CentOS 46 Xen virtual machine on top of fs.wn. It is still living in the old CB5 branch of our Quattor SVN. It is mainly used as a Web server, hosting this Trac Wiki.

We want a new mon in CB6, with a more recent OS (SL 55) installed on a KVM virtual machine on top of one of our dom0x.wn virtualization hosts.

Since we will upgrade the OS, we will also have to upgrade all the Web related stuffs (Apache, Trac, MySQL,...).

Planning the migration of our Trac Wikis on a test machine

Installation of the test machine

We used Quattor with the "base" machine-type. Its OS is Scientific Linux SL release 5.5 (Boron). Here are the hardware features of the VM : 2GB RAM, 30GB hard disk, 2 CPUs.

What are the needed RPMs ?

  • First, shutdown the two Quattor daemons, since we will have to change the machine manually.
  • Subversion is already installed and is version : subversion-1.4.2-4.el5_3.1
  • Python is already installed and is version : python-2.4.3-27.el5
  • Install DAG repo :
rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS//rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm
RPM installation of trac and dependencies :
[root@laranja ~]# yum install trac
Installing:
 trac                                                          noarch                                             0.12-2.el5.rf
Installing for dependencies:
 httpd                                                         x86_64                                             2.2.3-43.sl5.3
 mod_python                                                    x86_64                                             3.2.8-3.1
 python-genshi                                                 noarch                                             0.6-1.el5.rf
 python-setuptools                                             noarch                                             0.6c5-2.el5
 python-sqlite2                                                x86_64                                             2.3.3-1.el5.rf

Trac migration of our Wikis

From trac-0.10.4-1.el4.rf on mon to trac-0.12-2.el5.rf on laranja.

  • On mon, get the list of the Wikis to migrate :
[root@mon ~]# ls -al /var/www/trac/
total 28
drwxr-xr-x   7 apache apache 4096 Dec 17  2010 .
drwxr-xr-x  12 root   root   4096 Apr 30  2008 ..
drwxr-xr-x  10 apache apache 4096 Feb  2  2007 future-exps
drwxr-xr-x  11 apache apache 4096 Jan  2  2007 t2b
drwxr-xr-x  10 apache apache 4096 Dec 23  2007 t2b-iihe
drwxr-xr-x  10 apache apache 4096 Jun 18  2008 vsc
drwxr-xr-x  10 apache apache 4096 Dec 17  2010 vsc-hydra
  • On laranja, create a new Trac environment for each Wiki :
(we will only give details for t2b Wiki)

[root@laranja ~]# trac-admin /var/www/trac/t2b initenv
...
Project Name [My Project]> T2B
...
Database connection string [sqlite:db/trac.db]>
...
Project environment for 'T2B' created.

You may now configure the environment by editing the file:

  /var/www/trac/t2b/conf/trac.ini

If you'd like to take this new project environment for a test drive,
try running the Trac standalone web server <tt>tracd</tt>:

  tracd --port 8000 /var/www/trac/t2b

Then point your browser to http://localhost:8000/t2b.
...
  • On mon, dump the trac database :
[root@mon ~]# sqlite3 /var/www/trac/t2b/db/trac.db ".dump" >> sqlite_trac_t2b.sql
and copy the dump of the db to laranja :
[root@mon ~]# scp sqlite_trac_t2b.sql laranja.wn.iihe.ac.be:~/
  • On laranja, create a db from the dump and put it in the right place :
[root@laranja ~]# cat sqlite_trac_t2b.sql | sqlite3  trac.db
[root@laranja ~]# mv trac.db  /var/www/trac/t2b/db/
mv: overwrite `/var/www/trac/t2b/db/trac.db'? y
and set apache as the owner :
chown -R apache:apache /var/www/trac
  • Don't forget to also migrate the attachments :
on mon :
[root@mon ~]# scp -r /var/www/trac/t2b/attachments/* laranja.wn.iihe.ac.be:/var/www/trac/t2b/attachments/
on laranja :
[root@laranja ~]# chown -R apache:apache /var/www/trac/t2b/attachments/*
  • On laranja, upgrade the Trac projects :
[root@laranja ~]# trac-admin /var/www/trac/t2b upgrade
[root@laranja ~]# trac-admin /var/www/trac/t2b wiki upgrade
  • Change the trac.ini of the project... [yet to be done]

Trac authentication system

On mon, a Perl script was launch everyday to refresh the list of authorized users. This script cannot be run on a "base" machine (you need glite stuffs). I'm still looking for a solution...

Apache migration

RPMs installation

We first tried :

include { 'rpms/web_server' };

but there were too many border effects because this template also installs other RPMs not directly related to httpd (glibc, kernel, etc.). Here is what was finally added to the config/trac :

"/software/packages"=pkg_repl("distcache","1.4.5-14.1","x86_64");
"/software/packages"=pkg_repl("mod_auth_kerb","5.1-3.el5","x86_64");
"/software/packages"=pkg_repl("mod_auth_mysql","3.0.0-3.2.el5_3","x86_64");
"/software/packages"=pkg_repl("mod_auth_pgsql","2.0.3-2.3.1","x86_64");
"/software/packages"=pkg_repl("mod_authz_ldap","0.26-9.el5","x86_64");
"/software/packages"=pkg_repl("mod_nss","1.0.3-8.el5","x86_64");
"/software/packages"=pkg_repl("mod_perl","2.0.4-6.el5","x86_64");
"/software/packages"=pkg_repl("perl-BSD-Resource","1.28-1.fc6.1","x86_64");
"/software/packages"=pkg_repl("mod_python","3.2.8-3.1","x86_64");
"/software/packages"=pkg_repl("mod_ssl","2.2.3-31.sl5.4","x86_64");
"/software/packages"=pkg_repl("mod_dav_svn","1.4.2-4.el5_3.1","x86_64");
"/software/packages"=pkg_repl("php","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-cli","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-common","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-devel","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-ldap","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-mysql","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-odbc","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-pdo","5.1.6-27.el5","x86_64");
"/software/packages"=pkg_repl("php-pear","1.4.9-6.el5","noarch");
"/software/packages"=pkg_repl("php-pgsql","5.1.6-27.el5","x86_64");

Migration of the configuration

We first did a copy/paste of the /etc/httpd directory :

On master3 :
service httpd stop
cp -a /etc/httpd /etc/httpd_old
On mon :
rsync -avz -e "ssh" --delete /etc/httpd/ --exclude logs --exclude modules --exclude run master3:/etc/httpd/

But since we were moving from httpd version 2.0 to 2.2, we had to modify a few things in /etc/httpd/conf/httpd.conf, especially in the LoadModule section (some modules became deprecated). Here is a diff showing the details :

[root@mon ~]# diff old_httpd_config new_httpd_config 
1,4c1,10
< LoadModule access_module modules/mod_access.so
< LoadModule auth_module modules/mod_auth.so
< LoadModule auth_anon_module modules/mod_auth_anon.so
< LoadModule auth_dbm_module modules/mod_auth_dbm.so
---
> #LoadModule access_module modules/mod_access.so
> LoadModule authz_host_module modules/mod_authz_host.so
> LoadModule authz_user_module modules/mod_authz_user.so
> #LoadModule auth_module modules/mod_auth.so
> LoadModule auth_basic_module modules/mod_auth_basic.so
> #LoadModule auth_anon_module modules/mod_auth_anon.so
> LoadModule authn_anon_module modules/mod_authn_anon.so
> LoadModule authn_file_module modules/mod_authn_file.so
> #LoadModule auth_dbm_module modules/mod_auth_dbm.so
> LoadModule authn_dbm_module modules/mod_authn_dbm.so
7c13,14
< LoadModule auth_ldap_module modules/mod_auth_ldap.so
---
> #LoadModule auth_ldap_module modules/mod_auth_ldap.so
> LoadModule authnz_ldap_module modules/mod_authnz_ldap.so
28c35
< LoadModule imap_module modules/mod_imap.so
---
> #LoadModule imap_module modules/mod_imap.so

MySQL migration

MySQL RPMs installation

Since the new mon will be machine-type "base", we need add this in its profile :

include {'common/mysql/server'};

to get the MySQL RPMs installed.

Databases migration

  • Copy the /var/lib/mysql from mon to laranja :
On mon :
scp -r /var/lib/mysql laranja.wn.iihe.ac.be:/var/lib/
On laranja :
chown -R mysql:mysql /var/lib/mysql
service mysqld start
  • Use the command mysql_upgrade :
On laranja :
mysqlcheck --check-upgrade --all-databases --auto-repair -p
(you'll be asked the mysql password for root user...)
mysql_fix_privilege_tables --password=...
(replace the dots by the real password for root...)

Ganglia migration

Since there is no change of version for Ganglia, we simply have to copy/paste the data :

On mon :
rsync -avz -e "ssh" --delete /var/lib/ganglia/rrds/ master3:/var/lib/ganglia/rrds/

elog

It was not installed with an RPM.

OldPnfsFiles

Simply copying the CakePHP project /var/www/html/OldPnfsFiles/ should do the trick but the new mon has a newer version of PHP. This version is more strict on syntax negligence so a few typo's had to be fixed. REMARK: to have the OldPnfsFiles database working also on the new maite the correct files had to be copied.


Template:TracNotice