InstallationBEgridClient0
Installation of a Quattor deployment server release 13.1
Aim of this procedure
This procedure describes how to install a Quattor deployment server (also called "BEgrid client") release 13.1. The main services/tools of such a server are : AII, DHCP, HTTPD, RUNCHECK and SINDES.
Hardware needed
4096MB RAM + 40GB hard drive + 4 cores If you want to use a VM under KVM, than choose virtio for NICs and HD. You need two NICs : one in the private, and one in the public network.
OS installation
This procedure is based on SL58. Avoid RH6-like distros (too much troubles with pysvn and Python2.6 and GNUTLS used instead of OpenSSL), unless you are looking for troubles !
During the installation process, choose the following options for the machine type : "Server", "Development tools". And un-select "GNOME".
Basic configuration tasks
Once the OS is installed, do the following :
- choose a nice name, and create entries for the machine in the DNSs
- adapt /etc/sysconfig/network :
NETWORKING=yes NETWORKING_IPV6=no HOSTNAME=<hostname_of_your_choice> DNS1=193.190.247.140 DNS2=193.190.247.71 GATEWAY=193.190.247.65
- configure the NICs by editing the files /etc/sysconfig/network-scripts/ifcfg-eth<x> (x=0,1)
- disable Networkmanager :
chkconfig --level 2345 NetworkManager off
- enable network service :
chkconfig --level 2345 network on
- restart the network :
service network restart
- change hostname in environment with hostname command
- disable SELinux :
setenforce 0 vim /etc/sysconfig/selinux --> SELINUX=disabled
- configure ntp (add the line "server ntp.vub.ac.be" at the end of /etc/ntp.conf)
- disable yum-autoupdate (remove yum.cron from /etc/cron.daily)
- configure the firewall, but don't be too strict at the beginning : during the installation process, it is enough to allow only inputs from the private and public networks !
Packages installation
- Repositories configuration : edit the following repo files in /etc/yum.repo.d/ :
# cat /etc/yum.repos.d/quattor.repo [quattor-13.1] name=quattor13.1 enabled=1 baseurl=http://yum.quattor.org/13.1/ gpgcheck=0 [quattor-components] name=quattor-components enabled=1 baseurl=http://quattor.web.lal.in2p3.fr/packages/ncm-components/ gpgcheck=0 [quattor-externals] name=quattor-externals enabled=1 baseurl=http://quattor.web.lal.in2p3.fr/packages/quattor/externals/ gpgcheck=0 # cat /etc/yum.repos.d/sl5_epel.repo [sl5-epel] name=Scientific Linux 5x - EPEL baseurl=http://quattor.web.lal.in2p3.fr/packages/os/sl5/epel/ enabled=1 gpgcheck=0 # cat /etc/yum.repos.d/sl5_addons.repo [sl5-addons] name=Scientific Linux 5x - addons baseurl=http://quattor.web.lal.in2p3.fr/packages/os/sl5/addons enabled=1 gpgcheck=0 # cat /etc/yum.repos.d/java.repo [java] name=java enabled=1 baseurl=http://quattor.web.lal.in2p3.fr/packages/java/ gpgcheck=0
Also check that dag repo is disabled.
- RPMs installation :
yum install aii-dhcp aii-ks aii-pxelinux aii-server cdb-sync dhcp dnsmasq httpd mod_ssl ncm-lib-blockdevices neon-devel squid subversion subversion-devel tftp-server java
AII configuration
DHCP
- Edit /etc/dhcpd.conf in order to have the following content :
# Use this to enble / disable dynamic dns updates globally.
ddns-update-style ad-hoc;
# write here your network name
shared-network iihe.ac.be {
deny unknown-clients;
not authoritative;
# Write here your domain name
option domain-name "iihe.ac.be";
# Parameters for the installation via PXE using pxelinux
filename "pxelinux.0";
# Uncommnent this line if ISC DHCP ver. 2
# option dhcp-class-identifier "PXEClient";
# Uncommnent this line if ISC DHCP ver. 3
option vendor-class-identifier "PXEClient";
option vendor-encapsulated-options 01:04:00:00:00:00:ff;
# Complete with (at least) the gateway + DNS.
# Hosts entries will be inserted
# automatically by AII in this section
subnet 193.190.247.0 netmask 255.255.255.0 {
option routers 193.190.247.65;
option domain-name-servers 193.190.247.140;
}
subnet 193.190.247.96 netmask 255.255.255.224 {
option routers 193.190.247.113;
option domain-name-servers 193.190.198.10;
}
subnet 192.168.0.0 netmask 255.255.0.0 {
option routers 192.168.10.100;
option domain-name-servers 192.168.10.100;
}
}
- Service starting :
chkconfig --add dhcpd chkconfig --level 345 dhcpd on
TFTP
- Edit /etc/xinetd.d/tftp :
server_args = -s /osinstall/nbp disable = no
- Restart the service :
service xinetd restart
OS base install
- Create and feed the /osinstall sub-directories (OS base installation done during kickstart step) :
mkdir -p /osinstall/nbp mkdir -p /osinstall/ks ln -s /osinstall/ks /var/www/html/ks cp -a /usr/lib/syslinux/pxelinux.0 /osinstall/nbp/ cp -a /usr/share/doc/aii-server-13.1.0/eg/localboot.cfg /osinstall/nbp/pxelinux.cfg/ mkdir /osinstall/nbp/sl630_x86_64 cd /osinstall/nbp/sl630_x86_64 wget http://linuxsoft.cern.ch/scientific/6.3/x86_64/os/images/pxeboot/initrd.img wget http://linuxsoft.cern.ch/scientific/6.3/x86_64/os/images/pxeboot/vmlinuz
Acknowledgement script
- Do the following :
cp -a /usr/sbin/aii-installack.cgi /var/www/cgi-bin/ chmod o+rx /var/www/cgi-bin/aii-installack.cgi
Aii-shellfe configuration
- Create /etc/aii/aii-shellfe.conf with the following content (replace "dhcp55" by the real short name of your Quattor deployment server) :
cdburl = https://dhcp55.iihe.ac.be:444/profiles profile_prefix = profile_ use_fqdn = 1
- Apache must be a sudoers. Add the following lines at the end of /etc/sudoers (replace "dhcp55" by the real short name of your Quattor deployment server) :
apache dhcp55.iihe.ac.be=(ALL) NOPASSWD: /usr/sbin/aii-shellfe apache dhcp55.wn.iihe.ac.be=(ALL) NOPASSWD: /usr/sbin/aii-shellfe
and comment the following line in /etc/sudoers :
Defaults requiretty
Deployment scripts
The "official" way to operate Quattor deployments as well as the SCDB tools needed, and how to install and configure them, is described here is full details : https://trac.lal.in2p3.fr/Quattor/wiki/Download/SCDB#Installationofdeploymentscripts
However, at IIHE, we prefer to work in another way : when the sysadmin wants to deploy some changes he has just committed, he must run the command "runcheck" from within a shell in the Quattor deployment server. This command is in fact a Python script that does the following things :
- checkout of the Quattor templates into a tmp directory;
2. ant-build of the templates; 3. if the the build was successful, the xml profiles are copied to the webserver directory so that they are available for download by the client machines, and 4. the client machines are notified.
The installation and configuration of runcheck is described here :
- Log in on the Quattor deployment server and download and untar the following tarball :
wget http://quattor.begrid.be/begrid/install/cb-v4-client.tar.gz tar xvzf cb-v4-client.tar.gz
- /opt/CB<x> (where <x> is the version number of the Centralized BEgrid Repository) is the usual place to put the software :
cp -a cp /opt/CBx
(Change the value of x according to actual version of the Centralized BEgrid Repository you want to use.)
- Let's now give explanations by sub-directories :
- /opt/CBx/keys : contains the BEgrid CA certificate and a valid user *.p12 file used to connect authenticate against the SVN Quattor repository
- /opt/CBx/subversion: some subversion specific parameters; edit the servers file:
- correct full path to key (.p12 file)
- plain-text password for the key (it does not prompt for the password)
- /opt/CBx/tmp: will contain the checkout and build files.
- /opt/CBx/private: here you can put private files (such as passwords and certificate in the templates. passwd.tpl; pub_key.tpl)
- svncheck does this by simple copy from this directory into cfg/clusters. So keep that structure.
- remove the template cluster given as example, otherwise runcheck will try to build it later ...
- /opt/CBx/private/<clutername-glite-version>/passwd.tpl :
- This file contains the passwords that wil be used for your site.
- You can pick any password you like.
- (Unless certain nodes are not configured with Quattor, in that case they must match whitch the non Quattor nodes).
- /opt/CBx/private/<clutername-glite-version>/local_users.tpl :
- ???
- Not needed for a CE or a WN.
- /opt/CBx/private/<clutername-glite-version>/pub_key.tpl :
- Contains the SSH key that will be used for remote SSH access to the nodes.
- More info on generating a key can be found here:
- /opt/CBx/private/<clutername-glite-version>/<your_machine_fqdn>.tpl :
- This is the place where to put the *.tpl files containing the hostcert.pem and hostkey.pem of your machine.
- Go to this page to know how to generate these private templates.
- /opt/CBx/svncheck: this is the code written by Jean-François Roche :
- in config.conf you can specify most needed parameters :
- svn_repos: point it to the trunk of the centralised-begrid repository
- cluster_regexp: a regexp to build only these clusters matching the regexp
- also adapt parameters in the email section
- Update of the pysvn library :
- runcheck uses pysvn library to access the Quattor SVN;
- the library is stored in /opt/CBx/svncheck/pysvn;
- the already existing library is outdated (built for i686 architecture), it must be updated;
- to build new pysvn :
cd /root/ wget http://pysvn.barrys-emacs.org/source_kits/pysvn-1.6.2.tar.gz tar xvzf pysvn-1.6.2.tar.gz cd pysvn-1.6.2
and follow the instructions in the INSTALL.html file.
- once the build is finished, replace the old library by the new one :
mv /opt/CBx/svncheck/pysvn /opt/CBx/svncheck/pysvn_old mkdir /opt/CBx/svncheck/pysvn cp -a /root/pysvn-1.6.2/Source/pysvn/_* /opt/CBx/svncheck/pysvn/
HTTPD
- Basic configuration of the service :
chkconfig --add httpd chkconfig --level 345 httpd on service httpd start mkdir -p /var/www/https/profiles
- Configuration of the reverse proxy with a cache :
- Check that the modules mod_proxy and mod_cache are installed and loaded (see the httpd configuration).
- Reverse proxy is the only one supported by Quattor: your profiles will point to the RPM repository at quattor.begrid.be, but in fact your local Quattor deployment server will get the RPMs, (in theory optionally) cache them, and provide them to node that is being installed.
- Using a disk cache is preferred to lower the load on the CB and the network (and it should be faster).
- Create the file /etc/httpd/conf.d/cb-cache.conf with the following content :
#
# Reverse Proxy (Added for AII)
#
# Comment this line if modules are already loaded in your default httpd.conf
LoadModule proxy_module modules/mod_proxy.so
<IfModule mod_setenvif.c>
BrowserMatch "rpm/.*" nokeepalive force-response-1.0
BrowserMatch "Python-urllib/.*" nokeepalive force-response-1.0
</IfModule>
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1
</Proxy>
ProxyMaxForwards 15
ProxyReceiveBufferSize 0
ProxyTimeout 300
<Location /begrid/>
ProxyPass http://quattor.begrid.be/begrid/
ProxyPassReverse /
</Location>
<Location /packages/>
ProxyPass http://quattor.web.lal.in2p3.fr/packages/
ProxyPassReverse /
</Location>
<Location /13.1/>
ProxyPass http://yum.quattor.org/13.1/
ProxyPassReverse /
</Location>
#
# Disk Cache (Added for AII)
#
# Comment these lines if modules are already loaded in your default httpd.conf
LoadModule cache_module modules/mod_cache.so
LoadModule disk_cache_module modules/mod_disk_cache.so
## Directory to host the cache
CacheRoot /var/www/cache
## Max size of total cache in kb (obsoleted by Apache 2.2, use htcacheclean instead as explained below)
#CacheSize 15000000
CacheEnable disk /begrid
CacheEnable disk /packages
CacheEnable disk /13.1
## CacheDirLevels*CacheDirLength must be smaller than 20 !!
## don't set this higher than necessary
## following setting will create 64*64=4096 subdirectories
## for all possible hashes 64^22
CacheDirLevels 2
CacheDirLength 1
## in bytes (1GB, should be enough for openoffice)
CacheMaxFileSize 1000000000
CacheMinFileSize 1
## expire after 100 days
CacheDefaultExpire 8640000
CacheMaxExpire 10000000
- Create the cache directory and restart Apache :
mkdir /var/www/cache;chown apache.apache /var/www/cache /etc/init.d/httpd restart
- Since Apache 2.2, the CacheSize directive is not used anymore. So to limit the size of the disk space allocated for caching, you will have to use htcacheclean. For that, create the following cron job in /etc/cron.hourly/htcacheclean-cron.sh :
#!/bin/sh htcacheclean -v -n -p/var/www/cache -l15000000K
SINDES
You will find a presentation of SINDES on this page.
A few things you should now about about the current status of SINDES :
- SINDES is not part of Quattor, it is not maintained any more. Some other solutions (like FreeIPA) are being investigated by developers...
- Some SINDES RPMs are still available in BEgrid repositories. Though quite old now, the latest version available from BEgrid repos is still working with Quattor release 13.1 (the AII hook, aii_sindes, is still there).
- Even if SINDES is not mandatory, it is the only way to secure deployment of machines using SSL.
Now, here is the procedure to install and configure SINDES, and make it working with Quattor 13.1 :
- Installation of RPMs :
- You first need to enable the old BEgrid repos, by creating the file /etc/yum.repos.d/cb-v5-sl5.repo :
[cb-v5] name=CB server - client repo - SL5 baseurl=http://quattor.begrid.be/begrid/install/apt/RPMS.cb-v5_i386_sl5/ enabled = 1 [quattor] name=Quattor repo - SL4 #baseurl=http://quattorsw.web.cern.ch/quattorsw/software/quattor/yum/1.3/i386/RPMS.quattor_sl4 baseurl=http://quattor.begrid.be/begrid/install/apt/RPMS.quattor_i386_sl4/ enabled = 1 [rpmforge] name = Red Hat Enterprise - RPMforge.net - dag #baseurl = http://apt.sw.be/redhat/el5/en//dag #mirrorlist = http://apt.sw.be/redhat/el5/en/mirrors-rpmforge #mirrorlist = ///etc/yum.repos.d/mirrors-rpmforge baseurl = http://quattor.begrid.be/begrid/install/apt/RPMS.dag_i386_el5/ enabled = 1 protect = 0
- Packages installation :
wget http://quattor.begrid.be/begrid/install/apt/RPMS.cb-v5_i386_sl5/cb-client-sindes-0.6.0-sl50.1.noarch.rpm yum localinstall cb-client-sindes-0.6.0-sl50.1.noarch.rpm
The result should be :
Installing: cb-client-sindes noarch 0.6.0-sl50.1 /cb-client-sindes-0.6.0-sl50.1.noarch 7.0 k Installing for dependencies: SINDES-Shell-bin noarch 0.5-34 cb-v5 9.1 k SINDES-ca noarch 0.9.99-2.el5 cb-v5 15 k perl-Config-IniFiles noarch 2.72-2.el5.2 sl5-epel 49 k perl-Crypt-SSLeay x86_64 0.57-3.el5.rfx sl5-addons 96 k perl-Date-Manip noarch 5.56-1.el5.rf sl5-addons 211 k perl-IO-stringy noarch 2.110-5.el5 sl5-epel 70 k perl-IPC-Shareable noarch 0.60-3.el5 sl5-epel 39 k perl-List-MoreUtils x86_64 0.33-5.el5 sl5-epel 75 k perl-Log-Dispatch noarch 2.21-1.el5.rf rpmforge 81 k perl-Log-Dispatch-FileRotate noarch 1.19-1.el5.rf sl5-addons 24 k perl-Log-Log4perl noarch 1.15-1.el5.rf rpmforge 385 k perl-MIME-Lite noarch 3.021-1.el5.rf rpmforge 95 k perl-Mail-Sender noarch 0.8.13-2.el5.1 sl5-epel 53 k perl-Mail-Sendmail noarch 0.79-9.el5.1 sl5-epel 27 k perl-MailTools noarch 2.02-1.el5.rf rpmforge 100 k perl-Params-Validate x86_64 0.91-1.el5.rf sl5-addons 105 k perl-SINDES-GetCertificate noarch 0.9.99-1.el5 cb-v5 19 k perl-SINDES-Shell noarch 0.5-30 cb-v5 21 k perl-SINDES-common noarch 0.5-24 cb-v5 47 k perl-Term-Shell noarch 0.02_cern-1 quattor 32 k perl-TimeDate noarch 1:1.16-5.el5 sl-base 32 k perl-XML-DOM noarch 1.44-2.el5.rf rpmforge 188 k perl-XML-RegExp noarch 0.03-2.el5 sl5-epel 8.2 k
- Disable the old BEgrid repos, by setting enabled=0 everywhere in /etc/yum.repos.d/cb-v5-sl5.repo.
- We also aii_sindes.pm :
wget http://quattor.begrid.be/begrid/Central_BEGrid_Repository/i386_quattor_sl4/aii_sindes-0.2.4-1.noarch.rpm yum --nogpgcheck localinstall aii_sindes-0.2.4-1.noarch.rpm
- And since aii_sindes.pem is expected to be in a directory AII :
mkdir /usr/lib/perl/AII cp -a /usr/lib/perl/NCM/Component/aii_sindes.pm /usr/lib/perl/AII/
and replace this line in /usr/lib/perl/AII/aii_sindes :
package NCM::Component::aii_sindes;
by :
package AII::aii_sindes;
- To configure SINDES, follow the instructions on this page.
- Adapt AII configuration to SINDES, by adding the following lines to /etc/aii/aii-shellfe.conf :
cert_file = /etc/sindes/certs/apache.crt key_file = /etc/sindes/keys/apache.key ca_file = /etc/sindes/certs/ca.crt
- Important requirement on the client side : openssl should be upgraded to at least the same version as on the deployment server, that is 0.9.8e-26. And since SINDES should be working before the SPMA step, so that the machine is able to download its XML profile, the following things must be done :
- Make sure that you have at least version 0.9.8e-26 of openssl in client machines configuration.
- Add openssl package in the list AII_OSINSTALL_BASE_PACKAGES (see the quattor/aii/ks/config template).