Installing dcache pool

/!\ This is obsoleted by this page

On quattor svn

• copy/paste the content of an existing dcache pool profile (e.g. profile_behar053.iihe.ac.be)
• as we want to use private only address
  • make sure the gridftp door is disabled:

variable DOOR_GRIDFTP = false;

• • and do the necessary change (static route in maite) to ensure public FQDN (e.g behar053.iihe.ac.be) is mapped to private FQDN (e.g behar053.wn.iihe.ac.be)
    • suggestion: maybe better than static routes, add rules to /etc/hosts in maite. This should be done via quattor of course.
• in the profile tpl uncomment the line that will make spma failing to allow us to setup the XFS for the pool partitions

"/software/packages"=pkg_del("ganglia");

• make sure add all the corresponding changes (e.g. in databases.tpl, os_version_db.tpl,... ) in both CB5 and CB6
  • suggestion: use search command on svn (example: search on behar053)
• commit, runcheck and prepare for installation (aii-shellfe usual commands)

On the pool

• on the RAID adapter BIOS configure the RAID arrays
  • use stripe size of 128kB
  • At most 12 disks in RAID6
  • enable cache write back feature
  • do this for every RAID array
  • after saving the RAID building process may take hours/days to complete
• restart the box and install (first pass)
• spma will fail because we explicitly introduce an rpm conflict
• copy the content of the variable MAKE_FS_SCRIPT to make_xfs.sh
• adapt the script to the new hardware specs
  • variable sunit=256 for 128kB stripe size
  • variable data_disks=10 for 12 disks in RAID6
• run the script

chmod 755 make_xfs.sh
./make_xfs.sh

Notice it may not work entirely at the first try because of various reasons. Make sure all the steps are correctly executed. 

If not do the necessary changes and commit these changes back to the profile tpl for the record and for further use

• prepare the pool to restart
  • in the profile tpl comment the line blocking spma and commit to svn

#"/software/packages"=pkg_del("ganglia");

• • run the following command to allow quattor to install the rest of the componenents

mv ks-post-install.log  ks-post-install.log.1
reboot

• After it restarts follow the quattor configuration (second pass) with:

tail -f /var/log/ncm-cdispd.log

• When the configuration is finished look at /storage/1/. If it is empty you will have to configure the dcache manually:
  • scp ccqg:./dCacheSetup /opt/d-cache/config/
  • adapt in this file the java version according to the one actually installed

java=/usr/java/jdk1.6.0_26/bin/java

• • edit /opt/d-cache/etc/node_config and paste the following content

ADMINDOOR=no
ADMIN_NODE=maite.iihe.ac.be
DCACHE_BASE_DIR=/opt/d-cache
DCACHE_HOME=/opt/d-cache
DCAP=yes
DIRDOMAIN=no
GPLAZMASERVICE=no
GRIDFTP=no
GSIDCAP=yes
HTTPDOMAIN=no
INFOPROVIDER=no
LMDOMAIN=no
NAMESPACE=pnfs
NODE_TYPE=pool
NUMBER_OF_MOVERS=100
PNFSMANAGER=no
PNFS_INSTALL_DIR=/opt/pnfs
PNFS_OVERWRITE=no
PNFS_ROOT=/pnfs
PNFS_START=yes
POOLMANAGER=no
POOL_PATH=/opt/d-cache/etc
SERVER_ID=iihe
SRM=no
UTILITYDOMAIN=no
XROOTD=no

• • run the install script

/opt/d-cache/install/install.sh

• • edit /opt/d-cache/config/behar053.poollist where behar053 is an example replace accordingly:

[root@behar053 ~]# cat /opt/d-cache/config/behar053.poollist
behar053_1  /storage/1/pool  sticky=allowed recover-space recover-control recover-anyway lfs=precious tag.hostname=behar053
behar053_2  /storage/2/pool  sticky=allowed recover-space recover-control recover-anyway lfs=precious tag.hostname=behar053
behar053_3  /storage/3/pool  sticky=allowed recover-space recover-control recover-anyway lfs=precious tag.hostname=behar053

• • create the pools

/opt/d-cache/bin/dcache pool create 18438g /storage/1/pool
/opt/d-cache/bin/dcache pool create 18438g /storage/2/pool
/opt/d-cache/bin/dcache pool create 14750g /storage/3/pool

  Recompute the size (18438g and 14750g) for 3TB disks and keep a safe margin (few MB smaller that the expected netto formatted size)

  22300 for /1 ;27880 for /2 and /3

• • view the result

[root@behar053 ~]# /opt/d-cache/bin/dcache pool ls
Pool       Domain         LFS          Size   Free Path
behar053_1 behar053Domain precious    18438  18438 /storage/1/pool
behar053_2 behar053Domain precious    18438  18438 /storage/2/pool
behar053_3 behar053Domain precious    14750  14750 /storage/3/pool
Disk space is measured in GiB.

• • add necessary lines in maite in /opt/d-cache/config/PoolManager.conf and restart dcache deamons

/opt/d-cache/bin/dcache restart

• • back on the pool, configure dcache via quattor

ncm-ncd --co dcache

• • configure symlink via quattor

ncm-ncd --co symlink

• • last steps

scp ccqg:./dcache-core /etc/rc.d/init.d/dcache-core
scp ccqg:./dcache-pool /etc/rc.d/init.d/dcache-pool
chkconfig --add dcache-core
chkconfig --add dcache-pool

• • reboot
  • after it restarts follow the configuration

tail -f /var/log/ncm-cdispd.log

• • you may have to stop gridftp it is not yet the case

/opt/d-cache/bin/dcache status
/opt/d-cache/bin/dcache stop
/opt/d-cache/bin/dcache start
/opt/d-cache/bin/dcache stop gridftp
ncm-ncd --co dcache

• • run on maite

ncm-ncd --co dcache

Template:TracNotice