ObsoleteInstructions1

From T2B Wiki
Revision as of 12:28, 26 August 2015 by Maintenance script (talk | contribs) (Created page with " === Obsolete instructions for crabserver deployment === *login to vocms104 and go to the working directory <pre> ssh -X jmmaes@lxplus.cern.ch ssh crab@vocms104 # cd /h...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Obsolete instructions for crabserver deployment

  • login to vocms104 and go to the working directory
ssh -X jmmaes@lxplus.cern.ch
ssh crab@vocms104
# cd /home/crab/WMAgent/WMAgent_CRAB3_server/ # obsolete
cd /data/wmagent/WMAgent_CRAB3_server
  • Follow instructions under section CRABServer specifications here
  • Check out the WMcore code from svn
svn co svn+ssh://jmmaes@svn.cern.ch/reps/CMSDMWM/Infrastructure/trunk/Deployment
  • Start the deployment, but check the wmagent/deploy and crabserver/deploy that you are using the correct version. I'm using 0.7.9a and 3.0.0
cd Deployment/
./Deploy  -s prep -t v300 $PWD/.. crabserver
./Deploy  -s sw -t v300 $PWD/.. crabserver
./Deploy  -s post -t v300 $PWD/.. crabserver
  • Set the environment after making a WMAgent.secrets file
export WMAGENT_SECRETS_LOCATION=/home/crab/WMAgent.secrets
export X509_HOST_CERT=/home/crab/WMAgent/WMAgent_client_CRAB3_test/usercert.pem
export X509_HOST_KEY=/home/crab/WMAgent/WMAgent_client_CRAB3_test/userkey.pem
source /afs/cern.ch/cms/LCG/LCG-2/UI/cms_ui_env.sh

# Claudio settings
#export WMAGENT_SECRETS_LOCATION=/data/wmagent/admin/WMAgent.secrets
#export X509_HOST_CERT=/data/wmagent/admin/certs/usercert.pem
#export X509_HOST_KEY=/data/wmagent/admin/certs/userkey-unencrypted.pem

Instructions

  • login to vocms104 and go to the working directory
ssh -X jmmaes@lxplus.cern.ch 
ssh crab@vocms104
cd /home/crab/WMAgent/WMAgent_client_CRAB3
  • Check out the WMcore code from svn
### not this one #svn co svn+ssh://jmmaes@svn.cern.ch/reps/CMSDMWM/Infrastructure -r HEAD
svn co svn+ssh://jmmaes@svn.cern.ch/reps/CMSDMWM/Infrastructure/branches/deployment-reorg/Deployment
  • Start the deployment
cd Deployment/
./Deploy -t 0.7.5 -s prep /home/crab/WMAgent/WMAgent_client_CRAB3 wmagent
./Deploy -t 0.7.5 -s sw /home/crab/WMAgent/WMAgent_client_CRAB3 wmagent
./Deploy -t 0.7.5 -s post /home/crab/WMAgent/WMAgent_client_CRAB3 wmagent
  • Next step is making a secrets file and copy the usercert and userkey
  • Set the environment
export WMAGENT_SECRETS_LOCATION=/home/crab/WMAgent.secrets
export X509_HOST_CERT=/home/crab/WMAgent/WMAgent_client_CRAB3_test/usercert.pem
export X509_HOST_KEY=/home/crab/WMAgent/WMAgent_client_CRAB3_test/userkey.pem
source /afs/cern.ch/cms/LCG/LCG-2/UI/cms_ui_env.sh
  • go to the current project and start the components
cd ../current/
./config/wmagent/manage activate-agent
./config/wmagent/manage activate-reqmgr
./config/wmagent/manage activate-workqueue
./config/wmagent/manage start-services    # Creates empty databases for all three activated components
./config/wmagent/manage init-agent        # Creates the tables for all three activated components

./config/wmagent/manage start-agent
  • Make the needed modifications in the BossAir plugin
vi config/wmagent/config.py
https://twiki.cern.ch/twiki/bin/viewauth/CMS/WMAgentRunAnalysis
  • Some modifications are made according to this page
  • Additional configuration instructions here


Obsolete setup

  • In this section I'll keep some pointers and some bits and pieces of code.
  • The WMAgent is installed on vocms104
ssh -X jmmaes@lxplus.cern.ch
ssh crab@vocms104
cd /home/crab/WMAgent/WMAgent_client_CRAB3_test
  • CRAB3 documentation can be found here. This page was not so usefull. Since vocms104 is serving as a crab server already some things are in place like the gridFTP
  • Since the link on the previous page was faulty I downloaded the whole infrastructure from here.
svn co svn+ssh://jmmaes@svn.cern.ch/reps/CMSDMWM/Infrastructure -r HEAD
  • The procedure to install the WMAgent I took from here, but it seems to be not working
  • An alternative page to find instructions is here
  • After checking out I deploy the WMAgent:
cd /home/crab/WMAgent/WMAgent_client_CRAB3_test/Infrastructure/trunk/Deployment
./Deploy  -s prep /home/crab/WMAgent/WMAgent_client_CRAB3_test WMAgent
./Deploy  -s sw /home/crab/WMAgent/WMAgent_client_CRAB3_test WMAgent
./Deploy  -s post /home/crab/WMAgent/WMAgent_client_CRAB3_test WMAgent
  • This seems to fail. Alternative try:
cd /home/crab/WMAgent/WMAgent_client_CRAB3_test/Infrastructure/branches/deployment-reorg/Deployment
./Deploy -t 0.7.4 -s prep /home/crab/WMAgent/WMAgent_client_CRAB3_test wmagent
./Deploy -t 0.7.4 -s sw /home/crab/WMAgent/WMAgent_client_CRAB3_test wmagent
./Deploy -t 0.7.4 -s post /home/crab/WMAgent/WMAgent_client_CRAB3_test wmagent
  • Next step is making a secrets file and copy the usercert and userkey
  • Set the environment
export WMAGENT_SECRETS_LOCATION=/home/crab/WMAgent/WMAgent_client_CRAB3_test/WMAgent.secrets
export X509_HOST_CERT=/home/crab/WMAgent/WMAgent_client_CRAB3_test/usercert.pem
export export X509_HOST_KEY=/home/crab/WMAgent/WMAgent_client_CRAB3_test/userkey.pem
  • There is a problem in the start-agent phase due to a missing library. Fix:
svn co svn+ssh://svn.cern.ch/reps/CMSDMWM/WMCore/tags/0.7.3 WMCORE
cp -r WMCORE/src/python/IMProv current/apps/wmcore/lib/python2.6/site-packages/
  • In the following file a change needs to be made:
0.7.4/sw/slc5_amd64_gcc434/cms/wmcore/0.7.4/lib/python2.6/site-packages/PSetTweaks/WMTweak.py
  • change the line
"lfnGroup = lambda j : str(j.get("counter", 0) / 1000).zfill(4) "
  • to
"lfnGroup = lambda j : str(j.get("counter", 0) / 500).zfill(4) "
  • There are also some known issues with the stopping of the agent. It happens that some of the components are not stopped properly and need to be killed by hand.
ps aux | grep -i wmagent #for i in $(ps aux | grep -i wmagent | awk '{print $2}'); do echo "kill -9 $i"; kill -9 $i; done
ps aux | grep python
kill -9 ''pid''
rm <tt>find . -name Daemon.xml</tt>
  • Now I need to adapt the code so that I use the BossAir plugins and GridFTP. There seems to be an issue in restarting the agents/services. The reason is probably due to a bug in 0.7.4, therefore I need to check out 0.7.5
  • This was done by easily repeating the Deploy step with a different version. I've made the same fixes as I did in 0.7.4. To properly start all services I've ran:
./config/wmagent/manage activate-agent
./config/wmagent/manage activate-reqmgr
./config/wmagent/manage activate-workqueue
./config/wmagent/manage start-services    # Creates empty databases for all three activated components
./config/wmagent/manage init-agent        # Creates the tables for all three activated components

./config/wmagent/manage start-agent
  • Before running this I've set up the environment by doing
export SCRAM_ARCH=slc5_amd64_gcc434
source /afs/cern.ch/project/gd/apps/cms/cmsset_default.sh
scram list CMSSW (to see the releases available)
cmsrel CMSSW_4_1_5 (to create the project for this particular release)
cd CMSSW_4_1_5/src
cmsenv


Template:TracNotice

Retrieved from ‘https://t2bwiki.iihe.ac.be/index.php?title=ObsoleteInstructions1&oldid=196’