CrabNewIIHE

CMSSW/Crab Usage at IIHE

The aim of this page is to give you the information you need to use CMSSW and Crab at IIHE. For general information on how to use Crab, we strongly advise you to go to the following link :

https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCrab

In case of troubles with CRAB, before sending a mail to the T2 admins or to the "Crab Feedback" Hypernews mailing list, please read this :

https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCrabFaq

KNOWN ISSUES

• autocompletion with csh doesn't seem to work
  • actually, is seems to work as intended
  • TAB only completes unambigous prefixes
  • use CTRL-D to list all possible matches
• as the old /user is not mounted, you will need to recreate the $HOME/.globus directory and copy your usercert.pem and userkey.pem files.
  • don't forget to check the permissions of the files.

CMSSW environment

• CMSSW enviroment is under variable VO_CMS_SW_DIR is the correct release. (This also means that this variable will not have to be set manually anymore)

source $VO_CMS_SW_DIR/cmsset_default.sh

source $VO_CMS_SW_DIR/cmsset_default.csh

Crab environment

• CRAB is available under /user/cmssoft/crab/latest

source /user/cmssoft/crab/latest/crab.sh

source /user/cmssoft/crab/latest/crab.csh

New becms group

• becms is no longer a VO, but from now on a group in VO CMS.
• you now have a unique account name in CMS. This is your SiteDB account.
  • you also should have an account with that login under the new UIs
  • try m1.iihe.ac.be
  • login is based on rsa keys. Talk to the admins if you can not login.
• Requirements:
  • Have an account in /cms/becms group and be registered in SiteDB
  • Instructions
• This group gives you write access to your personal /store/user path on the storage
  • at least for IIHE
  • everyone else has read access
  • you are not able to write to /pnfs/iihe/becms
• add to crab.cfg

[GRID]
virtual_organization    = cms
group                   = becms

Data Publication

You'll find updated general information here : https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCrabForPublication

However - and this is specific to IIHE's T2, you won't find this information in CERN's CMS Wiki - you must have the following in the [GRID] section :

[GRID]
virtual_organization    = cms
group                   = becms

Having this in your [GRID] section, you will avoid jobs ending with the 60307 error code.

Here is a typical example of a crab.cfg for publication valid for IIHE's T2 :

[CRAB]
jobtype                 = cmssw
scheduler               = glite
use_server              = 1

[CMSSW]
datasetpath             = None
pset                    = pub_example.py
total_number_of_events  = 1000
#events_per_job          =
number_of_jobs          = 10
output_file             = pub_example.root

[USER]
return_data             = 0
check_user_remote_dir   = 0

copy_data               = 1
storage_element         = T2_BE_IIHE

publish_data            = 1
publish_data_name       = pub_example
dbs_url_for_publication = https://cmsdbsprod.cern.ch:8443/cms_dbs_ph_analysis_02_writer/servlet/DBSServlet

ui_working_dir          = pub_example

[GRID]
proxy_server            = myproxy.cern.ch
virtual_organization    = cms
group                   = becms

We heavily insist on the fact that (excerpt from CERN's CMS Wiki) :

"all the parameters related publication have to be added in the configuration file before creation of jobs, even if the publication step is executed after retrieving of job output. "

The crab configuration file MUST NOT be modify between "crab -create" and "crab -publish".

CRAB_2_2_1_pre2 (obsolete)

• This feature is now fully tested, but not yet available in official CRAB release
• available on /user/cmssoft/crab/CRAB_2_2_1_pre2_IIHE (use only CRAB_2_2_1_pre2_IIHE)
• works: create,submit,kill,getoutput,publish
  • does not work: postMortem
• add to crab.cfg

[CRAB]
scheduler = pbs

[USER]
copycommand = cmscp

[PBS]
server = gridce.wn.iihe.ac.be
queue = localgrid

cp_command = cp
rfio_server = /localgrid/<your_login>

use_proxy = 1

middleware = LCG

NB: the direct PBS submission doesn't work if in the crab.cfg you set your user interface working directory variable: ui_working_dir.

CRAB_2_7_7

The release 2_7_7 has been modified to allow direct PBS submission at IIHE. You will find this customized release in /user/cmssoft/crab/CRAB_2_7_7_IIHE. Here is the right way to use it :

• You project must be located in /localgrid/<your_user_name>/.
• You must create a proxy (it will be needed in case of stage out to the SE maite) :

voms-proxy-init --voms cms:/cms/becms

• Set your environment as usual, sourcing the customized CRAB release :

cd /localgrid/<your_user_name>/...blablabla.../CMSSW_X_Y_Z/src/
source $VO_CMS_SW_DIR/cmsset_default.(c)sh
cmsenv
source /user/cmssoft/crab/CRAB_2_7_7_IIHE/crab.(c)sh

• Modify your crab.cfg. It should look like this :

[CRAB]
jobtype                 = cmssw
scheduler               = pbs
use_server              = 0

[CMSSW]
datasetpath = /RelValProdTTbar/JobRobot-MC_3XY_V24_JobRobot-v1/GEN-SIM-DIGI-RECO
pset                    = tutorial.py
events_per_job          = 2
number_of_jobs          = 1
output_file             = outfile.root
ignore_edm_output       = 1

[USER]
copy_data             = 1
storage_element       = T2_BE_IIHE
user_remote_dir       = crabpbstest
publish_data            = 1
publish_data_name       = crabpbstest
dbs_url_for_publication = https://cmsdbsprod.cern.ch:8443/cms_dbs_ph_analysis_02_writer/servlet/DBSServlet

thresholdLevel          = 0
xml_report              = Report.xml

[GRID]
virtual_organization    = cms
group                   = becms

[PBS]
server = cream02.iihe.ac.be
queue = localgrid

• You may now type the usual CRAB commands.

Copy project to /user on UIs

• To copy a working project from msa/beo on the master.iihe.ac.be UIs to the /user disk of the UIs you should do the following steps
  • login on master.iihe.ac.be and go to your working CMSSW project
  • Make a tarbal of the 'src' directory of your project on msa/beo disk. Don't copy other things than the src directory. You might also want to skip the crab working directories (crab_0_xxx_yyy) since they are useless to copy. You have to finish the running crabjobs on the msa/beo disks

cd CMSSW_x_y_z 
tar --exclude '/crab_0' -cvf projectxyz.tar src

• • login on m1.iihe.ac.be
  • make a new project and go to src direcory

 
scramv1 p CMSSW CMSSW_x_y_z
cd CMSSW_x_y_z

• • untar the tarbal

tar -xvf /msax/location/of/projectxyz.tar 

• • compile!
• If you have your code synchroinised with a CVS repository you should do the following
  • login to master
  • Look for the file ~/.cvspass and copy it to a location accessible from the UI, e.g. /msa1/username (*Disclaimer*: this file contains your password, remove it after you did the following steps)
  • modify the permissions of the file so you can copy them onto /user on the UI

chmod +r /msa1/username/.cvspass 

• • login to the UI
  • copy the file .cvspass

cp /msa1/username/.cvspass .

• • modify the permissions

 
chmod 600 .cvspass

• • when you now ask ls -l .cvspass you should see that the permission are -rw-------