SingularityContainers

To make SL7/CC8 flavours available for everyone while the cluster is using another version, we make use of containers.

Prerequisites

Helper scripts are in /group/userscripts/, so make sure either it is in your $PATH variable with:

export PATH=$PATH:/group/userscripts/

or just call the full path:

/group/userscripts/sl7

To test code on the mX UIs

Simply go into an EL7 environment:

sl7 bash

This should give you a prompt, where your SL7 code should work.

Just to convince yourself, you can cross-check the OS release:

cat /etc/redhat-release

To use SL7 inside a cluster job

You can simply ask your script to be run inside the sl7 container:

sl7 /user/$USER/MYSUPERSCRIPT.sh

To send it to the cluster, the job.sh file you now will send to using your .sub file to condor_submit should contain:

/group/userscripts/sl7 /user/$USER/MYSUPERSCRIPT.sh

Please note that you then need, in your script, to go to TMPDIR.
It is HIGHLY recommended to play with this first, by printing path and env variables, to check everything inside the container.

To be more exhaustive, what the script sl7 does is first to add your PATH & LD_LIBRARY_PATH to singularity, and then launch your script using the following line:

singularity exec -B /cvmfs -B /pnfs -B /user -B /scratch /cvmfs/singularity.opensciencegrid.org/opensciencegrid/osgvo-el7:latest MYSCRIPT.sh

where:

• exec: action to do for singularity, here will just execute your command in the container specified
• -B /mountpoint: is used to have the path present in the container if needed.
• /cvmfs/.../cms:rhel7: the path on /cvmfs of the container used.
• MYSCRIPT.sh: script or command to execute

N.B. You can look at more containers inside /cvmfs/singularity.opensciencegrid.org/opensciencegrid/.
Or you can also create your own container, with for instance specific software versions, it is quite simple.
You can use the guide here, or google singularity container in your favorite search engine :)

If you need grid commands

The above containers work perfectly on our cluster. However, they have the drawback of not being updated often. Unfortunately, to be able to work in a grid environment, the CA (certificate authority) files need to be kept up-to-date.
For this reason, CMS also provides some containers. Unfortunately, not all software is inside (python in el9 is lacking atm). But the grid commands do work. For them, run the scrip:

/cvmfs/cms.cern.ch/common/cmssw-el7
or
/cvmfs/cms.cern.ch/common/cmssw-el8

Some more info on these containers can be found here