SingularityContainers: Difference between revisions
(Created page with "To make SL7 available for everyone while the cluster is still in SL6, we make use of containers. <br> === To test code on the mX UIs === Simply go into an EL7 environment: s...") |
No edit summary |
||
| (5 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
To make SL7 available for everyone while the cluster is | To make SL6/SL7/CC8 flavours available for everyone while the cluster is using another version, we make use of containers. This works for SL7 on the olc PBS cluster that is in SL6. This also works for SL6 & SL8 on the new HTCondor CC7 Cluster, just replace SL7 with the version you want. | ||
<br> | |||
=== Prerequisites === | |||
Helper scripts are in '''/group/userscripts/''', so make sure either it is in your $PATH variable with: | |||
export PATH=$PATH:/group/userscripts/ | |||
or just call the full path: | |||
/group/userscripts/sl6 | |||
<br> | <br> | ||
| Line 12: | Line 19: | ||
<br> | <br> | ||
=== To use SL7 inside a cluster job === | === To use SL7 inside a cluster job === | ||
You can simply ask your script to be | You can simply ask your script to be run inside the sl7 container: | ||
sl7 /user/$USER/MYSUPERSCRIPT.sh | sl7 /user/$USER/MYSUPERSCRIPT.sh | ||
To send it to the cluster, the job.sh file you now will send to qsub should contain: | |||
/group/userscripts/sl7 /user/$USER/MYSUPERSCRIPT.sh | |||
Please note that you then need, in your script, to go to TMPDIR. | Please note that you then need, in your script, to go to TMPDIR. | ||
| Line 19: | Line 29: | ||
To be more exhaustive, what the script '''sl7''' does is first to add your PATH & LD_LIBRARY_PATH to singularity, and then launch your script using the following line: | To be more exhaustive, what the script '''sl7''' does is first to add your PATH & LD_LIBRARY_PATH to singularity, and then launch your script using the following line: | ||
singularity exec -B /cvmfs -B /pnfs -B /user -B /scratch /cvmfs/singularity.opensciencegrid.org/ | singularity exec -B /cvmfs -B /pnfs -B /user -B /scratch /cvmfs/singularity.opensciencegrid.org/opensciencegrid/osgvo-el7:latest MYSCRIPT.sh | ||
where: | where: | ||
| Line 27: | Line 37: | ||
* '''MYSCRIPT.sh''': script or command to execute | * '''MYSCRIPT.sh''': script or command to execute | ||
<br>'''N.B.''' | <br>'''N.B.''' You can look at more containers inside '''/cvmfs/singularity.opensciencegrid.org/opensciencegrid/'''. | ||
<br>Or you can also create your own container, with for instance specific software versions, it is quite simple. | |||
<br>You can use the guide [[SingularityContainerCreation|here]], or google ''singularity container'' in your favorite search engine :) | <br>You can use the guide [[SingularityContainerCreation|here]], or google ''singularity container'' in your favorite search engine :) | ||
Latest revision as of 14:51, 3 June 2022
To make SL6/SL7/CC8 flavours available for everyone while the cluster is using another version, we make use of containers. This works for SL7 on the olc PBS cluster that is in SL6. This also works for SL6 & SL8 on the new HTCondor CC7 Cluster, just replace SL7 with the version you want.
Prerequisites
Helper scripts are in /group/userscripts/, so make sure either it is in your $PATH variable with:
export PATH=$PATH:/group/userscripts/
or just call the full path:
/group/userscripts/sl6
To test code on the mX UIs
Simply go into an EL7 environment:
sl7 bash
This should give you a prompt, where your SL7 code should work.
Just to convince yourself, you can cross-check the OS release:
cat /etc/redhat-release
To use SL7 inside a cluster job
You can simply ask your script to be run inside the sl7 container:
sl7 /user/$USER/MYSUPERSCRIPT.sh
To send it to the cluster, the job.sh file you now will send to qsub should contain:
/group/userscripts/sl7 /user/$USER/MYSUPERSCRIPT.sh
Please note that you then need, in your script, to go to TMPDIR.
It is HIGHLY recommended to play with this first, by printing path and env variables, to check everything inside the container.
To be more exhaustive, what the script sl7 does is first to add your PATH & LD_LIBRARY_PATH to singularity, and then launch your script using the following line:
singularity exec -B /cvmfs -B /pnfs -B /user -B /scratch /cvmfs/singularity.opensciencegrid.org/opensciencegrid/osgvo-el7:latest MYSCRIPT.sh
where:
- exec: action to do for singularity, here will just execute your command in the container specified
- -B /mountpoint: is used to have the path present in the container if needed.
- /cvmfs/.../cms:rhel7: the path on /cvmfs of the container used.
- MYSCRIPT.sh: script or command to execute
N.B. You can look at more containers inside /cvmfs/singularity.opensciencegrid.org/opensciencegrid/.
Or you can also create your own container, with for instance specific software versions, it is quite simple.
You can use the guide here, or google singularity container in your favorite search engine :)