OtherSoftware: Difference between revisions
No edit summary |
|||
| (8 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
== Software available from the T2 UI machines == | == Software available from the T2 UI machines == | ||
The T2 has many more software than what is available directly from the M machines. All are provided by cern's centrally managed files system called cvmfs. | The T2 has many more software than what is available directly from the M machines. All are provided by cern's centrally managed files system called cvmfs. | ||
They are divided in a CMS specific area and a more general area aimed at high energy in general. | They are divided in a CMS specific area and a more general area aimed at high energy in general. <br> | ||
The CMS specific area comes with a home brew script that makes it easy for searching specific versions of the software you need. However, the set-up is a lot more complicated than the general purpose area. Therefore we recommend the general purpose area unless you do not find the specific version you need in it. | |||
=== General purpose area === | |||
!!! N.B.: For now, the recommended release is [https://lcginfo.cern.ch/release/102/ '''LCG_102'''] with platform '''x86_64-centos7-gcc11-opt''', with a list of all packages included [https://lcginfo.cern.ch/release_packages/x86_64-centos7-gcc11-opt/102/ here] !!! Just source the softare area like this: | |||
source /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_102 x86_64-centos7-gcc11-opt | |||
Instead of trying to explain in details how the /cvmfs sub-directories are organized, we will limit ourselves in this wiki to give some relevant links about the general purpose area | ====The LCG environments, a group of packages neatly built into a coherent environment ==== | ||
Instead of trying to explain in details how the /cvmfs/sft.cern.ch sub-directories are organized, we will limit ourselves in this wiki to give some relevant links and commands about the general purpose area. | |||
A high-level description of the software packages and how they are grouped in LCG Configurations can be found here: [http://lcginfo.cern.ch lcginfo.cern.ch].<br> | |||
You can search by LCG release, platforms, but more importantly by package name.<br> | |||
Then from the package, you can select the specific version you want, and find which LCG release contains it.<br> | |||
Finally just find a arch-OS-compiler combo that suits you (see below), and voila ! <br> | |||
==== Load a specific software repository ==== | |||
The software packages are grouped in LCG Configurations. In each of these configurations, you will normally find a set of software versions that are compatible with each others. To get the list of LCG Configurations : | |||
The software packages are grouped in LCG Configurations. In each of these configurations, you will normally find set of software that are compatible with each others. To get the list of LCG Configurations : | <pre>/cvmfs/sft.cern.ch/lcg/views/checkSetupViews.sh</pre> | ||
Now, let's say you want to know more about the '''"LCG_102"''' configuration, and get what architecture/OS/compiler are available for it : | |||
<pre>/cvmfs/sft.cern.ch/lcg/views/checkSetupViews.sh LCG_102 | |||
Available <arch-os-complier> for LCG_102 : | |||
aarch64-centos7-gcc11-opt | |||
arm64-mac12-clang131-opt | |||
x86_64-centos7-clang12-dbg | |||
x86_64-centos7-clang12-opt | |||
x86_64-centos7-gcc11-dbg | |||
x86_64-centos7-gcc11-opt | |||
x86_64-centos7-gcc8-dbg | |||
x86_64-centos7-gcc8-opt | |||
x86_64-centos8-gcc11-dbg | |||
x86_64-centos8-gcc11-opt | |||
x86_64-centos9-gcc11-dbg | |||
x86_64-centos9-gcc11-opt | |||
x86_64-mac11-clang120-opt | |||
x86_64-ubuntu2004-gcc9-opt | |||
x86_64-ubuntu2204-gcc11-opt | |||
</pre> | |||
If you are on an Centos 7 UI (like the T2B cluster) and want gcc11 compiler (prefer gcc compiler to others on the cluster unless you know what you are doing), issue the following command : | |||
<pre>source /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_102 x86_64-centos7-gcc11-opt</pre> | |||
Check the version of gcc : | Check the version of gcc : | ||
<pre>gcc -v | |||
gcc version 11.2.0 (GCC)</pre> | |||
At this moment your environment has been changed to use all the versions in this specific software area, just check by issuing '''env'''. | |||
=== GFAL issue when loading a different version of python === | |||
When the software repository you want to use contains a version of python that is higher than the one installed on the cluster, you might encounter a problem with the gfal libraries: | |||
<pre> | |||
'import site' failed; use -v for traceback | |||
Traceback (most recent call last): | |||
File "/usr/bin/gfal-mkdir", line 24, in <module> | |||
from gfal2_util.shell import Gfal2Shell | |||
ImportError: No module named gfal2_util.shell | |||
</pre> | |||
This happens because of a mismatch between the software used and the libraries installed. The easiest solution is to use python 3 and a specific version of the UI (user interface) software like in the following example: | |||
<pre> | |||
source /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_93python3 x86_64-centos7-gcc7-opt | |||
source /cvmfs/grid.cern.ch/emi3ui-latest/etc/profile.d/setup-ui-example.sh | |||
</pre> | |||
<br><br> | |||
=== CMS specific area === | === CMS specific area === | ||
!! Prefer loading an LCG complete environment (see above), this way you are sure your environment and packages are coherent whith each other !! | |||
The T2 hosts many software that are packaged with specific versions of CMSSW but that can be used in a stand alone mode. A tool was designed to easily find what is available and which versions can be used.<br> | The T2 hosts many software that are packaged with specific versions of CMSSW but that can be used in a stand alone mode. A tool was designed to easily find what is available and which versions can be used.<br> | ||
To list all the available software: | To list all the available software: | ||
<pre> | <pre> | ||
/swmgrs/cmss/soft.pl --list | /swmgrs/cmss/soft.pl --list | ||
</pre> | </pre> | ||
Most used software are specific version of gcc that are needed, or a root or geant4 version. To find all the available gcc versions: | |||
<pre> | <pre> | ||
/swmgrs/cmss/soft.pl --versions gcc | /swmgrs/cmss/soft.pl --versions gcc | ||
| Line 61: | Line 87: | ||
Now, to get a given version of gcc (4.6.2 in this example) : | Now, to get a given version of gcc (4.6.2 in this example) : | ||
<pre> | <pre> | ||
/swmgrs/cmss/soft.pl --load gcc/4.6.2 | /swmgrs/cmss/soft.pl --load gcc/4.6.2 | ||
</pre> | </pre> | ||
This prints the full path of the init.sh that you have to source to get the desired version of gcc : | This prints the full path of the init.sh that you have to source to get the desired version of gcc : | ||
<pre> | <pre> | ||
source /cvmfs/cms.cern.ch/slc6_amd64_gcc462/external/gcc/4.6.2/etc/profile.d/init.sh | source /cvmfs/cms.cern.ch/slc6_amd64_gcc462/external/gcc/4.6.2/etc/profile.d/init.sh | ||
| Line 76: | Line 99: | ||
Let's now check the version of gfortran : | Let's now check the version of gfortran : | ||
<pre> | <pre> | ||
gfortran --version | gfortran --version | ||
Latest revision as of 13:35, 17 February 2023
Software available from the T2 UI machines
The T2 has many more software than what is available directly from the M machines. All are provided by cern's centrally managed files system called cvmfs.
They are divided in a CMS specific area and a more general area aimed at high energy in general.
The CMS specific area comes with a home brew script that makes it easy for searching specific versions of the software you need. However, the set-up is a lot more complicated than the general purpose area. Therefore we recommend the general purpose area unless you do not find the specific version you need in it.
General purpose area
!!! N.B.: For now, the recommended release is LCG_102 with platform x86_64-centos7-gcc11-opt, with a list of all packages included here !!! Just source the softare area like this:
source /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_102 x86_64-centos7-gcc11-opt
The LCG environments, a group of packages neatly built into a coherent environment
Instead of trying to explain in details how the /cvmfs/sft.cern.ch sub-directories are organized, we will limit ourselves in this wiki to give some relevant links and commands about the general purpose area.
A high-level description of the software packages and how they are grouped in LCG Configurations can be found here: lcginfo.cern.ch.
You can search by LCG release, platforms, but more importantly by package name.
Then from the package, you can select the specific version you want, and find which LCG release contains it.
Finally just find a arch-OS-compiler combo that suits you (see below), and voila !
Load a specific software repository
The software packages are grouped in LCG Configurations. In each of these configurations, you will normally find a set of software versions that are compatible with each others. To get the list of LCG Configurations :
/cvmfs/sft.cern.ch/lcg/views/checkSetupViews.sh
Now, let's say you want to know more about the "LCG_102" configuration, and get what architecture/OS/compiler are available for it :
/cvmfs/sft.cern.ch/lcg/views/checkSetupViews.sh LCG_102 Available <arch-os-complier> for LCG_102 : aarch64-centos7-gcc11-opt arm64-mac12-clang131-opt x86_64-centos7-clang12-dbg x86_64-centos7-clang12-opt x86_64-centos7-gcc11-dbg x86_64-centos7-gcc11-opt x86_64-centos7-gcc8-dbg x86_64-centos7-gcc8-opt x86_64-centos8-gcc11-dbg x86_64-centos8-gcc11-opt x86_64-centos9-gcc11-dbg x86_64-centos9-gcc11-opt x86_64-mac11-clang120-opt x86_64-ubuntu2004-gcc9-opt x86_64-ubuntu2204-gcc11-opt
If you are on an Centos 7 UI (like the T2B cluster) and want gcc11 compiler (prefer gcc compiler to others on the cluster unless you know what you are doing), issue the following command :
source /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_102 x86_64-centos7-gcc11-opt
Check the version of gcc :
gcc -v gcc version 11.2.0 (GCC)
At this moment your environment has been changed to use all the versions in this specific software area, just check by issuing env.
GFAL issue when loading a different version of python
When the software repository you want to use contains a version of python that is higher than the one installed on the cluster, you might encounter a problem with the gfal libraries:
'import site' failed; use -v for traceback
Traceback (most recent call last):
File "/usr/bin/gfal-mkdir", line 24, in <module>
from gfal2_util.shell import Gfal2Shell
ImportError: No module named gfal2_util.shell
This happens because of a mismatch between the software used and the libraries installed. The easiest solution is to use python 3 and a specific version of the UI (user interface) software like in the following example:
source /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_93python3 x86_64-centos7-gcc7-opt
source /cvmfs/grid.cern.ch/emi3ui-latest/etc/profile.d/setup-ui-example.sh
CMS specific area
!! Prefer loading an LCG complete environment (see above), this way you are sure your environment and packages are coherent whith each other !!
The T2 hosts many software that are packaged with specific versions of CMSSW but that can be used in a stand alone mode. A tool was designed to easily find what is available and which versions can be used.
To list all the available software:
/swmgrs/cmss/soft.pl --list
Most used software are specific version of gcc that are needed, or a root or geant4 version. To find all the available gcc versions:
/swmgrs/cmss/soft.pl --versions gcc
Now, to get a given version of gcc (4.6.2 in this example) :
/swmgrs/cmss/soft.pl --load gcc/4.6.2
This prints the full path of the init.sh that you have to source to get the desired version of gcc :
source /cvmfs/cms.cern.ch/slc6_amd64_gcc462/external/gcc/4.6.2/etc/profile.d/init.sh
Let's now check the version of gfortran :
gfortran --version GNU Fortran (GCC) 4.6.2