HTCondor: Difference between revisions

From T2B Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(One intermediate revision by the same user not shown)
Line 4: Line 4:
Changes in the cluster:
Changes in the cluster:
* '''OS''': EL7 flavor (CentOS7)
* '''OS''': EL7 flavor (CentOS7)
* '''Batch System''': HTCondor v8.9.8
* '''Batch System''': HTCondor v9.0.12
* '''Computing Element''': HTCondor-ce
* '''Computing Element''': HTCondor-ce
* '''Mass Storage''': /pnfs is now accessible using posix nfs in read/write mode
* '''Mass Storage''': /pnfs is now accessible using posix nfs in read/write mode
* '''User Interfaces (UI)''': m2, m3, m6, m7
* '''User Interfaces (UI)''': mshort (m1, m2, m3), mlong (m4, m5, m6, m7)




Line 22: Line 22:
* '''should_transfer_files = YES''': [not recommended] that will copy everything not mentionned in your .sub files along with the .out & .err logs. So you need to remove all files you don't want copied before your job ends (like unneeded inputs, intermediary file, etc)
* '''should_transfer_files = YES''': [not recommended] that will copy everything not mentionned in your .sub files along with the .out & .err logs. So you need to remove all files you don't want copied before your job ends (like unneeded inputs, intermediary file, etc)
* '''should_transfer_files = NO''': you need to copy files yourself within your job script. You can easilly copy files to /user & /pnfs with a simple cp, or grid tools if you prefer.
* '''should_transfer_files = NO''': you need to copy files yourself within your job script. You can easilly copy files to /user & /pnfs with a simple cp, or grid tools if you prefer.
<br>
===== Known Issues =====
* kde.sh issue: does not affect your job
* $HOME: it is not defined (yet), so calling anything with it will fail

Latest revision as of 12:57, 3 January 2024

! Please note that those pages are not intended to be what our Official Documentation will look like, but a rather a quick response to all your enthusiasm in using the new HTCondor cluster !


Changes in the cluster:

  • OS: EL7 flavor (CentOS7)
  • Batch System: HTCondor v9.0.12
  • Computing Element: HTCondor-ce
  • Mass Storage: /pnfs is now accessible using posix nfs in read/write mode
  • User Interfaces (UI): mshort (m1, m2, m3), mlong (m4, m5, m6, m7)


HTCondor features Documentation:


! Important !
/user and /pnfs are both accessible everywhere on the cluster.
So there is no need to use HTCondor transfer features when submitting jobs to transfer files to/from any of those 2 directories. Just call/copy them directly from within your jobs.

You have 2 ways to copy files:

  • should_transfer_files = YES: [not recommended] that will copy everything not mentionned in your .sub files along with the .out & .err logs. So you need to remove all files you don't want copied before your job ends (like unneeded inputs, intermediary file, etc)
  • should_transfer_files = NO: you need to copy files yourself within your job script. You can easilly copy files to /user & /pnfs with a simple cp, or grid tools if you prefer.