HTCondor: Difference between revisions
Jump to navigation
Jump to search
(Created page with "! Please note that those pages are not intended to be what our Official Documentation will look like, but a rather a quick response to all your enthusiasm in using the new HTC...") |
No edit summary |
||
| (11 intermediate revisions by the same user not shown) | |||
| Line 3: | Line 3: | ||
Changes in the cluster: | Changes in the cluster: | ||
* | * '''OS''': EL7 flavor (CentOS7) | ||
* | * '''Batch System''': HTCondor v9.0.12 | ||
* | * '''Computing Element''': HTCondor-ce | ||
* | * '''Mass Storage''': /pnfs is now accessible using posix nfs in read/write mode | ||
* '''User Interfaces (UI)''': mshort (m1, m2, m3), mlong (m4, m5, m6, m7) | |||
HTCondor features Documentation: | HTCondor features Documentation: | ||
* [[HTCFirstSubmissionGuide|First guide to job submission]] | * [[HTCFirstSubmissionGuide|First guide to job submission]] | ||
* [ | * [https://htcondor.readthedocs.io/en/latest/apis/python-bindings/tutorials/index.html Using the python API] | ||
<br> | |||
'''! Important !'''<br> | |||
'''/user''' and '''/pnfs''' are both accessible everywhere on the cluster.<br> | |||
So there is no need to use HTCondor transfer features when submitting jobs to transfer files to/from any of those 2 directories. Just call/copy them directly from within your jobs. | |||
You have 2 ways to copy files: | |||
* '''should_transfer_files = YES''': [not recommended] that will copy everything not mentionned in your .sub files along with the .out & .err logs. So you need to remove all files you don't want copied before your job ends (like unneeded inputs, intermediary file, etc) | |||
* '''should_transfer_files = NO''': you need to copy files yourself within your job script. You can easilly copy files to /user & /pnfs with a simple cp, or grid tools if you prefer. | |||
Latest revision as of 12:57, 3 January 2024
! Please note that those pages are not intended to be what our Official Documentation will look like, but a rather a quick response to all your enthusiasm in using the new HTCondor cluster !
Changes in the cluster:
- OS: EL7 flavor (CentOS7)
- Batch System: HTCondor v9.0.12
- Computing Element: HTCondor-ce
- Mass Storage: /pnfs is now accessible using posix nfs in read/write mode
- User Interfaces (UI): mshort (m1, m2, m3), mlong (m4, m5, m6, m7)
HTCondor features Documentation:
! Important !
/user and /pnfs are both accessible everywhere on the cluster.
So there is no need to use HTCondor transfer features when submitting jobs to transfer files to/from any of those 2 directories. Just call/copy them directly from within your jobs.
You have 2 ways to copy files:
- should_transfer_files = YES: [not recommended] that will copy everything not mentionned in your .sub files along with the .out & .err logs. So you need to remove all files you don't want copied before your job ends (like unneeded inputs, intermediary file, etc)
- should_transfer_files = NO: you need to copy files yourself within your job script. You can easilly copy files to /user & /pnfs with a simple cp, or grid tools if you prefer.