HTCondor: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
| (6 intermediate revisions by the same user not shown) | |||
| Line 4: | Line 4: | ||
Changes in the cluster: | Changes in the cluster: | ||
* '''OS''': EL7 flavor (CentOS7) | * '''OS''': EL7 flavor (CentOS7) | ||
* '''Batch System''': HTCondor | * '''Batch System''': HTCondor v9.0.12 | ||
* '''Computing Element''': HTCondor-ce | * '''Computing Element''': HTCondor-ce | ||
* '''Mass Storage''': /pnfs is now accessible using posix nfs in read/write mode | * '''Mass Storage''': /pnfs is now accessible using posix nfs in read/write mode | ||
* '''User Interfaces (UI)''': mshort (m1, m2, m3), mlong (m4, m5, m6, m7) | |||
| Line 12: | Line 13: | ||
* [[HTCFirstSubmissionGuide|First guide to job submission]] | * [[HTCFirstSubmissionGuide|First guide to job submission]] | ||
* [https://htcondor.readthedocs.io/en/latest/apis/python-bindings/tutorials/index.html Using the python API] | * [https://htcondor.readthedocs.io/en/latest/apis/python-bindings/tutorials/index.html Using the python API] | ||
<br> | |||
'''! Important !'''<br> | |||
'''/user''' and '''/pnfs''' are both accessible everywhere on the cluster.<br> | |||
So there is no need to use HTCondor transfer features when submitting jobs to transfer files to/from any of those 2 directories. Just call/copy them directly from within your jobs. | |||
You have 2 ways to copy files: | |||
* '''should_transfer_files = YES''': [not recommended] that will copy everything not mentionned in your .sub files along with the .out & .err logs. So you need to remove all files you don't want copied before your job ends (like unneeded inputs, intermediary file, etc) | |||
* '''should_transfer_files = NO''': you need to copy files yourself within your job script. You can easilly copy files to /user & /pnfs with a simple cp, or grid tools if you prefer. | |||
Latest revision as of 12:57, 3 January 2024
! Please note that those pages are not intended to be what our Official Documentation will look like, but a rather a quick response to all your enthusiasm in using the new HTCondor cluster !
Changes in the cluster:
- OS: EL7 flavor (CentOS7)
- Batch System: HTCondor v9.0.12
- Computing Element: HTCondor-ce
- Mass Storage: /pnfs is now accessible using posix nfs in read/write mode
- User Interfaces (UI): mshort (m1, m2, m3), mlong (m4, m5, m6, m7)
HTCondor features Documentation:
! Important !
/user and /pnfs are both accessible everywhere on the cluster.
So there is no need to use HTCondor transfer features when submitting jobs to transfer files to/from any of those 2 directories. Just call/copy them directly from within your jobs.
You have 2 ways to copy files:
- should_transfer_files = YES: [not recommended] that will copy everything not mentionned in your .sub files along with the .out & .err logs. So you need to remove all files you don't want copied before your job ends (like unneeded inputs, intermediary file, etc)
- should_transfer_files = NO: you need to copy files yourself within your job script. You can easilly copy files to /user & /pnfs with a simple cp, or grid tools if you prefer.