HTCFirstSubmissionGuide: Difference between revisions

From T2B Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
Line 1: Line 1:
This is only a quick first draft for you to learn more on how to use HTCondor.
=== First time submitting a job ===
For new users, we recommend following [https://indico.cern.ch/event/936993/contributions/4022073/attachments/2105538/3540926/2020-Koch-User-Tutorial.pdf this presentation], that should give you an idea of how to submit jobs.<br>
Then to practice the basics of job submission on a HTCondor cluster, some exercises are proposed on this [https://en.wikitolearn.org/Course:HTCondor Wiki].


<br>
===== ''T2B Specifics'' =====
=== Status of Machine Migration to the New Cluster ===
- '''File Transfers:'''
* '''User Interfaces (UI)''': m2, m3, m6, m7
: Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''') and copy files yourself within your script.
* '''Worker nodes''': ~10700 slots


<br>
- '''Always adapt requested resources to your job'''
: You need to adapt the resources you request to what you estimate your job will need. Requesting more than what you really need is wasteful and deprive your fellow users from ressources they might require.
: To do so, just add to your submit file the following lines:
<pre>request_cpus = 1
request_memory = 200MB
request_disk = 1GB</pre>


=== File Transfers ===
: Note that for a job, if you need more than 1 cpu / 4GB of memory / 10GB of disk, please be careful and sure of what you are doing.
Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''') and copy files yourself within your script.


<br>
<br>
Line 18: Line 23:
<br>
<br>
=== HTCondor Workshop Presentation ===
=== HTCondor Workshop Presentation ===
Every 6 months, there is an HTCondor workshop. Presentations are usually very helpfull, especially if you want to go into details of HTCondor (API, DAGMan, ...). You can find the agenda of the latest one [https://indico.cern.ch/event/936993/timetable/#20200921.detailed here]
Every 6 months, there is an HTCondor workshop. Presentations are usually very helpful, especially if you want to go into details of HTCondor (API, DAGMan, ...). You can find the agenda of the latest one [https://indico.cern.ch/event/936993/timetable/#20200921.detailed here]
 
For new users, we recommend following [https://indico.cern.ch/event/936993/contributions/4022073/attachments/2105538/3540926/2020-Koch-User-Tutorial.pdf this presentation], that should give you an idea of how to submit jobs.
 
<br>
=== Self-learning material ===
For real beginners that want to learn the basics of job submission on a HTCondor cluster, some exercises are proposed on this [https://en.wikitolearn.org/Course:HTCondor Wiki].

Revision as of 13:24, 14 December 2022

First time submitting a job

For new users, we recommend following this presentation, that should give you an idea of how to submit jobs.
Then to practice the basics of job submission on a HTCondor cluster, some exercises are proposed on this Wiki.

T2B Specifics

- File Transfers:

Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms (should_transfer_files = NO) and copy files yourself within your script.

- Always adapt requested resources to your job

You need to adapt the resources you request to what you estimate your job will need. Requesting more than what you really need is wasteful and deprive your fellow users from ressources they might require.
To do so, just add to your submit file the following lines:
request_cpus = 1
request_memory = 200MB
request_disk = 1GB
Note that for a job, if you need more than 1 cpu / 4GB of memory / 10GB of disk, please be careful and sure of what you are doing.


HTCondor Official Documentation

Have a look at the official User Manual on the HTCondor website.
It is very well done and explains all available features.


HTCondor Workshop Presentation

Every 6 months, there is an HTCondor workshop. Presentations are usually very helpful, especially if you want to go into details of HTCondor (API, DAGMan, ...). You can find the agenda of the latest one here