HTCFirstSubmissionGuide - Revision history

Admin: /* T2B Specifics */

2025-04-10T12:39:43Z

T2B Specifics

← Older revision		Revision as of 12:39, 10 April 2025
Line 7:		Line 7:
	- '''File Transfers:'''		- '''File Transfers:'''
	: Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''') and copy files yourself within your script.		: Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''') and copy files yourself within your script.


			- '''Sending many jobs:'''
			:If you plan on sending > 1k jobs at the same time (not a problem), please add '''max_idle = 200''' to your .sub file.<br>
			:This will make sure only 200 jobs materialize in the queue at one time, not stressing the scheduler machine (even if you send >50k jobs in total).

Admin at 09:58, 18 February 2025

2025-02-18T09:58:24Z

← Older revision		Revision as of 09:58, 18 February 2025
Line 45:		Line 45:
	ssh schedd02.wn.iihe.ac.be		ssh schedd02.wn.iihe.ac.be
	:there you can do the condor_submit_dag commands and it will work. Note than condor_q commands to see the advancements of your DAG jobs can still be performed on the mX machines.		:there you can do the condor_submit_dag commands and it will work. Note than condor_q commands to see the advancements of your DAG jobs can still be performed on the mX machines.

			:'''NB:''' as we are now having 2 OSes, some requirements - ''to put in your .sub file'' - are necessary to ensure your jobs arrive on the right cluster:
			<pre>For EL7 workflows:
			requirements = TARGET.OpSysAndVer == "CentOS7"

			For EL9 workflows:
			requirements = TARGET.OpSysAndVer == "AlmaLinux9"</pre>

	=== What is my job doing ? Is something wrong ? ===		=== What is my job doing ? Is something wrong ? ===

Admin: /* T2B Specifics */

2024-11-05T13:14:29Z

T2B Specifics

← Older revision		Revision as of 13:14, 5 November 2024
Line 39:		Line 39:

	- '''Sending DAG jobs'''		- '''Sending DAG jobs'''
	:For now, sending DAG jobs only works when you are directly on the scheduler, so first connect to any mX machine with the -A (forwarding agent) option:		:For now, sending DAG jobs only works when you are directly on the scheduler, so first add your ssh key to local keyring agent and then connect to any mX machine with the -A (forwarding agent) option:
			ssh-add
	ssh -A mshort.iihe.ac.be		ssh -A mshort.iihe.ac.be
	:then connect to the scheduler:		:then connect to the scheduler:
	ssh schedd02.wn.iihe.ac.be		ssh schedd02.wn.iihe.ac.be
	:there you can do the condor_submit_dag commands and it will work. Note than condor_q commands to see the advancements of your DAG jobs can still be performed on the mX machines.		:there you can do the condor_submit_dag commands and it will work. Note than condor_q commands to see the advancements of your DAG jobs can still be performed on the mX machines.


	=== What is my job doing ? Is something wrong ? ===		=== What is my job doing ? Is something wrong ? ===

Admin: /* First time submitting a job = */

2024-11-05T10:19:06Z

First time submitting a job =

← Older revision		Revision as of 10:19, 5 November 2024
Line 1:		Line 1:
	== First time submitting a job ===		== First time submitting a job ==
	For new users, we recommend following [https://indico.cern.ch/event/936993/contributions/4022073/attachments/2105538/3540926/2020-Koch-User-Tutorial.pdf this presentation], that should give you an idea of how to submit jobs.<br>		For new users, we recommend following [https://indico.cern.ch/event/936993/contributions/4022073/attachments/2105538/3540926/2020-Koch-User-Tutorial.pdf this presentation], that should give you an idea of how to submit jobs.<br>
	Then to practice the basics of job submission on a HTCondor cluster, some exercises are proposed on this [https://en.wikitolearn.org/Course:HTCondor Wiki].		Then to practice the basics of job submission on a HTCondor cluster, some exercises are proposed on this [https://en.wikitolearn.org/Course:HTCondor Wiki].

	=== ''T2B Specifics'' ===		=== ''T2B Specifics'' ===

	- '''File Transfers:'''		- '''File Transfers:'''
	: Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''') and copy files yourself within your script.		: Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''') and copy files yourself within your script.
Line 34:		Line 35:
	:Note that if you want to reserve more than the standard 1 core / 600MB of memory, simply add your request_* [specified just above] like this:		:Note that if you want to reserve more than the standard 1 core / 600MB of memory, simply add your request_* [specified just above] like this:
	condor_submit -interactive request_cpus=2		condor_submit -interactive request_cpus=2



			- '''Sending DAG jobs'''
			:For now, sending DAG jobs only works when you are directly on the scheduler, so first connect to any mX machine with the -A (forwarding agent) option:
			ssh -A mshort.iihe.ac.be
			:then connect to the scheduler:
			ssh schedd02.wn.iihe.ac.be
			:there you can do the condor_submit_dag commands and it will work. Note than condor_q commands to see the advancements of your DAG jobs can still be performed on the mX machines.

Admin at 13:18, 3 January 2024

2024-01-03T13:18:24Z

← Older revision		Revision as of 13:18, 3 January 2024
Line 1:		Line 1:
	=== First time submitting a job ===		== First time submitting a job ===
	For new users, we recommend following [https://indico.cern.ch/event/936993/contributions/4022073/attachments/2105538/3540926/2020-Koch-User-Tutorial.pdf this presentation], that should give you an idea of how to submit jobs.<br>		For new users, we recommend following [https://indico.cern.ch/event/936993/contributions/4022073/attachments/2105538/3540926/2020-Koch-User-Tutorial.pdf this presentation], that should give you an idea of how to submit jobs.<br>
	Then to practice the basics of job submission on a HTCondor cluster, some exercises are proposed on this [https://en.wikitolearn.org/Course:HTCondor Wiki].		Then to practice the basics of job submission on a HTCondor cluster, some exercises are proposed on this [https://en.wikitolearn.org/Course:HTCondor Wiki].

	===== ''T2B Specifics'' =====		=== ''T2B Specifics'' ===
	- '''File Transfers:'''		- '''File Transfers:'''
	: Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''') and copy files yourself within your script.		: Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''') and copy files yourself within your script.


	- '''Always adapt requested resources to your job'''		- '''Always adapt requested resources to your job'''
Line 15:		Line 16:

	: Note that for a job, if you need more than 1 cpu / 4GB of memory / 10GB of disk, please be careful and sure of what you are doing.		: Note that for a job, if you need more than 1 cpu / 4GB of memory / 10GB of disk, please be careful and sure of what you are doing.


			- '''Where am I when I start a job ?'''
			:You should always prefer using $HOME, which in a job equates to a local unique directory in the /scratch of the local disk,eg: /scratch/condor/dir_275928.
			: $TMPDIR = $HOME/tmp, so it is also on the local disk and unique


			- '''Efficient use of local disks on worker nodes'''
			:Note that local disks are now exclusively NVMEs which are much -much- faster than network protocols used in writing to /pnfs or /user.<br>
			:So for repeated reads (like cycling through events with files O(1GB)) it is more efficient to copy file locally first then open it.<br>
			:Same for write, prefer writing locally then copying the file to /pnfs.


			- '''Can I have a shell/interactive job on the batch system ?'''
			:Yes! If you want to make tests, or run things interactively, with dedicated core/memory for you, just run:
			condor_submit -i
			:Note that if you want to reserve more than the standard 1 core / 600MB of memory, simply add your request_* [specified just above] like this:
			condor_submit -interactive request_cpus=2


			=== What is my job doing ? Is something wrong ? ===
			For debugging jobs in queue or what jobs running are doing, please follow our [[HTCondorDebug\|HTCondor Debug]] section

	<br>		<br>

Admin at 13:24, 14 December 2022

2022-12-14T13:24:23Z

← Older revision		Revision as of 13:24, 14 December 2022
Line 1:		Line 1:
	~~This is only~~ a ~~quick first draft for~~ you to ~~learn more~~ on ~~how to use~~ HTCondor.		=== First time submitting a job ===
			For new users, we recommend following [https://indico.cern.ch/event/936993/contributions/4022073/attachments/2105538/3540926/2020-Koch-User-Tutorial.pdf this presentation], that should give you an idea of how to submit jobs.<br>
			Then to practice the basics of job submission on a HTCondor cluster, some exercises are proposed on this [https://en.wikitolearn.org/Course:HTCondor Wiki].

	~~<br>~~		===== ''T2B Specifics'' =====
	=== ~~Status of Machine Migration to the New Cluster~~ ===		- '''File Transfers:'''
	* '''~~User Interfaces (UI)~~''': m2, ~~m3, m6, m7~~		: Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''') and copy files yourself within your script.
	* '''~~Worker nodes~~'''~~: ~10700 slots~~

	<br>		- '''Always adapt requested resources to your job'''
			: You need to adapt the resources you request to what you estimate your job will need. Requesting more than what you really need is wasteful and deprive your fellow users from ressources they might require.
			: To do so, just add to your submit file the following lines:
			<pre>request_cpus = 1
			request_memory = 200MB
			request_disk = 1GB</pre>

	~~=== File Transfers ===~~		: Note that for a job, if you need more than 1 cpu / 4GB of memory / 10GB of disk, please be careful and sure of what you are doing.
	~~Please note~~ that ~~contrary to what is usually shown in documentation and examples~~, ~~we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''')~~ and ~~copy files yourself within your script~~.

	<br>		<br>
Line 18:		Line 23:
	<br>		<br>
	=== HTCondor Workshop Presentation ===		=== HTCondor Workshop Presentation ===
	Every 6 months, there is an HTCondor workshop. Presentations are usually very ~~helpfull~~, especially if you want to go into details of HTCondor (API, DAGMan, ...). You can find the agenda of the latest one [https://indico.cern.ch/event/936993/timetable/#20200921.detailed here]		Every 6 months, there is an HTCondor workshop. Presentations are usually very helpful, especially if you want to go into details of HTCondor (API, DAGMan, ...). You can find the agenda of the latest one [https://indico.cern.ch/event/936993/timetable/#20200921.detailed here]

	For new users, we recommend following [https://indico.cern.ch/event/936993/contributions/4022073/attachments/2105538/3540926/2020-Koch-User-Tutorial.pdf this presentation], that should give you an idea of how to submit jobs.

	~~<br>~~
	~~=== Self-learning material ===~~
	~~For real beginners that want to learn the basics of job submission on a HTCondor cluster, some exercises are proposed on this [https://en.wikitolearn.org/Course:HTCondor Wiki].~~

Admin at 08:48, 8 December 2021

2021-12-08T08:48:07Z

← Older revision		Revision as of 08:48, 8 December 2021
Line 4:		Line 4:
	=== Status of Machine Migration to the New Cluster ===		=== Status of Machine Migration to the New Cluster ===
	* '''User Interfaces (UI)''': m2, m3, m6, m7		* '''User Interfaces (UI)''': m2, m3, m6, m7
	* '''Worker nodes''': ~~~7000~~ slots		* '''Worker nodes''': ~10700 slots

	<br>		<br>

Admin: /* Status of Machine Migration to the New Cluster */

2021-06-03T08:48:42Z

Status of Machine Migration to the New Cluster

← Older revision		Revision as of 08:48, 3 June 2021
Line 3:		Line 3:
	<br>		<br>
	=== Status of Machine Migration to the New Cluster ===		=== Status of Machine Migration to the New Cluster ===
	* '''User Interfaces (UI)''': m2, m3		* '''User Interfaces (UI)''': m2, m3, m6, m7
	* '''Worker nodes''': ~7000 slots		* '''Worker nodes''': ~7000 slots

	<br>		<br>

	=== File Transfers ===		=== File Transfers ===
	Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''') and copy files yourself within your script.		Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''') and copy files yourself within your script.

Admin at 15:53, 12 January 2021

2021-01-12T15:53:00Z

← Older revision		Revision as of 15:53, 12 January 2021
Line 5:		Line 5:
	* '''User Interfaces (UI)''': m2, m3		* '''User Interfaces (UI)''': m2, m3
	* '''Worker nodes''': ~7000 slots		* '''Worker nodes''': ~7000 slots

			<br>
			=== File Transfers ===
			Please note that contrary to what is usually shown in documentation and examples, we recommend not using HTCondor file transfer mechanisms ('''should_transfer_files = NO''') and copy files yourself within your script.

	<br>		<br>

Admin at 12:35, 2 October 2020

2020-10-02T12:35:33Z

← Older revision		Revision as of 12:35, 2 October 2020
Line 16:		Line 16:

	For new users, we recommend following [https://indico.cern.ch/event/936993/contributions/4022073/attachments/2105538/3540926/2020-Koch-User-Tutorial.pdf this presentation], that should give you an idea of how to submit jobs.		For new users, we recommend following [https://indico.cern.ch/event/936993/contributions/4022073/attachments/2105538/3540926/2020-Koch-User-Tutorial.pdf this presentation], that should give you an idea of how to submit jobs.

			<br>
			=== Self-learning material ===
			For real beginners that want to learn the basics of job submission on a HTCondor cluster, some exercises are proposed on this [https://en.wikitolearn.org/Course:HTCondor Wiki].