LocalSubmission - Revision history

Admin at 09:30, 14 February 2022

2022-02-14T09:30:10Z

← Older revision		Revision as of 09:30, 14 February 2022
Line 13:		Line 13:
	=== Procedure ===		=== Procedure ===

	*Log in to an UI		*Log in to an UI (m1, m8, m9)
	ssh ~~mshort~~.iihe.ac.be		ssh m1.iihe.ac.be

	*Make a directory and prepare an executable.		*Make a directory and prepare an executable.
Line 24:		Line 24:

	*Execute the following command to submit the script to the local queue		*Execute the following command to submit the script to the local queue
	qsub ~~-q express~~ -o script.stdout -e script.stderr script.sh		qsub -o script.stdout -e script.stderr script.sh

	*Follow the progress of your job on the UI		*Follow the progress of your job on the UI

ROMAIN ROUGNY rrougny@vub.ac.be: /* Comments */

2018-10-05T14:34:33Z

Comments

← Older revision		Revision as of 14:34, 5 October 2018
Line 36:		Line 36:

	==== Comments ====		==== Comments ====
	*In case you would like to access a root file you should copy it to the '''$TMPDIR''' (=/scratch/jobid.cream02.ac.be/) space on the workernode.		*In case you would like to access a root file you should copy it to the '''$TMPDIR''' (=/scratch/jobid.cream02.ac.be/) space on the workernode unique to each job.
	**/scratch is the native disk of the workernode and is several 100 GBs big.		**/scratch is the native disk of the workernode and is several 100 GBs big.
	**Each job is allotted a working directory that is cleaned automatically at the end of the job. This directory is stored in the variable $TMPDIR		**Each job is allotted a working directory that is cleaned automatically at the end of the job. This directory is stored in the variable $TMPDIR
Line 45:		Line 45:
	If you use the local submission, please notice that you potentially can slow down our site. So please, copy all the files you will use during the job to $TMPDIR to avoid this.		If you use the local submission, please notice that you potentially can slow down our site. So please, copy all the files you will use during the job to $TMPDIR to avoid this.
	dccp dcap://maite.iihe.ac.be/pnfs/iihe/..../MYFILE $TMPDIR/		dccp dcap://maite.iihe.ac.be/pnfs/iihe/..../MYFILE $TMPDIR/


	==== FAQ ====		==== FAQ ====

Rougny Romain rrougny@ua.ac.be: /* How to make your proxy available during batch jobs (for instance to write to /pnfs) */

2018-01-12T19:52:49Z

How to make your proxy available during batch jobs (for instance to write to /pnfs)

← Older revision		Revision as of 19:52, 12 January 2018
Line 69:		Line 69:

	* In your sh script you send to qsub, add the line:		* In your sh script you send to qsub, add the line:
	export X509_USER_PROXY=/user/$USER/~~x509up_~~$(id -u $USER) # Or the name of the proxy you copied before if you changed the name		export X509_USER_PROXY=/user/$USER/x509up_u$(id -u $USER) # Or the name of the proxy you copied before if you changed the name

	* Then technically, to copy a file made in your job in the /scratch area, you just do:		* Then technically, to copy a file made in your job in the /scratch area, you just do:

ROMAIN ROUGNY rrougny@vub.ac.be: /* How to make your proxy available during batch jobs (for instance to write to /pnfs) */

2016-11-23T14:42:10Z

How to make your proxy available during batch jobs (for instance to write to /pnfs)

← Older revision		Revision as of 14:42, 23 November 2016
Line 62:		Line 62:

	* Create a proxy with long validity time:		* Create a proxy with long validity time:
	voms-proxy-init --voms MYEXPERIMENT--valid 192:0		voms-proxy-init --voms MYEXPERIMENT --valid 192:0
			MYEXPERIMENT is one of cms, icecube, solidexperiment.org, beapps, ...

	* Copy it to /user		* Copy it to /user

ROMAIN ROUGNY rrougny@vub.ac.be at 14:36, 23 November 2016

2016-11-23T14:36:34Z

Show changes

ROMAIN ROUGNY rrougny@vub.ac.be at 14:11, 23 November 2016

2016-11-23T14:11:54Z

← Older revision		Revision as of 14:11, 23 November 2016
Line 59:		Line 59:
	</pre>		</pre>

	==== How to make your proxy available during batch jobs ====		==== How to make your proxy available during batch jobs (for instance to write to /pnfs) ====

	* Create a proxy with long validity time:		* Create a proxy with long validity time:

ROMAIN ROUGNY rrougny@vub.ac.be: /* Comments and FAQ */

2016-11-23T14:11:04Z

Comments and FAQ

← Older revision		Revision as of 14:11, 23 November 2016
Line 37:		Line 37:

	=== Comments and FAQ ===		=== Comments and FAQ ===
	*In case you would like to access a root file you should copy it to the /scratch space on the workernode.
			*In case you would like to access a root file you should copy it to the '''$TMPDIR''' (=/scratch/jobid.cream02.ac.be/) space on the workernode.
	**/scratch is the native disk of the workernode and is several 100 GBs big.		**/scratch is the native disk of the workernode and is several 100 GBs big.
	**Each job is allotted a working directory that is cleaned automatically at the end of the job. This directory is ~~store~~ in the variable $TMPDIR		**Each job is allotted a working directory that is cleaned automatically at the end of the job. This directory is stored in the variable $TMPDIR
	**Your procedure should look like this:		**Do not read root files from /user. This directory is not physically located on the workernode, it is mounted from the fileserver. Doing this will put a big load on the fileserver potentially causing the UIs to be slow.
	**copy the necessary root from /localgrid (if you have any) to $TMPDIR
	**Make sure the output of the job is also written to $TMPDIR
	**Copy your output files back to /localgrid
	**Do not read root files from /~~localgrid~~. This directory is not physically located on the workernode, it is mounted from the fileserver. Doing this will put a big load on the fileserver potentially causing the UIs to be slow.

	'''**** IMPORTANT *****'''<br>		'''**** IMPORTANT *****'''<br>
	If you use the local submission, please notice that you potentially can slow down our site. So please, copy all the files you will use during the job to ~~/scratch~~ to avoid this. <br>		If you use the local submission, please notice that you potentially can slow down our site. So please, copy all the files you will use during the job to $TMPDIR to avoid this. <br>
	~~Many thanks, <br>~~
	~~The Admin Team~~

	*How to set CMSSW environment in a batch job?
			==== How to set CMSSW environment in a batch job ====

	Add the following lines to your script :		Add the following lines to your script :
Line 57:		Line 54:
	pwd=$PWD		pwd=$PWD
	source $VO_CMS_SW_DIR/cmsset_default.sh # make scram available		source $VO_CMS_SW_DIR/cmsset_default.sh # make scram available
	cd /~~localgrid~~/<USER ~~NAME>~~/path/to/~~CMSSW_4_1_4~~/src/ # your local CMSSW release		cd /user/$USER/path/to/CMSSW_X_Y_Z/src/ # your local CMSSW release
	eval `scram runtime -sh` # don't use cmsenv, won't work on batch		eval `scram runtime -sh` # don't use cmsenv, won't work on batch
	cd $pwd		cd $pwd
	</pre>		</pre>

	*How to make your proxy available during batch jobs?		==== How to make your proxy available during batch jobs ====

	~~Make sure you have~~ a ~~valid~~ proxy ~~and copy it to some place on /localgrid~~ :		* Create a proxy with long validity time:
	~~<pre>~~
	~~cp $X509_USER_PROXY ~/myProxy~~		voms-proxy-init --valid 192:0
	</~~pre>~~
			* Copy it to /user

	~~note that myProxy needs to be a file.~~		cp $X509_USER_PROXY /user/$USER/

	~~Add~~ the ~~following~~ line ~~to your script~~ :		* In your sh script you send to qsub, add the line:
	~~<pre>~~
	~~export X509_USER_PROXY=~/myProxy~~
	~~</pre>~~

	*How to avoid my short jobs from being blocked in the ~~waiting queue when~~ the ~~site is full ?~~		export X509_USER_PROXY=/user/$USER/x509up_$(id -u $USER) # Or the name of the proxy you copied before if you changed the name

	~~If you intend to submit short jobs~~, ~~then it is wise to specify explicitly~~ to ~~the batch system their estimated maximum walltime. You can do this by adding an option to the qsub command :~~		* Then technically, to copy a file made in your job in the /scratch area, you just do:
	~~<pre>~~
	~~qsub -q localgrid@cream02 -o script.stdout -e script.stderr -l walltime=<HH:MM:SS> script.sh~~
	~~</pre>~~
	~~or by adding the following line at the beginning of~~ your job ~~script :~~
	~~<pre>~~
	~~#PBS -l walltime=<HH:MM:SS>~~
	</~~pre>~~
	~~Proceeding this way~~, ~~your jobs priority will grow faster as time goes by, increasing the chances of being executed first. (The shorter they are, the faster their priority will increase over the time.)~~

	~~But be aware that if your jobs are running longer then the specified maximum walltime, they will be killed by the batch system~~. ~~So, don't hesitate to overestimate a bit this maximum walltime~~.		gfal-copy file://$TMPDIR/MYFILE srm://maite.iihe.ac.be:8443/pnfs/iihe/MY/DIR/MYFILE

	=== Stop your jobs ===		=== Stop your jobs ===

OLIVIER DEVROEDE omdevroe@vub.ac.be: /* Comments and FAQ */

2016-09-13T09:34:06Z

Comments and FAQ

← Older revision		Revision as of 09:34, 13 September 2016
Line 66:		Line 66:
	Make sure you have a valid proxy and copy it to some place on /localgrid :		Make sure you have a valid proxy and copy it to some place on /localgrid :
	<pre>		<pre>
	cp $X509_USER_PROXY ~/~~some/place~~		cp $X509_USER_PROXY ~/myProxy
	</pre>		</pre>

			note that myProxy needs to be a file.

	Add the following line to your script :		Add the following line to your script :
	<pre>		<pre>
	export X509_USER_PROXY=~/~~some/place~~		export X509_USER_PROXY=~/myProxy
	</pre>		</pre>

OLIVIER DEVROEDE omdevroe@vub.ac.be at 09:20, 13 September 2016

2016-09-13T09:20:45Z

← Older revision		Revision as of 09:20, 13 September 2016
Line 66:		Line 66:
	Make sure you have a valid proxy and copy it to some place on /localgrid :		Make sure you have a valid proxy and copy it to some place on /localgrid :
	<pre>		<pre>
	cp $X509_USER_PROXY ~~/localgrid/<USER NAME>~~/some/place		cp $X509_USER_PROXY ~/some/place
	</pre>		</pre>

	Add the following line to your script :		Add the following line to your script :
	<pre>		<pre>
	export X509_USER_PROXY=~~/localgrid/<USER NAME>~~/some/place		export X509_USER_PROXY=~/some/place
	</pre>		</pre>

Romain Rougny: /* Procedure */

2016-04-06T12:20:26Z

Procedure

← Older revision		Revision as of 12:20, 6 April 2016
Line 19:		Line 19:
	</pre>		</pre>
	*Paste following code into script.sh. (see below)		*Paste following code into script.sh. (see below)
	*Due to the setup of the Tier2 the output of the script will be placed on the ''/localgrid'' partition which is mounted on both the UI's on the workernodes. Therefore you need to prepare a directory to make sure the output is stored correctly. The localgrid partition can be used as a sandbox for temporary placing input and output files. Do not store any files there permanently.
	~~<pre>~~
	~~mkdir /localgrid/$USER/directsubmissiontest~~
	~~</pre>~~
	*Execute the following command to submit the script to the local queue		*Execute the following command to submit the script to the local queue
	<pre>		<pre>
Line 33:		Line 29:


	*Your job finished if you don't see it anymore with qstat. You should now be able to find your output files in the directory you've create ~~on localgrid~~		*Your job are finished if you don't see it anymore with qstat. You should now be able to find your output files in the directory you've create:
	<pre>		<pre>
	/~~localgrid~~/$USER/directsubmissiontest/script.stdout		/user/$USER/directsubmissiontest/script.stdout
	/~~localgrid~~/$USER/directsubmissiontest/script.stderr		/user/$USER/directsubmissiontest/script.stderr
	/~~localgrid~~/$USER/directsubmissiontest/		/user/$USER/directsubmissiontest/
	</pre>		</pre>