PBS

UI to use

Now only m0, m8 & m9 are still in SL6 and can submit jobs to the old PBS/QSUB qcluster.

Job submission

To submit a job, you just have to use the qsub command :

qsub myjob.sh

OPTIONS

• -q queueName : choose the queue you want [mandatory]

• -N jobName : name of the job

• -I : (capital i) pass in interactive mode

• -m mailaddress : set mail address (use in conjonction with -m) : MUST be @ulb.ac.be or @vub.ac.be

• -m [a|b|e] : send mail on job status change (a = aborted , b = begin, e = end)

• -l : resources options

For instance, if you want to use 2 cores: -lnodes=1:ppn=2

If you want to send more than 2500 jobs to the cluster, write all qsub commands in a text file, and use the script big-submission (more info here).

If you use MadGraph, read this section first or you risk crashing the cluster.

If you want to use the GPUs, please read here.

Job management

To see all jobs (running / queued), you can use the qstat command, or go to the JobView page to have a summary of what's running.

qstat

OPTIONS

• -u username : list only jobs submitted by username

• -n : show nodes where jobs are running

• -q : show the job repartition on queues

Job Statistics

All the log files from the batch system are synced every 30 minutes in:

/group/log/torque/

A simple script to analyze the logs and provide some statistics for the user is provided:

torque-user-info.py

Just execute it as is (it is in your $PATH, so executable from everywhere). It will print information like the following:

ID: 6077555  ExCode:   0 Mem:    0M cpuT:      0s wallT:      3s eff:  0.0%   STDIN
ID: 6077602  ExCode:   0 Mem:   50M cpuT:      0s wallT:      2s eff:  0.0%   STDIN
----------------------------------------------------------------------------------------------------------------------------------------------------------------
   user[G]	# Jobs	 <MEM> +- RMS        #HiMem  MAX Mem      <CPU time>    <walltime>    <Eff>   % WT/WT_TOT    # Jobs with Error code (% of user job)
----------------------------------------------------------------------------------------------------------------------------------------------------------------
    rougny[l]	    12	    13 +- 22    MB      0      52 MB  |  00:00:00:00  00:00:00:24  ( 0.0%) (-1.00% of tot) | # EC: 

If you want to test the batch system, you can follow the workbook here

Job Deletion

Use the following command:

qdel <JOBID>

To delete all your jobs, be patient while using the following line:

for j in $(qselect -u $USER);do timeout 3 qdel -a $j;done