Cluster Overview - Revision history

Admin: /* Data Storage & Directory Structure */

2025-03-03T08:17:39Z

Data Storage & Directory Structure

← Older revision		Revision as of 08:17, 3 March 2025
Line 83:		Line 83:
	* '''/user [/$USER]''' : this is your home directory. You have an enforced quota there, as it is an expensive storage with redundancy and daily backups (see below).		* '''/user [/$USER]''' : this is your home directory. You have an enforced quota there, as it is an expensive storage with redundancy and daily backups (see below).
	* '''/pnfs [/iihe/MYEXP/store/user/$USER]''' : this is where you can store a large amount of data, and is also [[GridStorageAccess\|grid-accessible]]. If you need more than a few TB, please contact us. There is no backups there, so be careful of what you do !		* '''/pnfs [/iihe/MYEXP/store/user/$USER]''' : this is where you can store a large amount of data, and is also [[GridStorageAccess\|grid-accessible]]. If you need more than a few TB, please contact us. There is no backups there, so be careful of what you do !
			** '''IMPORTANT:''' /pnfs is an immutable file system. This means that once data is written, it cannot be changed anymore. Therefore you should '''not''' put your scripts in this area.
	<br>		<br>
	There are other directories than you might want to take notice of:		There are other directories than you might want to take notice of:

Admin: /* How to Connect */

2024-10-16T11:07:48Z

How to Connect

← Older revision		Revision as of 11:07, 16 October 2024
Line 33:		Line 33:
	ssh -X -o ServerAliveInterval=100 <MYLOGIN>@mshort.iihe.ac.be		ssh -X -o ServerAliveInterval=100 <MYLOGIN>@mshort.iihe.ac.be
	:''Tip: the ''-o ServerAliveInterval=100'' option is used to keep your session alive for a long period of time ! You should not be disconnected during a whole day of work.''		:''Tip: the ''-o ServerAliveInterval=100'' option is used to keep your session alive for a long period of time ! You should not be disconnected during a whole day of work.''
			:''Tip: use aliases to connect easily! eg add to your ''~/.bashrc'' file the following: ''alias mshort='ssh -X -o ServerAliveInterval=100 <MYLOGIN>@mshort.iihe.ac.be'

	If connecting does not work, please follow the help [[Faq_t2b#Debugging_SSH_connection_to_mX_machines:\|here]]. After a successful login, you'll see this message :		If connecting does not work, please follow the help [[Faq_t2b#Debugging_SSH_connection_to_mX_machines:\|here]]. After a successful login, you'll see this message :

Admin: /* How to Connect */

2024-03-28T14:19:42Z

How to Connect

← Older revision		Revision as of 14:19, 28 March 2024
Line 68:		Line 68:
	* The quota used on /user (and /group). Here a hard limit is applied, so if you are at 100%, you will have many problems. Clean your space, and if you really need more contact us.<br><br>		* The quota used on /user (and /group). Here a hard limit is applied, so if you are at 100%, you will have many problems. Clean your space, and if you really need more contact us.<br><br>
	* The cpu time limit imposed per process, as we divided our UIs into 2 groups. Please note '''processes will be killed''' if they go over their CPU-time limit!		* The cpu time limit imposed per process, as we divided our UIs into 2 groups. Please note '''processes will be killed''' if they go over their CPU-time limit!
	:: '''The light task''' UIs <span style='color:red'>(max '''CPU''' time = 20 minutes)</span> : they are used for crab/local job submission, writing code, ~~compiling,~~ debugging ...		:: '''The light task''' UIs <span style='color:red'>(max '''CPU''' time = 20 minutes)</span> : they are used for crab/local job submission, writing code, debugging ...
	::<pre>mshort.iihe.ac.be : ~~m1.iihe.ac.be,~~ m2.iihe.ac.be, m3.iihe.ac.be </pre>		::<pre>mshort.iihe.ac.be : m2.iihe.ac.be, m3.iihe.ac.be </pre>
	:: '''The CPU-intensive''' UIs <span style='color:red'>(max '''CPU''' time = 1 hour)</span> : they are available for CPU-intensive and testing tasks/workflows, although you should prefer using local job submission ...		:: '''The CPU-intensive''' UIs <span style='color:red'>(max '''CPU''' time = 5 hour)</span> : they are available for CPU-intensive and testing tasks/workflows, although you should prefer using local job submission ...
	::<pre>mlong.iihe.ac.be : m4.iihe.ac.be, ~~m5.iihe.ac.be, m6.iihe.ac.be, m7~~.iihe.ac.be</pre>		::<pre>mlong.iihe.ac.be : m0.iihe.ac.be, m1.iihe.ac.be</pre>
	* Information about how heavily this UI is used. If any of them is red (ie above optimal usage), please consider using another UI. Please be mindful of other users and don't start too many processes, especially if the UI is already under charge.		* Information about how heavily this UI is used. If any of them is red (ie above optimal usage), please consider using another UI. Please be mindful of other users and don't start too many processes, especially if the UI is already under charge.
	<br>		<br>

Admin: /* Overview */

2024-03-25T11:19:15Z

Overview

← Older revision		Revision as of 11:19, 25 March 2024
Line 5:		Line 5:
	* The '''User Interfaces (UI)'''		* The '''User Interfaces (UI)'''
	::This is the cluster front-end, to use the cluster, you need to log into those machines		::This is the cluster front-end, to use the cluster, you need to log into those machines
	::::Servers : mshort [ ~~m1,~~ m2 , m3 ] , mlong [ m4, ~~m5, m6 , m7~~ ]		::::Servers : mshort [ m2 , m3 ] , mlong [ m0, m1 ]
	:: The '''File Server''' provides the user home on the UIs. It is a highly efficient & redundant storage node of ~120 TB capacity with regular backups.		:: The '''File Server''' provides the user home on the UIs. It is a highly efficient & redundant storage node of ~120 TB capacity with regular backups.

Admin: /* How to Connect */

2024-03-13T11:29:52Z

How to Connect

← Older revision		Revision as of 11:29, 13 March 2024
Line 30:		Line 30:

	To connect to the cluster, you need to have sent us your public ssh key.		To connect to the cluster, you need to have sent us your public ssh key.
	In a terminal, type the following (adapt <MYLOGIN> accordingly):		In a terminal, type the following (adapt <MYLOGIN> accordingly WITHOUT the brackets <>):
	ssh -X -o ServerAliveInterval=100 <MYLOGIN>@mshort.iihe.ac.be		ssh -X -o ServerAliveInterval=100 <MYLOGIN>@mshort.iihe.ac.be
	:''Tip: the ''-o ServerAliveInterval=100'' option is used to keep your session alive for a long period of time ! You should not be disconnected during a whole day of work.''		:''Tip: the ''-o ServerAliveInterval=100'' option is used to keep your session alive for a long period of time ! You should not be disconnected during a whole day of work.''

Admin: /* How to Connect */

2023-12-01T14:55:54Z

How to Connect

← Older revision		Revision as of 14:55, 1 December 2023
Line 73:		Line 73:
	::<pre>mlong.iihe.ac.be : m4.iihe.ac.be, m5.iihe.ac.be, m6.iihe.ac.be, m7.iihe.ac.be</pre>		::<pre>mlong.iihe.ac.be : m4.iihe.ac.be, m5.iihe.ac.be, m6.iihe.ac.be, m7.iihe.ac.be</pre>
	* Information about how heavily this UI is used. If any of them is red (ie above optimal usage), please consider using another UI. Please be mindful of other users and don't start too many processes, especially if the UI is already under charge.		* Information about how heavily this UI is used. If any of them is red (ie above optimal usage), please consider using another UI. Please be mindful of other users and don't start too many processes, especially if the UI is already under charge.
			<br>
			* Sometimes announcements are printed at the end. Please make sure you read those.
	<br>		<br>

Admin: /* How to Connect */

2023-12-01T14:54:03Z

How to Connect

← Older revision		Revision as of 14:54, 1 December 2023
Line 38:		Line 38:
	<span style='color:green'> (: Welcome to the T2B Cluster :) </span>		<span style='color:green'> (: Welcome to the T2B Cluster :) </span>
	<span style='color:green'> ________________________________</span><br>		<span style='color:green'> ________________________________</span><br>
	<span style='color:green'> The cluster is working properly</span><br~~><br>~~		<span style='color:green'> The cluster is working properly</span><br>
	~~Cluster: <span style='color:purple'>HTCondor</span> \| OS: <span style='color:purple'>CC7</span> \| Accessible via: <span style='color:cyan'>mshort/mlong</span~~>
	___________________________________________________________________________<br>		___________________________________________________________________________<br>
	Mail: <span style='color:blue'> grid_admin@listserv.vub.be</span>		Mail: <span style='color:blue'> grid_admin@listserv.vub.be</span> \| Chat: <span style='color:blue'> https://chat.iihe.ac.be</span>
	~~Wiki~~: <span style='color:blue'> https://~~t2bwiki~~.iihe.ac.be</span>		Wiki: <span style='color:blue'> https://t2bwiki.iihe.ac.be</span> \| Status: <span style='color:blue'>https://status.iihe.ac.be</span>
	~~Chat~~: <span style='color:blue'> https://~~chat~~.iihe.ac.be</span>
	Status: <span style='color:blue'>https://status.iihe.ac.be</span>
	___________________________________________________________________________<br>		___________________________________________________________________________<br>
	[/~~pnfs~~ ] => <span style='color:green'>~~101 GB~~</span> [08/~~12/2021~~]		<span style='color:cyan'>[/user]</span> => 224 / 500 GB <span style='color:green'>[44%]</span> --\|-- <span style='color:cyan'>[/pnfs]</span> => <span style='color:green'>101 GB</span> [01/12/2023]
	[/~~user + /group ]~~ => <span style='color:green'>~~45%~~</span> ~~used (271G left)~~
	___________________________________________________________________________<br>		___________________________________________________________________________<br>
	<span style='color:~~cyan~~'>Welcome on [m7]</span> ! You have <span style='color:~~red~~'>3600s (1 hours)</span> of cpu time per processes.		<span style='color:blue'>Welcome on [m7]</span> ! You have <span style='color:purple'>3600s (1 hours)</span> of cpu time per processes.
	There are <span style='color:green'>2 users</span> here \| Load: <span style='color:red'>7.56 /4 CPUs (189%)</span> \| Mem: <span style='color:~~red~~'>16% used</span><br>		There are <span style='color:green'>2 users</span> here \| Load: <span style='color:red'>7.56 /4 CPUs (189%)</span> \| Mem: <span style='color:green'>16% used</span><br>



	Please observe all the information in this message:		Please observe all the information in this message:
	* The ~~email used for the cluster support (please use this one rather than personal mail, this way everyone on the support team can answer and track the progress.)~~		* The header, telling you the health of the cluster. When there is an issue, the header of the welcome message will transform to:
	* The wiki link, ~~where~~ you ~~should go first to find~~ the ~~information~~
	* The chat link, where you can easily contact us for fast exchanges. IIHE users can use their intranet account, others can just create an account.
	* The status link, where you can see if the cluster ~~has any problems reported. Please make sure you are registered to receive updates~~. ~~Note that when~~ there is an issue, the header of the welcome message will transform to:

	<span style='color:red'> :( Welcome to the T2B Cluster ): </span>		<span style='color:red'> :( Welcome to the T2B Cluster ): </span>
Line 65:		Line 58:
	<span style='color:red'> More details at </span><span style='color:magenta'>status.iihe.ac.be</span>		<span style='color:red'> More details at </span><span style='color:magenta'>status.iihe.ac.be</span>
	<span style='color:red'> (Register to receive updates)</span>		<span style='color:red'> (Register to receive updates)</span>
	~~<br><br>~~		<br>

	* The cluster/batch system type, the OS flavour/version, and the load-balancing address to connect to. <br><br>

			* The email used for the cluster support (please use this one rather than personal mail, this way everyone on the support team can answer and track the progress.)
			* The wiki link, where you should go first to find the information
			* The chat link, where you can easily contact us for fast exchanges. IIHE users can use their intranet account, others can just create an account.
			* The status link, where you can see if the cluster has any problems reported. Please make sure you are registered to receive updates.
			<br>
	* The space used on the mass storage /pnfs, where storing a few TB is no problem. No hard limits are applied, but please contact us if you plan to go over 20 TB!		* The space used on the mass storage /pnfs, where storing a few TB is no problem. No hard limits are applied, but please contact us if you plan to go over 20 TB!
	* The quota used on /user (and /group). Here a hard limit is applied, so if you are at 100%, you will have many problems. Clean your space, and if you really need more contact us.<br><br>		* The quota used on /user (and /group). Here a hard limit is applied, so if you are at 100%, you will have many problems. Clean your space, and if you really need more contact us.<br><br>

Admin: /* Useful links */

2023-12-01T14:40:16Z

Useful links

← Older revision		Revision as of 14:40, 1 December 2023
Line 128:		Line 128:

	== Useful links ==		== Useful links ==
	[http://~~gangliat2b~~.iihe.ac.be/ganglia/ Ganglia Monitoring] : stats on all our servers.<br>		[http://ganglia.iihe.ac.be/ganglia/ Ganglia Monitoring] : stats on all our servers.<br>
	[http://status.iihe.ac.be Cluster Status] : current status of all T2B services. Check here before sending us an email. Please also consider registering to receive T2B issues and be informed when things are resolved.		[http://status.iihe.ac.be Cluster Status] : current status of all T2B services. Check here before sending us an email. Please also consider registering to receive T2B issues and be informed when things are resolved.

Admin at 14:39, 20 March 2023

2023-03-20T14:39:51Z

← Older revision		Revision as of 14:39, 20 March 2023
Line 41:		Line 41:
	Cluster: <span style='color:purple'>HTCondor</span> \| OS: <span style='color:purple'>CC7</span> \| Accessible via: <span style='color:cyan'>mshort/mlong</span>		Cluster: <span style='color:purple'>HTCondor</span> \| OS: <span style='color:purple'>CC7</span> \| Accessible via: <span style='color:cyan'>mshort/mlong</span>
	___________________________________________________________________________<br>		___________________________________________________________________________<br>
	Mail: <span style='color:blue'> grid_admin@listserv.vub~~.ac~~.be</span>		Mail: <span style='color:blue'> grid_admin@listserv.vub.be</span>
	Wiki: <span style='color:blue'> https://t2bwiki.iihe.ac.be</span>		Wiki: <span style='color:blue'> https://t2bwiki.iihe.ac.be</span>
	Chat: <span style='color:blue'> https://chat.iihe.ac.be</span>		Chat: <span style='color:blue'> https://chat.iihe.ac.be</span>

Admin: /* Backup */

2023-02-14T14:33:32Z

Backup

← Older revision		Revision as of 14:33, 14 February 2023
Line 124:		Line 124:

	== Backup ==		== Backup ==
	There are several areas that we regularly back up: '''/user''' , '''/group~~''' , '''/data~~''' , '''/ice3'''.<br>		There are several areas that we regularly back up: '''/user''' , '''/group''' , '''/ice3'''.<br>
	You can find more information on the backup frequency and how to access them [[Backup\|here]].		You can find more information on the backup frequency and how to access them [[Backup\|here]].