T2B Wiki - User contributions [en-gb]

BackupT2BCloud

2016-08-30T14:27:14Z

Stephane GERARD: /* Backup of the OpenNebula database */

== Backup of VMs ==
The whole /var/lib/one directory is mounted from the volta fileserver, and regular scheduled snapshots are done automatically to tesla. You can access to these snapshots going through /var/lib/one/.zfs.

== Backup of the OpenNebula database ==
The OpenNebula database being located in /var/lib/mysql, it is not backed up with the ZFS snapshots described in the previous section. That's why we have created a cron task to automatically do a regular dump of the mysql database into the /var/lib/one directory.

To avoid having to specify the user and password in the mysqldump command, we have created the file ~/.my.cnf with the following content :
<pre>
[mysqldump]
user=mysqluser
password=secret
</pre>

Here is the script that does a dump into a file :
<pre>
#!/bin/bash

# Create the backup directory if it doesn't exist
BACKUPDIR=/var/lib/one/one_mysql_db_backups
mkdir -p $BACKUPDIR

# Remove backups older than 7 days
find $BACKUPDIR -name 'one_db_mysqldump*' -mtime +7 -exec rm -f {} \;

# Make a dump of the db to a file in BACKUPDIR
DATE=`date +'%d-%m-%y_%H:%M:%S'`
FILENAME=$BACKUPDIR"/one_db_mysqldump_$DATE"
mysqldump opennebula > $FILENAME
</pre>
Dumps older than 7 days are removed.

Here is the crontab command :
<pre>
* 5,17 * * * /root/one_db_backup.sh > /var/log/one_db_backup.log 2>&1
</pre>

BackupT2BCloud

2016-08-30T14:11:19Z

Stephane GERARD: /* Backup of the OpenNebula database */

BackupT2BCloud

2016-08-30T13:42:04Z

Stephane GERARD: /* Backup of the OpenNebula database */

BackupT2BCloud

2016-08-30T13:35:23Z

Stephane GERARD:

BackupT2BCloud

2016-08-30T13:34:40Z

Stephane GERARD: Created page with "= Backup of VMs = The whole /var/lib/one directory is mounted from the volta fileserver, and regular scheduled snapshots are done automatically to tesla. You can access to the..."

= Backup of VMs =
The whole /var/lib/one directory is mounted from the volta fileserver, and regular scheduled snapshots are done automatically to tesla. You can access to these snapshots going through /var/lib/one/.zfs.

= Backup of the OpenNebula database =
The OpenNebula database being located in /var/lib/mysql, it is not backed up with the ZFS snapshots described in the previous section. That's why we have created a cron task to automatically do a regular dump of the mysql database into the /var/lib/one directory.

AdminPage

2016-08-30T13:25:08Z

Stephane GERARD: /* T2B Cloud */

{{DISPLAYTITLE:Page for Administrators}}
==== Management of the whole cluster ====
*[[elog]]
*[[ShutDownCluster| How to properly switch off the cluster]]
*[[PutClusterOn| How to properly put the cluster on]]
==== CMS Services ====
*[[Phedex]]
*[[Heartbeat]]
*[[LoadTest]]
*[[FroNTier]]
*[[ProdAgent]]
*[[GitForSiteConf| instructions to commit siteconf to git]]
==== Grid Configuration Issues ====
*[[UpdateCertificates| Update the certificates of all our machines]]
*[[CreamIssues| Issues with cream and how to solve them]]
*[[PBS_TMPDIR| PBS TMPDIR]]
*[[APEL| <strike>APEL</strike>(OBSOLETE)]]
*[[BDII]]
*[[FTS]]
*[[SL4_x86_64_WNs| <strike>SL4 x86_64 WNs</strike>(OBSOLETE)]]
*[[CE_oveloaded| CE overloaded]]
*[[RB]]
*[[IPMI]]
*[[CA_certificates| <strike>Upgrade CA certificates</strike> (OBSOLETE)]]
*[[Shutdown| Shutting down the cluster]]
*[[Software_Area_Switch| Software Area Switch]]
*[[KernelUpdate| Kernel mandatory updates for critical vulnerabilities]]
*[[Argus| Argus server and glexec on the workernodes]]
*[[ApelGapPublishing| Apel gap publishing]]
*[[UpdateCACertificates| Update IGTF CA certificates]]

==== Files section ====
*[[DCache| dCache]]
**[[DeleteObsoleteFiles| Find obsolete files]]
**[[DCacheAdminMode| dCache admin mode]]
**[[FindSizeOnDcache| find size of a directory on dcache]]
**[[DcachePoolConfig1912| dCache Pool Postinstallation steps]]
**[[DCacheMaxMovers| Adapt max mover]]
**[[pnfsScripts| scripts to see on what pools files in a directory reside and to move them to other pools]]
*[[OlPnfsFiles| Procedure for removal of old user files on pnfs]]
*[[GetLostFiles| Retrieve lost files from datasets]]
*[[StorageConsistency| Storage Consistency]]

==== Status and Monitoring ====
*[[ReservedWNs| List of reserved WNs]]
*[[Todo| Todo-list]]
*[[Monitoring]]
*[[Plans-Schedule| Plans/Schedule]]
*[[Grid_Troubleshooting_link| Grid Troubleshooting link]]
*[[Incident_reports| Incident Reports]]
*[[Dissapeared_software| How to put the software back]]
*[[Bad_WN| What to do when a WN sends a "bad_wn.pl" email to grid_admin ?]]
*[[Nagios_installation| Nagios Installation at IIHE]]
*[[Restart_DCache| How to restart DCache ]]

==== Info ====
*[[General_info| General info]]
*[[Installing_CMSSW| Installing CMSSW]]
*[[Installing_CRAB| Installing CRAB]]
*[[System_Benchmarks| System Benchmarks]]
*[[T2BTrac| T2B Trac config info]]
*[[HardWare| Hardware information]]
*[[NetworkSetup| Network Setup]]
*[[SetupMonitoringControlerSunfireV20z| Setup Monitoring of LSI Disk Controler on Sunfire V20z Server]]
*[[LDAP_UCL_IIHE| <strike>LDAP authentication system for the replication between UCL and IIHE sites</strike> (OBSOLETE)]]
*[[GridAdminSurvivalGuide| IIHE Grid-admin survival guide]]
*[[Solaris| Solaris 10]]
*[[SolarisSSD| Adding an SSD card and configuring RAID, zpools, filesystems and shares on the new Solaris fileserver]]
*[[LinuxAdminTricks| Linux tricks for admins]]
*[[CrabLocalPbsSubmission| <strike>How to implement local PBS submission with CRAB ?</strike>(OBSOLETE)]]
*[[AddNewUserFromUCLToLDAP| <strike>How to create an account for a CMS user from UCL ?</strike>(OBSOLETE)]]
*[[OSErrata| <strike>Deploying OS errata</strike>(OBSOLETE)]]
*[[BenchmarkHEPSPEC06| Howto benchmark a node with HEPSPEC06]]
*[[Installing_dcache_pool| Install a new dCache pool]]
*[[BackupUsersHomeDirs| Backup of the users home dirs on Jefke]]
*[[MonWebServicesMigration| Migration of mon and its Web services]]
*[[HOWTORestartNagiosTest| HOWTO restart a nagios test manually]]
*[[CompileAndInstallRoot| Compile and install ROOT]]
*[[CleanCreamdb| Clean creamdb]]
*Reboot campaign for the workernodes :
**[[KernelUpdate| Reboot after a kernel update]]
**[[UpgradeWNstoSL5.5| Reboot after an OS upgrade]]
*[[ManageAllAdminScriptsWithGit| Central management of all the admin scripts with Git]]
*[[ConfigProxyCvmfs| Configuration of a proxy for CVMFS]]
**[[RecoverCvmfs| How to recover CVMFS]]
*[[TestNFSPerformance| How to test NFS Performance]]
*[[TetexNotAvailableInSL6| Alternatives to Tetex]]
*[[NewMethodUpdateKernelWorkernodes| A new easy method to update kernel on the workernodes]]
*[[AutomaticMailSendingFromCluster| About automatic mail sending from the cluster]]
*[[T2BTracAccess| T2B Trac access configuration]]
*[[WorkingWithRHEL7| Surviving to RHEL7]]
*[[CCMWithKerberos| Experimental : Securing profiles with Kerberos]]
*[[MigrateToMediaWiki| Migration of T2B Wiki from Trac to MediaWiki]]
*[[motd|Message Of The Day (motd)]]
*[[LToS| Support of Long-tail of Science]]

==== Quattor ====
*[http://quattor.begrid.be/trac/centralised-begrid-v5/wiki/BEgridAndQuattor <strike>BEgrid wiki</strike>(OBSOLETE)]
*[[Test_things| <strike>Test things</strike>(OBSOLETE)]]
*[[Lemon_installation| <strike>Lemon installation</strike>(OBSOLETE)]]
*[[QuattorPointers| <strike>Pointers</strike>]]<strike> to more in-depth information on quattor</strike>(OBSOLETE)
*[[AddingMachineToCluster| <strike>Adding</strike>]]<strike> a new machine to the cluster</strike>(OBSOLETE)
*[[AutomaticMachineTemplateGeneration| Automatic generation of hardware and profile templates for new workernodes]]
*[[InstallationBEgridClient0| Installation of a Quattor deployment server release 13.1]]
*[[InstallFilesNewOS| How to add a new OS to the Quattor Repository]]
*[[GenerateRPMFromATagInGithub| How to build an RPM from a tag in Github]]
*[[HowtoMigrateWNToCB9| How to migrate workernodes from CB8 to CB9]]
*[[WorkingInCB9| Working in CB9 (Quattor release >= 14.2)]]
*[[AideMemoire| FAQ - Aide-mémoire - Howtos]]
*[[BuildANewPysvnOnAiiServer| Howto build a new pysvn on a SL63 AII server]]
*[[QuattorFreeIPA| Quattor and FreeIPA]]
*[[NewRuncheck| Rewrite of the runcheck script in Perl]]
*[[HardDisksManagement| Hard disks management]]
*[[Aquilon| Aquilon]]

==== KVM virtualization ====
*[[VirtWithKVM| Virtualization of the new CREAM-CE on dom02 with KVM]]
*[[VirtWithKVM1| Installation of the new virtualization server dom04]]
*[[CreateVM| Easy creation of virtual machines]]
*[[MonitoringvHostswithGanglia| Monitoring the KVM vHosts with Ganglia]]

==== T2B Cloud ====
*[[MigrationToOpenNebula| Transforming the KVM hypervisors farm into an OpenNebula cloud]]
*[[WorkingInT2BCloud| Working in the T2B cloud]]
*[[MigrateDBMySQL| Migrate one DB from sqlite to mysql]]
*[[BackupT2BCloud| Backup of the T2B Cloud]]

==== gUSE/WS-PGRADE portal ====
*[[PortalInstall| Portal installation]]
*[[PortalConfig| Portal configuration]]
*[[PortalOperations| Portal operations]]

==== Migration to EMI-3 ====
*[[MigrateBEgridToEMI3_part1| BEgrid facilities - Part 1]]
*[[MigrateBEgridToEMI3_part2| BEgrid facilities - Part 2]]

==== XEN ====
*[[Manage_XEN| Manage XEN]]
*[[XenQuattor| Xen and Quattor]]

==== CEPH ====
*[[UnderstandingCeph| Understanding Ceph]]
*[[InstallCephWithQuattor| Installing Ceph with Quattor]]
*[[ExperimentsWithCeph| Experiments with Ceph]]
*[[CephBasics| Operating a Ceph cluster]]

==== Logstash / Elasticsearch / Kibana (ELK) ====
machine: log10 | [http://log10.iihe.ac.be/index.html interface] | [http://log10.iihe.ac.be/HQ index manager]
* [[log_forwarding_with_quattor|Forwarding a log with rsyslog to logstash using quattor]]
* [[log_parsing_with_logstash|Parsing the logs with logstash]]

==== Network ====
* [[network_bond_and_tag|Bonding of 2 interfaces + tagging of 2 vlans on the bond (PRIV+PUB)|]]

AideMemoire

2016-08-02T14:19:58Z

Stephane GERARD: /* How to use ncm-metaconfig ? */

=== How to use ncm-metaconfig ? ===
The official documentation can be found [https://github.com/quattor/configuration-modules-core/blob/master/ncm-metaconfig/src/main/perl/metaconfig.pod here].

Here is an example ready to be included into a machine profile for test purposes :
<pre>
# first, you need to deploy the tt file on the machine
# (the standard way is to include tt files in the the ncm-metaconfig rpm....)
variable CONTENTS = <<EOF;
name = {
[% FILTER indent -%]
hosts = [% hosts.join(',') %]
port = [% port %]
master = [% master ? "TRUE" : "FALSE" %]
description = "[% description %]"
[% IF option.defined -%]
option = "[% option %]"
[% END -%]
[% END -%]
}
EOF
'/software/components/filecopy/services' = npush(
escape('/usr/share/templates/quattor/metaconfig/example/main.tt'), nlist('config', CONTENTS, 'perms', '0644')
);

# below, the real metaconfig work
include 'components/metaconfig/config';
include 'metaconfig/example/config';
prefix '/software/components/metaconfig/services/{/etc/example/exampled.conf}/contents';
'hosts' = list('server1', 'server3');
'port' = 800;
'master' = false;
'description' = 'My example';

# the tt file must be created before ncm-metaconfig runs
'/software/components/metaconfig/dependencies/pre' = push('filecopy');
</pre>

For this example to work, you need a directory metaconfig/example in your site, with the following content :

* config.pan :
<pre>
unique template metaconfig/example/config;

include {'metaconfig/example/schema'};

bind "/software/components/metaconfig/services/{/etc/example/exampled.conf}/contents" = example_service;

prefix "/software/components/metaconfig/services/{/etc/example/exampled.conf}";
"daemon" = list("exampled");
"module" = "example/main";
</pre>

* schema.pan :
<pre>
declaration template metaconfig/example/schema;

include { 'pan/types' };

type example_service = {
'hosts' : type_hostname[]
'port' : type_port
'master' : boolean
'description' : string
'option' ? string
};
</pre>

{{TracNotice|{{PAGENAME}}}}

AideMemoire

2016-08-02T14:11:45Z

Stephane GERARD: /* How to use ncm-metaconfig ? */

MigrateDBMySQL

2016-06-22T15:21:06Z

Stephane GERARD: Created page with "== Convert an OpenNebula DB from SQLite to MySQL == *The recipe described here [http://vadikgo.tumblr.com/post/34325489321/convert-an-opennebula-db-from-sqlite-to-mysql] was t..."

== Convert an OpenNebula DB from SQLite to MySQL ==
*The recipe described here [http://vadikgo.tumblr.com/post/34325489321/convert-an-opennebula-db-from-sqlite-to-mysql] was tested on the VSC-VUB cloud (sl6x, opennebula 4.12), and it works.

AdminPage

2016-06-22T15:17:52Z

Stephane GERARD: /* T2B Cloud */

{{DISPLAYTITLE:Page for Administrators}}
==== Management of the whole cluster ====
*[[elog]]
*[[ShutDownCluster| How to properly switch off the cluster]]
*[[PutClusterOn| How to properly put the cluster on]]
==== CMS Services ====
*[[Phedex]]
*[[Heartbeat]]
*[[LoadTest]]
*[[FroNTier]]
*[[ProdAgent]]
*[[GitForSiteConf| instructions to commit siteconf to git]]
==== Grid Configuration Issues ====
*[[UpdateCertificates| Update the certificates of all our machines]]
*[[CreamIssues| Issues with cream and how to solve them]]
*[[PBS_TMPDIR| PBS TMPDIR]]
*[[APEL| <strike>APEL</strike>(OBSOLETE)]]
*[[BDII]]
*[[FTS]]
*[[SL4_x86_64_WNs| <strike>SL4 x86_64 WNs</strike>(OBSOLETE)]]
*[[CE_oveloaded| CE overloaded]]
*[[RB]]
*[[IPMI]]
*[[CA_certificates| <strike>Upgrade CA certificates</strike> (OBSOLETE)]]
*[[Shutdown| Shutting down the cluster]]
*[[Software_Area_Switch| Software Area Switch]]
*[[KernelUpdate| Kernel mandatory updates for critical vulnerabilities]]
*[[Argus| Argus server and glexec on the workernodes]]
*[[ApelGapPublishing| Apel gap publishing]]
*[[UpdateCACertificates| Update IGTF CA certificates]]

==== Files section ====
*[[DCache| dCache]]
**[[DeleteObsoleteFiles| Find obsolete files]]
**[[DCacheAdminMode| dCache admin mode]]
**[[FindSizeOnDcache| find size of a directory on dcache]]
**[[DcachePoolConfig1912| dCache Pool Postinstallation steps]]
**[[DCacheMaxMovers| Adapt max mover]]
**[[pnfsScripts| scripts to see on what pools files in a directory reside and to move them to other pools]]
*[[OlPnfsFiles| Procedure for removal of old user files on pnfs]]
*[[GetLostFiles| Retrieve lost files from datasets]]
*[[StorageConsistency| Storage Consistency]]

==== Status and Monitoring ====
*[[ReservedWNs| List of reserved WNs]]
*[[Todo| Todo-list]]
*[[Monitoring]]
*[[Plans-Schedule| Plans/Schedule]]
*[[Grid_Troubleshooting_link| Grid Troubleshooting link]]
*[[Incident_reports| Incident Reports]]
*[[Dissapeared_software| How to put the software back]]
*[[Bad_WN| What to do when a WN sends a "bad_wn.pl" email to grid_admin ?]]
*[[Nagios_installation| Nagios Installation at IIHE]]
*[[Restart_DCache| How to restart DCache ]]

==== Info ====
*[[General_info| General info]]
*[[Installing_CMSSW| Installing CMSSW]]
*[[Installing_CRAB| Installing CRAB]]
*[[System_Benchmarks| System Benchmarks]]
*[[T2BTrac| T2B Trac config info]]
*[[HardWare| Hardware information]]
*[[NetworkSetup| Network Setup]]
*[[SetupMonitoringControlerSunfireV20z| Setup Monitoring of LSI Disk Controler on Sunfire V20z Server]]
*[[LDAP_UCL_IIHE| <strike>LDAP authentication system for the replication between UCL and IIHE sites</strike> (OBSOLETE)]]
*[[GridAdminSurvivalGuide| IIHE Grid-admin survival guide]]
*[[Solaris| Solaris 10]]
*[[SolarisSSD| Adding an SSD card and configuring RAID, zpools, filesystems and shares on the new Solaris fileserver]]
*[[LinuxAdminTricks| Linux tricks for admins]]
*[[CrabLocalPbsSubmission| <strike>How to implement local PBS submission with CRAB ?</strike>(OBSOLETE)]]
*[[AddNewUserFromUCLToLDAP| <strike>How to create an account for a CMS user from UCL ?</strike>(OBSOLETE)]]
*[[OSErrata| <strike>Deploying OS errata</strike>(OBSOLETE)]]
*[[BenchmarkHEPSPEC06| Howto benchmark a node with HEPSPEC06]]
*[[Installing_dcache_pool| Install a new dCache pool]]
*[[BackupUsersHomeDirs| Backup of the users home dirs on Jefke]]
*[[MonWebServicesMigration| Migration of mon and its Web services]]
*[[HOWTORestartNagiosTest| HOWTO restart a nagios test manually]]
*[[CompileAndInstallRoot| Compile and install ROOT]]
*[[CleanCreamdb| Clean creamdb]]
*Reboot campaign for the workernodes :
**[[KernelUpdate| Reboot after a kernel update]]
**[[UpgradeWNstoSL5.5| Reboot after an OS upgrade]]
*[[ManageAllAdminScriptsWithGit| Central management of all the admin scripts with Git]]
*[[ConfigProxyCvmfs| Configuration of a proxy for CVMFS]]
**[[RecoverCvmfs| How to recover CVMFS]]
*[[TestNFSPerformance| How to test NFS Performance]]
*[[TetexNotAvailableInSL6| Alternatives to Tetex]]
*[[NewMethodUpdateKernelWorkernodes| A new easy method to update kernel on the workernodes]]
*[[AutomaticMailSendingFromCluster| About automatic mail sending from the cluster]]
*[[T2BTracAccess| T2B Trac access configuration]]
*[[WorkingWithRHEL7| Surviving to RHEL7]]
*[[CCMWithKerberos| Experimental : Securing profiles with Kerberos]]
*[[MigrateToMediaWiki| Migration of T2B Wiki from Trac to MediaWiki]]
*[[motd|Message Of The Day (motd)]]
*[[LToS| Support of Long-tail of Science]]

==== Quattor ====
*[http://quattor.begrid.be/trac/centralised-begrid-v5/wiki/BEgridAndQuattor <strike>BEgrid wiki</strike>(OBSOLETE)]
*[[Test_things| <strike>Test things</strike>(OBSOLETE)]]
*[[Lemon_installation| <strike>Lemon installation</strike>(OBSOLETE)]]
*[[QuattorPointers| <strike>Pointers</strike>]]<strike> to more in-depth information on quattor</strike>(OBSOLETE)
*[[AddingMachineToCluster| <strike>Adding</strike>]]<strike> a new machine to the cluster</strike>(OBSOLETE)
*[[AutomaticMachineTemplateGeneration| Automatic generation of hardware and profile templates for new workernodes]]
*[[InstallationBEgridClient0| Installation of a Quattor deployment server release 13.1]]
*[[InstallFilesNewOS| How to add a new OS to the Quattor Repository]]
*[[GenerateRPMFromATagInGithub| How to build an RPM from a tag in Github]]
*[[HowtoMigrateWNToCB9| How to migrate workernodes from CB8 to CB9]]
*[[WorkingInCB9| Working in CB9 (Quattor release >= 14.2)]]
*[[AideMemoire| FAQ - Aide-mémoire - Howtos]]
*[[BuildANewPysvnOnAiiServer| Howto build a new pysvn on a SL63 AII server]]
*[[QuattorFreeIPA| Quattor and FreeIPA]]
*[[NewRuncheck| Rewrite of the runcheck script in Perl]]
*[[HardDisksManagement| Hard disks management]]
*[[Aquilon| Aquilon]]

==== KVM virtualization ====
*[[VirtWithKVM| Virtualization of the new CREAM-CE on dom02 with KVM]]
*[[VirtWithKVM1| Installation of the new virtualization server dom04]]
*[[CreateVM| Easy creation of virtual machines]]
*[[MonitoringvHostswithGanglia| Monitoring the KVM vHosts with Ganglia]]

==== T2B Cloud ====
*[[MigrationToOpenNebula| Transforming the KVM hypervisors farm into an OpenNebula cloud]]
*[[WorkingInT2BCloud| Working in the T2B cloud]]
*[[MigrateDBMySQL| Migrate one DB from sqlite to mysql]]

==== gUSE/WS-PGRADE portal ====
*[[PortalInstall| Portal installation]]
*[[PortalConfig| Portal configuration]]
*[[PortalOperations| Portal operations]]

==== Migration to EMI-3 ====
*[[MigrateBEgridToEMI3_part1| BEgrid facilities - Part 1]]
*[[MigrateBEgridToEMI3_part2| BEgrid facilities - Part 2]]

==== XEN ====
*[[Manage_XEN| Manage XEN]]
*[[XenQuattor| Xen and Quattor]]

==== CEPH ====
*[[UnderstandingCeph| Understanding Ceph]]
*[[InstallCephWithQuattor| Installing Ceph with Quattor]]
*[[ExperimentsWithCeph| Experiments with Ceph]]
*[[CephBasics| Operating a Ceph cluster]]

==== Logstash / Elasticsearch / Kibana (ELK) ====
machine: log10 | [http://log10.iihe.ac.be/index.html interface] | [http://log10.iihe.ac.be/HQ index manager]
* [[log_forwarding_with_quattor|Forwarding a log with rsyslog to logstash using quattor]]
* [[log_parsing_with_logstash|Parsing the logs with logstash]]

==== Network ====
* [[network_bond_and_tag|Bonding of 2 interfaces + tagging of 2 vlans on the bond (PRIV+PUB)|]]

LToS

2016-05-19T12:57:46Z

Stephane GERARD:

== Configuration of the CE ==
This [https://wiki.egi.eu/wiki/MAN12 link] explains how to set up the PUSP mechanism on the CE. However, if you apply these recipes to the letter, it will break the CE. Here are the actual configurations we have applied :
* /etc/glexec.conf
<pre>
[glexec]
create_target_proxy=no
lcas_db_file=/etc/lcas/lcas-glexec.db
lcas_debug_level=5
lcas_log_file=/var/log/glexec/lcas_lcmaps.log
lcas_log_level=5
lcmaps_db_file=/etc/lcmaps/lcmaps.db.glexec.pusp
lcmaps_debug_level=5
lcmaps_get_account_policy=combi_mapping
lcmaps_log_file=/var/log/glexec/lcas_lcmaps.log
lcmaps_log_level=5
lcmaps_voms_verification=no
linger=no
log_destination=file
log_file=/var/log/glexec/glexec.log
log_level=5
omission_private_key_white_list=tomcat
preserve_env_variables=
silent_logging=no
use_lcas=no
user_identity_switch_by=lcmaps
user_white_list=tomcat
</pre>

* /etc/lcmaps/lcmaps.db.glexec.pusp
<pre>
path = /usr/lib64/lcmaps

vomspoolaccount = "lcmaps_voms_poolaccount.mod"
"-gridmapfile /etc/lcmaps/gridmapfile"
"-gridmapdir /etc/grid-security/gridmapdir"
"-override_inconsistency"

vomslocalgroup = "lcmaps_voms_localgroup.mod"
"-groupmapfile /etc/lcmaps/groupmapfile"
"-mapmin 0 "

proxycheck = "lcmaps_verify_proxy.mod"
"-certdir /etc/grid-security/certificates"
"--allow-limited-proxy"

posixenf = "lcmaps_posix_enf.mod"
"-maxuid 1"
"-maxpgid 1"
"-maxsgid 32"

vomslocalaccount = "lcmaps_voms_localaccount.mod"
"-gridmapfile /etc/lcmaps/gridmapfile"
"-use_voms_gid"

robot_pool = "lcmaps_robot_poolaccount.mod"
"-gridmapfile /etc/grid-security/grid-mapfile"
"-gridmapdir /etc/grid-security/gridmapdir/"

poolaccount = "lcmaps_poolaccount.mod"
"-override_inconsistency"
"-gridmapfile /etc/grid-security/grid-mapfile"
"-gridmapdir /etc/grid-security/gridmapdir"

robot_ban_dn = "lcmaps_robot_ban_dn.mod"
"-banmapfile /etc/lcas/ban_users.db"

localaccount = "lcmaps_localaccount.mod"
"-gridmapfile /etc/grid-security/grid-mapfile"

ban_dn = "lcmaps_ban_dn.mod"
"-banmapfile /etc/lcas/ban_users.db"

robot_local = "lcmaps_robot_localaccount.mod"
"-gridmapfile /etc/grid-security/grid-mapfile"

# Policies:
voms:
proxycheck -> vomslocalgroup
vomslocalgroup -> vomslocalaccount
vomslocalaccount -> posixenf | vomspoolaccount
vomspoolaccount -> posixenf

standard:
proxycheck -> localaccount
localaccount -> posixenf | poolaccount
poolaccount -> posixenf

combi_mapping:
ban_dn -> robot_ban_dn
robot_ban_dn -> proxycheck
proxycheck -> robot_pool
~robot_pool -> robot_local
~robot_local -> vomslocalgroup
vomslocalgroup -> vomslocalaccount
vomslocalaccount -> posixenf | vomspoolaccount
vomspoolaccount -> posixenf
</pre>

== Creation of per-user sub-proxies for beapps VO ==
First of all, you (= the VO admin) need to get a robot certificate that you will register in beapps VO. After that, you have to extract the usercert and the private key in a directory (.globus_pusp) directory and set the correct permissions. Thanks to this [https://ndpfsvn.nikhef.nl/viewvc/mwsec/trunk/lcmaps-plugins-robot/tools/ script], you can create a PUSP for a given user (mdupont) by issuing the following command :
<pre>
./create_pusp -u mdupont -c ~/.globus_pusp/usercert.pem -k ~/.globus_pusp/userkey.pem
</pre>
Now, if you issue the command :
<pre>
voms-proxy-info --all

subject : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be/CN=user:mdupont
issuer : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be
identity : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be
type : RFC3820 compliant impersonation proxy
strength : 1024
path : /tmp/x509up_u20533
timeleft : 23:59:51
key usage : Digital Signature, Key Encipherment
</pre>
you will see that the voms extensions are missing from this proxy. To add the beapps voms extension :
<pre>
voms-proxy-init --voms beapps --noregen
</pre>
The --noregen option is important because you just want to add voms extensions to the pusp proxy that already exists in /tmp/x509up_u20533.

Now, it looks like a good old usual beapps proxy :
<pre>
voms-proxy-info --all

subject : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be/CN=user:mdupont/CN=730118287
issuer : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be/CN=user:mdupont
identity : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be
type : RFC3820 compliant impersonation proxy
strength : 1024
path : /tmp/x509up_u20533
timeleft : 11:59:57
key usage : Digital Signature, Key Encipherment
=== VO beapps extension information ===
VO : beapps
subject : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be
issuer : /DC=org/DC=terena/DC=tcs/C=BE/ST=Brussels/L=Brussels/O=Le reseau telematique belge de la recherche/CN=voms01.begrid.be
attribute : /beapps/Role=NULL/Capability=NULL
timeleft : 11:59:56
uri : voms01.begrid.be:18004
</pre>

== Further reading ==

* https://wiki.egi.eu/wiki/Usage_of_the_per_user_sub_proxy_in_EGI
* https://wiki.egi.eu/wiki/MAN12

LToS

2016-05-18T20:25:56Z

Stephane GERARD: /* Creation of per-user sub-proxies for beapps VO */

LToS

2016-05-18T20:23:13Z

Stephane GERARD: /* Creation of per-user sub-proxies for beapps VO */

LToS

2016-05-18T20:22:57Z

Stephane GERARD: /* Creation of per-user sub-proxies for beapps VO */

== Configuration of the CE ==
This [https://wiki.egi.eu/wiki/MAN12 link] explains how to set up the PUSP mechanism on the CE. However, if you apply these recipes to the letter, it will break the CE. Here are the actual configurations we have applied :
* /etc/glexec.conf
<pre>
[glexec]
create_target_proxy=no
lcas_db_file=/etc/lcas/lcas-glexec.db
lcas_debug_level=5
lcas_log_file=/var/log/glexec/lcas_lcmaps.log
lcas_log_level=5
lcmaps_db_file=/etc/lcmaps/lcmaps.db.glexec.pusp
lcmaps_debug_level=5
lcmaps_get_account_policy=combi_mapping
lcmaps_log_file=/var/log/glexec/lcas_lcmaps.log
lcmaps_log_level=5
lcmaps_voms_verification=no
linger=no
log_destination=file
log_file=/var/log/glexec/glexec.log
log_level=5
omission_private_key_white_list=tomcat
preserve_env_variables=
silent_logging=no
use_lcas=no
user_identity_switch_by=lcmaps
user_white_list=tomcat
</pre>

* /etc/lcmaps/lcmaps.db.glexec.pusp
<pre>
path = /usr/lib64/lcmaps

vomspoolaccount = "lcmaps_voms_poolaccount.mod"
"-gridmapfile /etc/lcmaps/gridmapfile"
"-gridmapdir /etc/grid-security/gridmapdir"
"-override_inconsistency"

vomslocalgroup = "lcmaps_voms_localgroup.mod"
"-groupmapfile /etc/lcmaps/groupmapfile"
"-mapmin 0 "

proxycheck = "lcmaps_verify_proxy.mod"
"-certdir /etc/grid-security/certificates"
"--allow-limited-proxy"

posixenf = "lcmaps_posix_enf.mod"
"-maxuid 1"
"-maxpgid 1"
"-maxsgid 32"

vomslocalaccount = "lcmaps_voms_localaccount.mod"
"-gridmapfile /etc/lcmaps/gridmapfile"
"-use_voms_gid"

robot_pool = "lcmaps_robot_poolaccount.mod"
"-gridmapfile /etc/grid-security/grid-mapfile"
"-gridmapdir /etc/grid-security/gridmapdir/"

poolaccount = "lcmaps_poolaccount.mod"
"-override_inconsistency"
"-gridmapfile /etc/grid-security/grid-mapfile"
"-gridmapdir /etc/grid-security/gridmapdir"

robot_ban_dn = "lcmaps_robot_ban_dn.mod"
"-banmapfile /etc/lcas/ban_users.db"

localaccount = "lcmaps_localaccount.mod"
"-gridmapfile /etc/grid-security/grid-mapfile"

ban_dn = "lcmaps_ban_dn.mod"
"-banmapfile /etc/lcas/ban_users.db"

robot_local = "lcmaps_robot_localaccount.mod"
"-gridmapfile /etc/grid-security/grid-mapfile"

# Policies:
voms:
proxycheck -> vomslocalgroup
vomslocalgroup -> vomslocalaccount
vomslocalaccount -> posixenf | vomspoolaccount
vomspoolaccount -> posixenf

standard:
proxycheck -> localaccount
localaccount -> posixenf | poolaccount
poolaccount -> posixenf

combi_mapping:
ban_dn -> robot_ban_dn
robot_ban_dn -> proxycheck
proxycheck -> robot_pool
~robot_pool -> robot_local
~robot_local -> vomslocalgroup
vomslocalgroup -> vomslocalaccount
vomslocalaccount -> posixenf | vomspoolaccount
vomspoolaccount -> posixenf
</pre>

== Creation of per-user sub-proxies for beapps VO ==
First of all, you (= the VO admin) need to get a robot certificate that you will register in beapps VO. After that, you have to extract the usercert and the private key in a directory (.globus_pusp) directory and set the correct permissions. Thanks to this [https://ndpfsvn.nikhef.nl/viewvc/mwsec/trunk/lcmaps-plugins-robot/tools/ script], you can create a PUSP for a given user (mdupont) by issuing the following command :
<pre>
./create_pusp -u mdupont -c ~/.globus_pusp/usercert.pem -k ~/.globus_pusp/userkey.pem
</pre>
Now, if you issue the command :
<pre>
voms-proxy-info --all
subject : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be/CN=user:mdupont
issuer : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be
identity : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be
type : RFC3820 compliant impersonation proxy
strength : 1024
path : /tmp/x509up_u20533
timeleft : 23:59:51
key usage : Digital Signature, Key Encipherment
</pre>
you will see that the voms extensions are missing from this proxy. To add the beapps voms extension :
<pre>
voms-proxy-init --voms beapps --noregen
</pre>
The --noregen option is important because you just want to add voms extensions to the pusp proxy that already exists in /tmp/x509up_u20533.

LToS

2016-05-18T20:22:12Z

Stephane GERARD: /* Creation of per-user sub-proxies for beapps VO */

LToS

2016-05-18T20:15:48Z

Stephane GERARD:

LToS

2016-05-18T19:52:43Z

Stephane GERARD: /* Configuration of the CE */

LToS

2016-05-18T19:45:06Z

Stephane GERARD:

LToS

2016-05-18T14:50:41Z

Stephane GERARD:

LToS

2016-05-18T14:48:07Z

Stephane GERARD: /* Configuration of the CE */

== Configuration of the CE ==
This [https://wiki.egi.eu/wiki/MAN12 link] explains how to set up the PUSP mechanism on the CE.

LToS

2016-05-18T14:47:48Z

Stephane GERARD: Created page with "== Configuration of the CE == This [link https://wiki.egi.eu/wiki/MAN12] explains how to set up the PUSP mechanism on the CE."

== Configuration of the CE ==
This [link https://wiki.egi.eu/wiki/MAN12] explains how to set up the PUSP mechanism on the CE.

AdminPage

2016-05-18T14:45:58Z

Stephane GERARD:

{{DISPLAYTITLE:Page for Administrators}}
==== Management of the whole cluster ====
*[[elog]]
*[[ShutDownCluster| How to properly switch off the cluster]]
*[[PutClusterOn| How to properly put the cluster on]]
==== CMS Services ====
*[[Phedex]]
*[[Heartbeat]]
*[[LoadTest]]
*[[FroNTier]]
*[[ProdAgent]]
*[[GitForSiteConf| instructions to commit siteconf to git]]
==== Grid Configuration Issues ====
*[[UpdateCertificates| Update the certificates of all our machines]]
*[[CreamIssues| Issues with cream and how to solve them]]
*[[PBS_TMPDIR| PBS TMPDIR]]
*[[APEL| <strike>APEL</strike>(OBSOLETE)]]
*[[BDII]]
*[[FTS]]
*[[SL4_x86_64_WNs| <strike>SL4 x86_64 WNs</strike>(OBSOLETE)]]
*[[CE_oveloaded| CE overloaded]]
*[[RB]]
*[[IPMI]]
*[[CA_certificates| <strike>Upgrade CA certificates</strike> (OBSOLETE)]]
*[[Shutdown| Shutting down the cluster]]
*[[Software_Area_Switch| Software Area Switch]]
*[[KernelUpdate| Kernel mandatory updates for critical vulnerabilities]]
*[[Argus| Argus server and glexec on the workernodes]]
*[[ApelGapPublishing| Apel gap publishing]]
*[[UpdateCACertificates| Update IGTF CA certificates]]

==== Files section ====
*[[DCache| dCache]]
**[[DeleteObsoleteFiles| Find obsolete files]]
**[[DCacheAdminMode| dCache admin mode]]
**[[FindSizeOnDcache| find size of a directory on dcache]]
**[[DcachePoolConfig1912| dCache Pool Postinstallation steps]]
**[[DCacheMaxMovers| Adapt max mover]]
**[[pnfsScripts| scripts to see on what pools files in a directory reside and to move them to other pools]]
*[[OlPnfsFiles| Procedure for removal of old user files on pnfs]]
*[[GetLostFiles| Retrieve lost files from datasets]]
*[[StorageConsistency| Storage Consistency]]

==== Status and Monitoring ====
*[[ReservedWNs| List of reserved WNs]]
*[[Todo| Todo-list]]
*[[Monitoring]]
*[[Plans-Schedule| Plans/Schedule]]
*[[Grid_Troubleshooting_link| Grid Troubleshooting link]]
*[[Incident_reports| Incident Reports]]
*[[Dissapeared_software| How to put the software back]]
*[[Bad_WN| What to do when a WN sends a "bad_wn.pl" email to grid_admin ?]]
*[[Nagios_installation| Nagios Installation at IIHE]]
*[[Restart_DCache| How to restart DCache ]]

==== Info ====
*[[General_info| General info]]
*[[Installing_CMSSW| Installing CMSSW]]
*[[Installing_CRAB| Installing CRAB]]
*[[System_Benchmarks| System Benchmarks]]
*[[T2BTrac| T2B Trac config info]]
*[[HardWare| Hardware information]]
*[[NetworkSetup| Network Setup]]
*[[SetupMonitoringControlerSunfireV20z| Setup Monitoring of LSI Disk Controler on Sunfire V20z Server]]
*[[LDAP_UCL_IIHE| <strike>LDAP authentication system for the replication between UCL and IIHE sites</strike> (OBSOLETE)]]
*[[GridAdminSurvivalGuide| IIHE Grid-admin survival guide]]
*[[Solaris| Solaris 10]]
*[[SolarisSSD| Adding an SSD card and configuring RAID, zpools, filesystems and shares on the new Solaris fileserver]]
*[[LinuxAdminTricks| Linux tricks for admins]]
*[[CrabLocalPbsSubmission| <strike>How to implement local PBS submission with CRAB ?</strike>(OBSOLETE)]]
*[[AddNewUserFromUCLToLDAP| <strike>How to create an account for a CMS user from UCL ?</strike>(OBSOLETE)]]
*[[OSErrata| <strike>Deploying OS errata</strike>(OBSOLETE)]]
*[[BenchmarkHEPSPEC06| Howto benchmark a node with HEPSPEC06]]
*[[Installing_dcache_pool| Install a new dCache pool]]
*[[BackupUsersHomeDirs| Backup of the users home dirs on Jefke]]
*[[MonWebServicesMigration| Migration of mon and its Web services]]
*[[HOWTORestartNagiosTest| HOWTO restart a nagios test manually]]
*[[CompileAndInstallRoot| Compile and install ROOT]]
*[[CleanCreamdb| Clean creamdb]]
*Reboot campaign for the workernodes :
**[[KernelUpdate| Reboot after a kernel update]]
**[[UpgradeWNstoSL5.5| Reboot after an OS upgrade]]
*[[ManageAllAdminScriptsWithGit| Central management of all the admin scripts with Git]]
*[[ConfigProxyCvmfs| Configuration of a proxy for CVMFS]]
**[[RecoverCvmfs| How to recover CVMFS]]
*[[TestNFSPerformance| How to test NFS Performance]]
*[[TetexNotAvailableInSL6| Alternatives to Tetex]]
*[[NewMethodUpdateKernelWorkernodes| A new easy method to update kernel on the workernodes]]
*[[AutomaticMailSendingFromCluster| About automatic mail sending from the cluster]]
*[[T2BTracAccess| T2B Trac access configuration]]
*[[WorkingWithRHEL7| Surviving to RHEL7]]
*[[CCMWithKerberos| Experimental : Securing profiles with Kerberos]]
*[[MigrateToMediaWiki| Migration of T2B Wiki from Trac to MediaWiki]]
*[[motd|Message Of The Day (motd)]]
*[[LToS| Support of Long-tail of Science]]

==== Quattor ====
*[http://quattor.begrid.be/trac/centralised-begrid-v5/wiki/BEgridAndQuattor <strike>BEgrid wiki</strike>(OBSOLETE)]
*[[Test_things| <strike>Test things</strike>(OBSOLETE)]]
*[[Lemon_installation| <strike>Lemon installation</strike>(OBSOLETE)]]
*[[QuattorPointers| <strike>Pointers</strike>]]<strike> to more in-depth information on quattor</strike>(OBSOLETE)
*[[AddingMachineToCluster| <strike>Adding</strike>]]<strike> a new machine to the cluster</strike>(OBSOLETE)
*[[AutomaticMachineTemplateGeneration| Automatic generation of hardware and profile templates for new workernodes]]
*[[InstallationBEgridClient0| Installation of a Quattor deployment server release 13.1]]
*[[InstallFilesNewOS| How to add a new OS to the Quattor Repository]]
*[[GenerateRPMFromATagInGithub| How to build an RPM from a tag in Github]]
*[[HowtoMigrateWNToCB9| How to migrate workernodes from CB8 to CB9]]
*[[WorkingInCB9| Working in CB9 (Quattor release >= 14.2)]]
*[[AideMemoire| FAQ - Aide-mémoire - Howtos]]
*[[BuildANewPysvnOnAiiServer| Howto build a new pysvn on a SL63 AII server]]
*[[QuattorFreeIPA| Quattor and FreeIPA]]
*[[NewRuncheck| Rewrite of the runcheck script in Perl]]
*[[HardDisksManagement| Hard disks management]]
*[[Aquilon| Aquilon]]

==== KVM virtualization ====
*[[VirtWithKVM| Virtualization of the new CREAM-CE on dom02 with KVM]]
*[[VirtWithKVM1| Installation of the new virtualization server dom04]]
*[[CreateVM| Easy creation of virtual machines]]
*[[MonitoringvHostswithGanglia| Monitoring the KVM vHosts with Ganglia]]

==== T2B Cloud ====
*[[MigrationToOpenNebula| Transforming the KVM hypervisors farm into an OpenNebula cloud]]
*[[WorkingInT2BCloud| Working in the T2B cloud]]

==== gUSE/WS-PGRADE portal ====
*[[PortalInstall| Portal installation]]
*[[PortalConfig| Portal configuration]]
*[[PortalOperations| Portal operations]]

==== Migration to EMI-3 ====
*[[MigrateBEgridToEMI3_part1| BEgrid facilities - Part 1]]
*[[MigrateBEgridToEMI3_part2| BEgrid facilities - Part 2]]

==== XEN ====
*[[Manage_XEN| Manage XEN]]
*[[XenQuattor| Xen and Quattor]]

==== CEPH ====
*[[UnderstandingCeph| Understanding Ceph]]
*[[InstallCephWithQuattor| Installing Ceph with Quattor]]
*[[ExperimentsWithCeph| Experiments with Ceph]]
*[[CephBasics| Operating a Ceph cluster]]

==== Logstash / Elasticsearch / Kibana (ELK) ====
machine: log10 | [http://log10.iihe.ac.be/index.html interface] | [http://log10.iihe.ac.be/HQ index manager]
* [[log_forwarding_with_quattor|Forwarding a log with rsyslog to logstash using quattor]]
* [[log_parsing_with_logstash|Parsing the logs with logstash]]

==== Network ====
* [[network_bond_and_tag|Bonding of 2 interfaces + tagging of 2 vlans on the bond (PRIV+PUB)|]]

AdminPage

2016-05-18T14:45:22Z

Stephane GERARD:

{{DISPLAYTITLE:Page for Administrators}}
==== Management of the whole cluster ====
*[[elog]]
*[[ShutDownCluster| How to properly switch off the cluster]]
*[[PutClusterOn| How to properly put the cluster on]]
==== CMS Services ====
*[[Phedex]]
*[[Heartbeat]]
*[[LoadTest]]
*[[FroNTier]]
*[[ProdAgent]]
*[[GitForSiteConf| instructions to commit siteconf to git]]
==== Grid Configuration Issues ====
*[[UpdateCertificates| Update the certificates of all our machines]]
*[[CreamIssues| Issues with cream and how to solve them]]
*[[PBS_TMPDIR| PBS TMPDIR]]
*[[APEL| <strike>APEL</strike>(OBSOLETE)]]
*[[BDII]]
*[[FTS]]
*[[SL4_x86_64_WNs| <strike>SL4 x86_64 WNs</strike>(OBSOLETE)]]
*[[CE_oveloaded| CE overloaded]]
*[[RB]]
*[[IPMI]]
*[[CA_certificates| <strike>Upgrade CA certificates</strike> (OBSOLETE)]]
*[[Shutdown| Shutting down the cluster]]
*[[Software_Area_Switch| Software Area Switch]]
*[[KernelUpdate| Kernel mandatory updates for critical vulnerabilities]]
*[[Argus| Argus server and glexec on the workernodes]]
*[[ApelGapPublishing| Apel gap publishing]]
*[[UpdateCACertificates| Update IGTF CA certificates]]

==== Files section ====
*[[DCache| dCache]]
**[[DeleteObsoleteFiles| Find obsolete files]]
**[[DCacheAdminMode| dCache admin mode]]
**[[FindSizeOnDcache| find size of a directory on dcache]]
**[[DcachePoolConfig1912| dCache Pool Postinstallation steps]]
**[[DCacheMaxMovers| Adapt max mover]]
**[[pnfsScripts| scripts to see on what pools files in a directory reside and to move them to other pools]]
*[[OlPnfsFiles| Procedure for removal of old user files on pnfs]]
*[[GetLostFiles| Retrieve lost files from datasets]]
*[[StorageConsistency| Storage Consistency]]

==== Status and Monitoring ====
*[[ReservedWNs| List of reserved WNs]]
*[[Todo| Todo-list]]
*[[Monitoring]]
*[[Plans-Schedule| Plans/Schedule]]
*[[Grid_Troubleshooting_link| Grid Troubleshooting link]]
*[[Incident_reports| Incident Reports]]
*[[Dissapeared_software| How to put the software back]]
*[[Bad_WN| What to do when a WN sends a "bad_wn.pl" email to grid_admin ?]]
*[[Nagios_installation| Nagios Installation at IIHE]]
*[[Restart_DCache| How to restart DCache ]]

==== Info ====
*[[General_info| General info]]
*[[Installing_CMSSW| Installing CMSSW]]
*[[Installing_CRAB| Installing CRAB]]
*[[System_Benchmarks| System Benchmarks]]
*[[T2BTrac| T2B Trac config info]]
*[[HardWare| Hardware information]]
*[[NetworkSetup| Network Setup]]
*[[SetupMonitoringControlerSunfireV20z| Setup Monitoring of LSI Disk Controler on Sunfire V20z Server]]
*[[LDAP_UCL_IIHE| <strike>LDAP authentication system for the replication between UCL and IIHE sites</strike> (OBSOLETE)]]
*[[GridAdminSurvivalGuide| IIHE Grid-admin survival guide]]
*[[Solaris| Solaris 10]]
*[[SolarisSSD| Adding an SSD card and configuring RAID, zpools, filesystems and shares on the new Solaris fileserver]]
*[[LinuxAdminTricks| Linux tricks for admins]]
*[[CrabLocalPbsSubmission| <strike>How to implement local PBS submission with CRAB ?</strike>(OBSOLETE)]]
*[[AddNewUserFromUCLToLDAP| <strike>How to create an account for a CMS user from UCL ?</strike>(OBSOLETE)]]
*[[OSErrata| <strike>Deploying OS errata</strike>(OBSOLETE)]]
*[[BenchmarkHEPSPEC06| Howto benchmark a node with HEPSPEC06]]
*[[Installing_dcache_pool| Install a new dCache pool]]
*[[BackupUsersHomeDirs| Backup of the users home dirs on Jefke]]
*[[MonWebServicesMigration| Migration of mon and its Web services]]
*[[HOWTORestartNagiosTest| HOWTO restart a nagios test manually]]
*[[CompileAndInstallRoot| Compile and install ROOT]]
*[[CleanCreamdb| Clean creamdb]]
*Reboot campaign for the workernodes :
**[[KernelUpdate| Reboot after a kernel update]]
**[[UpgradeWNstoSL5.5| Reboot after an OS upgrade]]
*[[ManageAllAdminScriptsWithGit| Central management of all the admin scripts with Git]]
*[[ConfigProxyCvmfs| Configuration of a proxy for CVMFS]]
**[[RecoverCvmfs| How to recover CVMFS]]
*[[TestNFSPerformance| How to test NFS Performance]]
*[[TetexNotAvailableInSL6| Alternatives to Tetex]]
*[[NewMethodUpdateKernelWorkernodes| A new easy method to update kernel on the workernodes]]
*[[AutomaticMailSendingFromCluster| About automatic mail sending from the cluster]]
*[[T2BTracAccess| T2B Trac access configuration]]
*[[WorkingWithRHEL7| Surviving to RHEL7]]
*[[CCMWithKerberos| Experimental : Securing profiles with Kerberos]]
*[[MigrateToMediaWiki| Migration of T2B Wiki from Trac to MediaWiki]]
*[[motd|Message Of The Day (motd)]]

==== Quattor ====
*[http://quattor.begrid.be/trac/centralised-begrid-v5/wiki/BEgridAndQuattor <strike>BEgrid wiki</strike>(OBSOLETE)]
*[[Test_things| <strike>Test things</strike>(OBSOLETE)]]
*[[Lemon_installation| <strike>Lemon installation</strike>(OBSOLETE)]]
*[[QuattorPointers| <strike>Pointers</strike>]]<strike> to more in-depth information on quattor</strike>(OBSOLETE)
*[[AddingMachineToCluster| <strike>Adding</strike>]]<strike> a new machine to the cluster</strike>(OBSOLETE)
*[[AutomaticMachineTemplateGeneration| Automatic generation of hardware and profile templates for new workernodes]]
*[[InstallationBEgridClient0| Installation of a Quattor deployment server release 13.1]]
*[[InstallFilesNewOS| How to add a new OS to the Quattor Repository]]
*[[GenerateRPMFromATagInGithub| How to build an RPM from a tag in Github]]
*[[HowtoMigrateWNToCB9| How to migrate workernodes from CB8 to CB9]]
*[[WorkingInCB9| Working in CB9 (Quattor release >= 14.2)]]
*[[AideMemoire| FAQ - Aide-mémoire - Howtos]]
*[[BuildANewPysvnOnAiiServer| Howto build a new pysvn on a SL63 AII server]]
*[[QuattorFreeIPA| Quattor and FreeIPA]]
*[[NewRuncheck| Rewrite of the runcheck script in Perl]]
*[[HardDisksManagement| Hard disks management]]
*[[Aquilon| Aquilon]]
*[[LToS| Support of Long-tail of Science]]

==== KVM virtualization ====
*[[VirtWithKVM| Virtualization of the new CREAM-CE on dom02 with KVM]]
*[[VirtWithKVM1| Installation of the new virtualization server dom04]]
*[[CreateVM| Easy creation of virtual machines]]
*[[MonitoringvHostswithGanglia| Monitoring the KVM vHosts with Ganglia]]

==== T2B Cloud ====
*[[MigrationToOpenNebula| Transforming the KVM hypervisors farm into an OpenNebula cloud]]
*[[WorkingInT2BCloud| Working in the T2B cloud]]

==== gUSE/WS-PGRADE portal ====
*[[PortalInstall| Portal installation]]
*[[PortalConfig| Portal configuration]]
*[[PortalOperations| Portal operations]]

==== Migration to EMI-3 ====
*[[MigrateBEgridToEMI3_part1| BEgrid facilities - Part 1]]
*[[MigrateBEgridToEMI3_part2| BEgrid facilities - Part 2]]

==== XEN ====
*[[Manage_XEN| Manage XEN]]
*[[XenQuattor| Xen and Quattor]]

==== CEPH ====
*[[UnderstandingCeph| Understanding Ceph]]
*[[InstallCephWithQuattor| Installing Ceph with Quattor]]
*[[ExperimentsWithCeph| Experiments with Ceph]]
*[[CephBasics| Operating a Ceph cluster]]

==== Logstash / Elasticsearch / Kibana (ELK) ====
machine: log10 | [http://log10.iihe.ac.be/index.html interface] | [http://log10.iihe.ac.be/HQ index manager]
* [[log_forwarding_with_quattor|Forwarding a log with rsyslog to logstash using quattor]]
* [[log_parsing_with_logstash|Parsing the logs with logstash]]

==== Network ====
* [[network_bond_and_tag|Bonding of 2 interfaces + tagging of 2 vlans on the bond (PRIV+PUB)|]]

ApelGapPublishing

2016-04-06T10:31:52Z

Stephane GERARD: /* How to activate the gap publisher ? */

=== Goal ===
On this page, we explain how to publish gap with the EMI3 apel.
=== Where to find the results of the apel sync tests ? ===
These results are available [http://goc-accounting.grid-support.ac.uk/rss/BEgrid-ULB-VUB_Sync.html here].
=== How to activate the gap publisher ? ===
Log in on apel and modify the config file /etc/apel/client.cfg following the explanations given in commented lines :
<pre>
# Which records to send:
# latest - just send the new records to the server
# gap - send records from between the specified dates (inclusive)
# this is only for individual job records
# all - send all records to the server. Don't do this for individual
# job records without talking to the apel team!
interval = latest
## only used if interval = gap
#gap_start = 2012-01-01
#gap_end = 2012-01-31
</pre>
So, let's say that there is a gap in April 2014. We must the change the config so that it contains :
<pre>
interval = gap
gap_start = 2014-04-01
gap_end = 2014-04-30
</pre>
Then, you can just wait for the apelclient to be started by cron, or you can run it manually with the following command :
<pre>
date --iso-8601=seconds --utc; /usr/bin/apelclient) >> /var/log/apelclient.log 2>&1
</pre>
When it's done, don't forget to put back interval to 'latest' and to comment gap_start and gap_end.

{{TracNotice|{{PAGENAME}}}}

WorkingWithRHEL7

2016-03-24T20:26:12Z

Stephane GERARD:

== Getting rid of firewalld and coming back to iptables ==
Here are the magic commands :
<pre>
systemctl stop firewalld
systemctl disable firewalld
yum remove firewalld
yum install iptables-services
systemctl enable iptables.service
systemctl enable ip6tables.service
</pre>
Don't forget to configure SSH with system-config-firewall-tui. And after that :
<pre>
systemctl start iptables.service
systemctl start ip6tables.service
</pre>

== Replacing NetworkManager by network ==
First, check if there is the '''GATEWAY''' parameter in the ifcfg-ethX of the NIC. (GATEWAY0 is not working without NM)
Then, type these commands :
<pre>
systemctl stop NetworkManager
systemctl disable NetworkManager
</pre>
And now, we can restart network :
<pre>
service network restart
</pre>

== Adoption of Predictable Network Interface Names ==
Names of NICs will change. To prepare yourself to the new naming rules, please read these documents :
*http://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
*http://blog.laimbock.com/2014/11/22/systemd-understanding-predictable-network-interface-names/

== Default filesystem : xfs ==
RHEL7 adopts XFS as the default filesystem. It might generates troubles with our Quattor scdb -> check our filesystem layout templates.

== Transition from SysVinit to Systemd ==
I find this [https://fedoraproject.org/wiki/SysVinit_to_Systemd_Cheatsheet link] useful.

Here are some useful commands :

=== List types of unit ===
<pre>
systemctl -t help
</pre>
=== List units of a certain type ===
<pre>
systemctl list-units --type=[unit_name]
</pre>
=== List of services ===
<pre>
systemctl list-units --type=service
</pre>
=== Check status of services ===
<pre>
systemctl status [service_name]
</pre>
If you want something more concise info :
<pre>
systemctl is-enabled [service_name]
systemctl is-active [service_name]
</pre>
=== Stop/start/restart service ===
<pre>
systemctl start [service_name]
systemctl stop [service_name]
systemctl restart [service_name]
</pre>
=== Enable/disable at boot time ===
<pre>
systemctl enable [service_name]
systemctl disable [service_name]
</pre>

{{TracNotice|{{PAGENAME}}}}

WorkingWithRHEL7

2016-03-24T20:24:08Z

Stephane GERARD: /* Transition from SysVinit to Systemd */

=== Getting rid of firewalld and coming back to iptables ===
Here are the magic commands :
<pre>
systemctl stop firewalld
systemctl disable firewalld
yum remove firewalld
yum install iptables-services
systemctl enable iptables.service
systemctl enable ip6tables.service
</pre>
Don't forget to configure SSH with system-config-firewall-tui. And after that :
<pre>
systemctl start iptables.service
systemctl start ip6tables.service
</pre>

=== Replacing NetworkManager by network ===
First, check if there is the '''GATEWAY''' parameter in the ifcfg-ethX of the NIC. (GATEWAY0 is not working without NM)
Then, type these commands :
<pre>
systemctl stop NetworkManager
systemctl disable NetworkManager
</pre>
And now, we can restart network :
<pre>
service network restart
</pre>

=== Adoption of Predictable Network Interface Names ===
Names of NICs will change. To prepare yourself to the new naming rules, please read these documents :
*http://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
*http://blog.laimbock.com/2014/11/22/systemd-understanding-predictable-network-interface-names/

=== Default filesystem : xfs ===
RHEL7 adopts XFS as the default filesystem. It might generates troubles with our Quattor scdb -> check our filesystem layout templates.

=== Transition from SysVinit to Systemd ===
I find this [https://fedoraproject.org/wiki/SysVinit_to_Systemd_Cheatsheet link] useful.

Here are some useful commands :

==== List types of unit ====
<pre>
systemctl -t help
</pre>
==== List units of a certain type ====
<pre>
systemctl list-units --type=[unit_name]
</pre>
==== List of services ====
<pre>
systemctl list-units --type=service
</pre>
==== Check status of services ====
<pre>
systemctl status [service_name]
</pre>
If you want something more concise info :
<pre>
systemctl is-enabled [service_name]
systemctl is-active [service_name]
</pre>
==== Stop/start/restart service ====
<pre>
systemctl start [service_name]
systemctl stop [service_name]
systemctl restart [service_name]
</pre>
==== Enable/disable at boot time ====
<pre>
systemctl enable [service_name]
systemctl disable [service_name]
</pre>

{{TracNotice|{{PAGENAME}}}}

WorkingWithRHEL7

2016-03-24T20:20:38Z

Stephane GERARD: /* Transition from SysVinit to Systemd */

WorkingWithRHEL7

2016-03-24T20:06:55Z

Stephane GERARD: /* Transition from SysVinit to Systemd */

=== Getting rid of firewalld and coming back to iptables ===
Here are the magic commands :
<pre>
systemctl stop firewalld
systemctl disable firewalld
yum remove firewalld
yum install iptables-services
systemctl enable iptables.service
systemctl enable ip6tables.service
</pre>
Don't forget to configure SSH with system-config-firewall-tui. And after that :
<pre>
systemctl start iptables.service
systemctl start ip6tables.service
</pre>

=== Replacing NetworkManager by network ===
First, check if there is the '''GATEWAY''' parameter in the ifcfg-ethX of the NIC. (GATEWAY0 is not working without NM)
Then, type these commands :
<pre>
systemctl stop NetworkManager
systemctl disable NetworkManager
</pre>
And now, we can restart network :
<pre>
service network restart
</pre>

=== Adoption of Predictable Network Interface Names ===
Names of NICs will change. To prepare yourself to the new naming rules, please read these documents :
*http://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
*http://blog.laimbock.com/2014/11/22/systemd-understanding-predictable-network-interface-names/

=== Default filesystem : xfs ===
RHEL7 adopts XFS as the default filesystem. It might generates troubles with our Quattor scdb -> check our filesystem layout templates.

=== Transition from SysVinit to Systemd ===
I find this [https://fedoraproject.org/wiki/SysVinit_to_Systemd_Cheatsheet link] useful.

Here are some useful commands :

* List types of unit
* Check status of services

{{TracNotice|{{PAGENAME}}}}

MigrateToMediaWiki

2016-03-18T16:19:37Z

Stephane GERARD: /* Create a new empty MediaWiki */

=== The plan of the migration ===
Here are the steps :
*creation of a new Apache webserver called "mona" with Quattor
*moving all the Trac wiki's to mona
*conversion of the Trac wiki's to MediaWiki (thanks to a homebrewed script)
*once everything is ready, switch to the new webserver in the DNS servers

=== Creation of the new webserver ===
It is a virtual machine with 60GB of disk size (see the hardware template hardware/machine/Virtual/virtual_kvm_mon for more details). The software configuration is done through the template config/t2b_docu_server.

=== Moving the Trac wiki's to the new webserver ===
For each wiki that was on the old mon server, create a new wiki on the new webserver with the trac-admin command. For example :
<pre>
trac-admin /var/www/trac/t2b initenv
</pre>
On the old webserver, you make a copy of each wiki to a nfs-shared directory. Going on with the previous example, it gives something like this :
<pre>
trac-admin /var/www/trac/t2b hotcopy /userbackup/sgerard/trac_backup_20-08-2015/t2b
</pre>
Now, going back to the new webserver, we can simply overwrite the content of /var/www/trac/t2b with the content of the back-up copy :
<pre>
cp -a -p /userbackup/sgerard/trac_backup_20-08-2015/t2b /var/www/trac/
</pre>
(Of course, it works because the nfs-share where the backup copy was done on the old server has been mounted on the new webserver thanks to Quattor ;-)

=== Conversion from Trac to MediaWiki ===
For each Trac wiki that you would like to migrate, you have to create a new empty MediaWiki. I will explain all the details below.

==== Create a new empty MediaWiki ====
Here are steps to create a wiki "test" :
<pre>
mkdir /var/www/mediawiki199/test
chown apache:apache /var/www/mediawiki199/test
cd /var/www/mediawiki199/test
ln -s /usr/share/mediawiki199/* .
rm LocalSetting.php AdminSettings.php images mw-config
cp -a /usr/share/mediawiki199/mw-config .
</pre>

Now, you have to open the URL of the wiki in your browser for the initialization. After you have filled in all the forms of the wizard, you arrive on page saying "Installation successful !", and that you can download the LocalSettings.php and copy it to the directory of your wiki. After it is copied, don't forget to adapt the permissions :
<pre>
chown apache:apache LocalSettings.php
chmod 640 LocalSettings.php
</pre>
After that, you can remove the mw-config directory because it is only necessary to run the wizard :
<pre>
rm -rf mw-config
</pre>

To enable image uploading, you need to do this in the directory of the wiki :
<pre>
mkdir images
chmod 755 images
chown apache:apache images
</pre>

And then, you need to add a few lines at the end of the LocalSetting.php config. file :
<pre>
$wgUploadDirectory = "images";
$wgUploadPath = "$wgScriptPath/images";
$wgEnableUploads = true;
</pre>

And that's it !

==== Conversion ====
Now, for each wiki, the next step is to export all the pages in txt format, convert them in MediaWiki format, and then import the converted pages in the corresponding MediaWiki. To avoid hours of tedious manual tasks, a script has been created, and it is available on our SVN "scripting" repository : http://mon.iihe.ac.be/repos/scripting/mon/convert_trac_to_mediawiki/. Here is an example showing how to use it :
<ol>
<li>The following commands will create a blank new wiki :
<pre>
ssh mona
svn co http://mon.iihe.ac.be/repos/scripting/mon/convert_trac_to_mediawiki
cd convert_trac_to_mediawiki
./export_all_pages.pl /var/www/trac/t2b /var/www/mediawiki119/t2b
</pre>
</li>
<li>Now you have to initialize the wiki. Go to the URL you have created :
<pre>
http://mona.iihe.ac.be/wiki/test
</pre>
and follow the instructions.
</li>
</ol>

=== Access restriction to wiki pages ===
Anybody can read the pages, but to modify the content, you need to login. Login is done through SSL, meaning that your certificate will be asked, and if its DN contains "O=BEGRID" or "DC=cern", it is accepted and an account is automatically created. This account remains associated to your DN.

Now, the technical details : we use "Extension:SSL authentication" that is documented [https://www.mediawiki.org/wiki/Extension:SSL_authentication here]. But to force the switch to HTTPS when clicking on "Log in", we had to add this in the httpd.conf :
<pre>
RewriteCond %{REQUEST_URI} ^/wiki/t2b/index.php$
RewriteCond %{QUERY_STRING} ^title=Special:UserLogin
RewriteCond %{REQUEST_METHOD} ^GET$
RewriteRule ^(.*)$ https://%{SERVER_NAME}/wiki/t2b/index.php/Main_Page [R]
</pre>

=== Sources ===
*http://mechanicalkeys.com/wiki/index.php?title=Install_MediaWiki_on_CentOS_6.4
*http://sharkysoft.com/wiki/how_to_configure_multiple_MediaWiki_instances_on_a_single_host
*https://www.mediawiki.org/wiki/Extension:TracWiki2MediaWiki

{{TracNotice|{{PAGENAME}}}}

Main Page

2016-02-04T10:07:26Z

Stephane GERARD: /* Information for new users */

== Welcome to the CMS Belgian T2 Wiki ==
<center><span style="font-size: 300%;"> [[first_access_to_t2b|=> FIRST ACCESS TO T2B <=]] </span></center>

=== General information for users ===
*[[Faq_t2b | FAQ]]
==== Information for new users ====
*[[T2bSupport| T2B Support guidelines]]
*[[First_access_to_t2b|Getting access to T2B]]
*[[Getting_a_certificate_for_the_T2|Certificates and VOs]]

==== Using the Tier2 computing resources ====
*[[policies| Policies concerning the usage of computing resources.]]
*[[CurrentStatus| Current status of the Tier2]]
*[[Getting_started_with_the_CMSSW_software| Getting started with the CMSSW software]]
*[[Using_Git| Using Git]]
*[[Getting_started_with_the_MadGraph_software| Getting started with the MadGraph software]]
*Submitting jobs with CRAB
**[[gridSubmission_withCrab| To the worldwide grid]]
*Submitting jobs without CRAB
**[[localSubmission| To the local resources]]
**[[gridSubmission| To the worldwide grid]]
*[[GridStorageAccess| How to handle data on Grid storage]]
*[[FAQ_CMSSW_on_the_Grid| FAQ CMSSW on the Grid on proxy and more!]]
*[[OtherSoftware| Other software available at the T2]]

==== Back-up procedures ====
*[[BackupDocsLinuxLaptop| Procedure to automate backups of personal documents (Linux laptops only)]]
*[[Backup| Backup scheme for the user disks]]
*[[AccidentalDeleteOfFiles| What to do if I have accidentally deleted some files on my personal computer ?]]

==== Other topics ====
*[[User_Meetings]] list of user meetings with added transparencies
*[[Basic_computing_skills| Basic computing skills]]
*[[CernLxplus| Useful info on use of lxplus.cern.ch]]

----

=== Dedicated twiki pages maintained by several research groups ===
*[[TtBar_Analysis_Framework| TtBar Analysis Framework (old)]]
*[[TopQuarkGroup| Top Quark Group wiki]]
*[[HEEP_Analysis_Framework| HEEP Analysis Framework]]
*[[V0_Analysis_wiki| V0 Analysis wiki]]
*[[Info_exchange| Higgs analysis]]

----

=== Service work for CMS ===
*[[DDT]]
*[[CMSSWDeployment]]
*[[ProdAgentAllUsers| Prodagent for users]]
*[[TestStoreTemp| Writing tests in /store/temp/user on T2 SE's]]
*[[CrabServerInstall| CRAB Server installation]]
*[[CrabValidation| Basic validation of CRAB releases]]

----

=== Obsolete twiki pages ===
*[[DIY-UI]]
*[[CrabIIHETransitionSLC4ToSLC5| Using CRAB at IIHE during the transition from SLC4 to SLC5]]
*[[DataSamplesRequests2007| Samples Requests for 2007]]
*[[DataSamplesRequests2008| Samples Requests for 2008]]
*[[CrabNewIIHE| Crab at IIHE]]
*[[OldMainPage| The old main page is kept here for reference]]

== Admin section ==
*[[AdminPage| Pages for administrators]]

== Getting started with MediaWiki ==
*[//meta.wikimedia.org/wiki/Help:Contents User's Guide]
*[//www.mediawiki.org/wiki/Manual:Configuration_settings Configuration settings list]
*[//www.mediawiki.org/wiki/Manual:FAQ MediaWiki FAQ]
*[https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]

AdminPage

2016-02-03T12:21:48Z

Stephane GERARD: /* Grid Configuration Issues */

{{DISPLAYTITLE:Page for Administrators}}
==== Management of the whole cluster ====
*[[elog]]
*[[ShutDownCluster| How to properly switch off the cluster]]
*[[PutClusterOn| How to properly put the cluster on]]
==== CMS Services ====
*[[Phedex]]
*[[Heartbeat]]
*[[LoadTest]]
*[[FroNTier]]
*[[ProdAgent]]
*[[GitForSiteConf| instructions to commit siteconf to git]]
==== Grid Configuration Issues ====
*[[UpdateCertificates| Update the certificates of all our machines]]
*[[CreamIssues| Issues with cream and how to solve them]]
*[[PBS_TMPDIR| PBS TMPDIR]]
*[[APEL]]
*[[BDII]]
*[[FTS]]
*[[SL4_x86_64_WNs| SL4 x86_64 WNs]]
*[[CE_oveloaded| CE overloaded]]
*[[RB]]
*[[IPMI]]
*[[CA_certificates| <strike>Upgrade CA certificates</strike> (OBSOLETE)]]
*[[Shutdown| Shutting down the cluster]]
*[[Software_Area_Switch| Software Area Switch]]
*[[KernelUpdate| Kernel mandatory updates for critical vulnerabilities]]
*[[Argus| Argus server and glexec on the workernodes]]
*[[ApelGapPublishing| Apel gap publishing]]
*[[UpdateCACertificates| Update IGTF CA certificates]]

==== Files section ====
*[[DCache| dCache]]
**[[DeleteObsoleteFiles| Find obsolete files]]
**[[DCacheAdminMode| dCache admin mode]]
**[[FindSizeOnDcache| find size of a directory on dcache]]
**[[DcachePoolConfig1912| dCache Pool Postinstallation steps]]
**[[DCacheMaxMovers| Adapt max mover]]
**[[pnfsScripts| scripts to see on what pools files in a directory reside and to move them to other pools]]
*[[OlPnfsFiles| Procedure for removal of old user files on pnfs]]
*[[GetLostFiles| Retrieve lost files from datasets]]
*[[StorageConsistency| Storage Consistency]]

==== Status and Monitoring ====
*[[ReservedWNs| List of reserved WNs]]
*[[Todo| Todo-list]]
*[[Monitoring]]
*[[Plans-Schedule| Plans/Schedule]]
*[[Grid_Troubleshooting_link| Grid Troubleshooting link]]
*[[Incident_reports| Incident Reports]]
*[[Dissapeared_software| How to put the software back]]
*[[Bad_WN| What to do when a WN sends a "bad_wn.pl" email to grid_admin ?]]
*[[Nagios_installation| Nagios Installation at IIHE]]
*[[Restart_DCache| How to restart DCache ]]

==== Info ====
*[[General_info| General info]]
*[[Installing_CMSSW| Installing CMSSW]]
*[[Installing_CRAB| Installing CRAB]]
*[[System_Benchmarks| System Benchmarks]]
*[[T2BTrac| T2B Trac config info]]
*[[HardWare| Hardware information]]
*[[NetworkSetup| Network Setup]]
*[[SetupMonitoringControlerSunfireV20z| Setup Monitoring of LSI Disk Controler on Sunfire V20z Server]]
*[[LDAP_UCL_IIHE| LDAP authentication system for the replication between UCL and IIHE sites]]
*[[GridAdminSurvivalGuide| IIHE Grid-admin survival guide]]
*[[Solaris| Solaris 10]]
*[[SolarisSSD| Adding an SSD card and configuring RAID, zpools, filesystems and shares on the new Solaris fileserver]]
*[[LinuxAdminTricks| Linux tricks for admins]]
*[[CrabLocalPbsSubmission| How to implement local PBS submission with CRAB ?]]
*[[AddNewUserFromUCLToLDAP| How to create an account for a CMS user from UCL ?]]
*[[OSErrata| Deploying OS errata]]
*[[BenchmarkHEPSPEC06| Howto benchmark a node with HEPSPEC06]]
*[[Installing_dcache_pool| Install a new Dcache pool]]
*[[BackupUsersHomeDirs| Backup of the users home dirs on Jefke]]
*[[MonWebServicesMigration| Migration of mon and its Web services]]
*[[HOWTORestartNagiosTest| HOWTO restart a nagios test manually]]
*[[CompileAndInstallRoot| Compile and install ROOT]]
*[[CleanCreamdb| Clean creamdb]]
*Reboot campaign for the workernodes :
**[[KernelUpdate| Reboot after a kernel update]]
**[[UpgradeWNstoSL5.5| Reboot after an OS upgrade]]
*[[ManageAllAdminScriptsWithGit| Central management of all the admin scripts with Git]]
*[[ConfigProxyCvmfs| Configuration of a proxy for CVMFS]]
**[[RecoverCvmfs| How to recover CVMFS]]
*[[TestNFSPerformance| How to test NFS Performance]]
*[[TetexNotAvailableInSL6| Alternatives to Tetex]]
*[[NewMethodUpdateKernelWorkernodes| A new easy method to update kernel on the workernodes]]
*[[AutomaticMailSendingFromCluster| About automatic mail sending from the cluster]]
*[[T2BTracAccess| T2B Trac access configuration]]
*[[WorkingWithRHEL7| Surviving to RHEL7]]
*[[CCMWithKerberos| Experimental : Securing profiles with Kerberos]]
*[[MigrateToMediaWiki| Migration of T2B Wiki from Trac to MediaWiki]]
*[[motd|Message Of The Day (motd)]]

==== Quattor ====
*[http://quattor.begrid.be/trac/centralised-begrid-v5/wiki/BEgridAndQuattor BEgrid wiki]
*[[Test_things| Test things]]
*[[Lemon_installation| Lemon installation]]
*[[QuattorPointers| Pointers]] to more in-depth information on quattor
*[[AddingMachineToCluster| Adding]] a new machine to the cluster
*[[AutomaticMachineTemplateGeneration| Automatic generation of hardware and profile templates for new workernodes]]
*[[InstallationBEgridClient0| Installation of a Quattor deployment server release 13.1]]
*[[InstallFilesNewOS| How to add a new OS to the Quattor Repository]]
*[[GenerateRPMFromATagInGithub| How to build an RPM from a tag in Github]]
*[[HowtoMigrateWNToCB9| How to migrate workernodes from CB8 to CB9]]
*[[WorkingInCB9| Working in CB9 (Quattor release >= 14.2)]]
*[[AideMemoire| FAQ - Aide-mémoire - Howtos]]
*[[BuildANewPysvnOnAiiServer| Howto build a new pysvn on a SL63 AII server]]
*[[QuattorFreeIPA| Quattor and FreeIPA]]
*[[NewRuncheck| Rewrite of the runcheck script in Perl]]
*[[HardDisksManagement| Hard disks management]]
*[[Aquilon| Aquilon]]

==== KVM virtualization ====
*[[VirtWithKVM| Virtualization of the new CREAM-CE on dom02 with KVM]]
*[[VirtWithKVM1| Installation of the new virtualization server dom04]]
*[[CreateVM| Easy creation of virtual machines]]
*[[MonitoringvHostswithGanglia| Monitoring the KVM vHosts with Ganglia]]

==== T2B Cloud ====
*[[MigrationToOpenNebula| Transforming the KVM hypervisors farm into an OpenNebula cloud]]
*[[WorkingInT2BCloud| Working in the T2B cloud]]

==== gUSE/WS-PGRADE portal ====
*[[PortalInstall| Portal installation]]
*[[PortalConfig| Portal configuration]]
*[[PortalOperations| Portal operations]]

==== Migration to EMI-3 ====
*[[MigrateBEgridToEMI3_part1| BEgrid facilities - Part 1]]
*[[MigrateBEgridToEMI3_part2| BEgrid facilities - Part 2]]

==== XEN ====
*[[Manage_XEN| Manage XEN]]
*[[XenQuattor| Xen and Quattor]]

==== CEPH ====
*[[UnderstandingCeph| Understanding Ceph]]
*[[InstallCephWithQuattor| Installing Ceph with Quattor]]
*[[ExperimentsWithCeph| Experiments with Ceph]]
*[[CephBasics| Operating a Ceph cluster]]

==== Logstash / Elasticsearch / Kibana (ELK) ====
machine: log10 | [http://log10.iihe.ac.be/index.html interface] | [http://log10.iihe.ac.be/HQ index manager]
* [[log_forwarding_with_quattor|Forwarding a log with rsyslog to logstash using quattor]]
* [[log_parsing_with_logstash|Parsing the logs with logstash]]

UpdateCACertificates

2016-02-03T11:08:13Z

Stephane GERARD: /* Procedure to update the IGTF CA certificates */

=== Procedure to update the IGTF CA certificates ===

*First, it's important to check if the new RPMs are well available here : http://repository.egi.eu/sw/production/cas/1/current/RPMS/
*Login to yum
*Go to dir /var/www/html
*Launch ./manage_repositories.pl and choose first option in the menu to sync the local repo with the distant one
*Launch ./manage_repositories.pl and choose second option in the menu to create a new snapshot
*Take note of the timestamp of the snapshot
*In the Quattor SVN, open the template site/global_variables and in the nlist REPO_YUM_SNAPSHOT_DATE, change the timestamp for the key 'ca'
*You can easily guess the rest : local build, commit, runcheck...

UpdateCACertificates

2016-02-03T11:06:29Z

Stephane GERARD: /* Procedure to update the IGTF CA certificates */

=== Procedure to update the IGTF CA certificates ===

*Login to yum
*Go to dir /var/www/html
*Launch ./manage_repositories.pl and choose first option in the menu to sync the local repo with the distant one
*Launch ./manage_repositories.pl and choose second option in the menu to create a new snapshot
*Take note of the timestamp of the snapshot
*In the Quattor SVN, open the template site/global_variables and in the nlist REPO_YUM_SNAPSHOT_DATE, change the timestamp for the key 'ca'
*You can easily guess the rest : local build, commit, runcheck...

UpdateCACertificates

2016-02-03T11:02:22Z

Stephane GERARD: Created page with "=== Procedure to update the IGTF CA certificates === *Login to yum *Go to dir /var/www/html *Launch ./manage_repositories.pl and choose first option in the menu to sync the l..."

=== Procedure to update the IGTF CA certificates ===

*Login to yum
*Go to dir /var/www/html
*Launch ./manage_repositories.pl and choose first option in the menu to sync the local repo with the distant one
*Launch ./manage_repositories.pl and choose second option in the menu to create a new snapshot
*Take not of the timestamp of the snapshot
*In the Quattor SVN, open the template site/global_variables and in the nlist REPO_YUM_SNAPSHOT_DATE, change the timestamp for the key 'ca'
*Build, commit, runcheck...

AdminPage

2016-02-03T10:57:01Z

Stephane GERARD: /* Grid Configuration Issues */

{{DISPLAYTITLE:Page for Administrators}}
==== Management of the whole cluster ====
*[[elog]]
*[[ShutDownCluster| How to properly switch off the cluster]]
*[[PutClusterOn| How to properly put the cluster on]]
==== CMS Services ====
*[[Phedex]]
*[[Heartbeat]]
*[[LoadTest]]
*[[FroNTier]]
*[[ProdAgent]]
*[[GitForSiteConf| instructions to commit siteconf to git]]
==== Grid Configuration Issues ====
*[[UpdateCertificates| Update the certificates of all our machines]]
*[[CreamIssues| Issues with cream and how to solve them]]
*[[PBS_TMPDIR| PBS TMPDIR]]
*[[APEL]]
*[[BDII]]
*[[FTS]]
*[[SL4_x86_64_WNs| SL4 x86_64 WNs]]
*[[CE_oveloaded| CE overloaded]]
*[[RB]]
*[[IPMI]]
*[[CA_certificates| Upgrade CA certificates]]
*[[Shutdown| Shutting down the cluster]]
*[[Software_Area_Switch| Software Area Switch]]
*[[KernelUpdate| Kernel mandatory updates for critical vulnerabilities]]
*[[Argus| Argus server and glexec on the workernodes]]
*[[ApelGapPublishing| Apel gap publishing]]
*[[UpdateCACertificates| Update IGTF CA certificates]]

==== Files section ====
*[[DCache| dCache]]
**[[DeleteObsoleteFiles| Find obsolete files]]
**[[DCacheAdminMode| dCache admin mode]]
**[[FindSizeOnDcache| find size of a directory on dcache]]
**[[DcachePoolConfig1912| dCache Pool Postinstallation steps]]
**[[DCacheMaxMovers| Adapt max mover]]
**[[pnfsScripts| scripts to see on what pools files in a directory reside and to move them to other pools]]
*[[OlPnfsFiles| Procedure for removal of old user files on pnfs]]
*[[GetLostFiles| Retrieve lost files from datasets]]
*[[StorageConsistency| Storage Consistency]]

==== Status and Monitoring ====
*[[ReservedWNs| List of reserved WNs]]
*[[Todo| Todo-list]]
*[[Monitoring]]
*[[Plans-Schedule| Plans/Schedule]]
*[[Grid_Troubleshooting_link| Grid Troubleshooting link]]
*[[Incident_reports| Incident Reports]]
*[[Dissapeared_software| How to put the software back]]
*[[Bad_WN| What to do when a WN sends a "bad_wn.pl" email to grid_admin ?]]
*[[Nagios_installation| Nagios Installation at IIHE]]
*[[Restart_DCache| How to restart DCache ]]

==== Info ====
*[[General_info| General info]]
*[[Installing_CMSSW| Installing CMSSW]]
*[[Installing_CRAB| Installing CRAB]]
*[[System_Benchmarks| System Benchmarks]]
*[[T2BTrac| T2B Trac config info]]
*[[HardWare| Hardware information]]
*[[NetworkSetup| Network Setup]]
*[[SetupMonitoringControlerSunfireV20z| Setup Monitoring of LSI Disk Controler on Sunfire V20z Server]]
*[[LDAP_UCL_IIHE| LDAP authentication system for the replication between UCL and IIHE sites]]
*[[GridAdminSurvivalGuide| IIHE Grid-admin survival guide]]
*[[Solaris| Solaris 10]]
*[[SolarisSSD| Adding an SSD card and configuring RAID, zpools, filesystems and shares on the new Solaris fileserver]]
*[[LinuxAdminTricks| Linux tricks for admins]]
*[[CrabLocalPbsSubmission| How to implement local PBS submission with CRAB ?]]
*[[AddNewUserFromUCLToLDAP| How to create an account for a CMS user from UCL ?]]
*[[OSErrata| Deploying OS errata]]
*[[BenchmarkHEPSPEC06| Howto benchmark a node with HEPSPEC06]]
*[[Installing_dcache_pool| Install a new Dcache pool]]
*[[BackupUsersHomeDirs| Backup of the users home dirs on Jefke]]
*[[MonWebServicesMigration| Migration of mon and its Web services]]
*[[HOWTORestartNagiosTest| HOWTO restart a nagios test manually]]
*[[CompileAndInstallRoot| Compile and install ROOT]]
*[[CleanCreamdb| Clean creamdb]]
*Reboot campaign for the workernodes :
**[[KernelUpdate| Reboot after a kernel update]]
**[[UpgradeWNstoSL5.5| Reboot after an OS upgrade]]
*[[ManageAllAdminScriptsWithGit| Central management of all the admin scripts with Git]]
*[[ConfigProxyCvmfs| Configuration of a proxy for CVMFS]]
**[[RecoverCvmfs| How to recover CVMFS]]
*[[TestNFSPerformance| How to test NFS Performance]]
*[[TetexNotAvailableInSL6| Alternatives to Tetex]]
*[[NewMethodUpdateKernelWorkernodes| A new easy method to update kernel on the workernodes]]
*[[AutomaticMailSendingFromCluster| About automatic mail sending from the cluster]]
*[[T2BTracAccess| T2B Trac access configuration]]
*[[WorkingWithRHEL7| Surviving to RHEL7]]
*[[CCMWithKerberos| Experimental : Securing profiles with Kerberos]]
*[[MigrateToMediaWiki| Migration of T2B Wiki from Trac to MediaWiki]]
*[[motd|Message Of The Day (motd)]]

==== Quattor ====
*[http://quattor.begrid.be/trac/centralised-begrid-v5/wiki/BEgridAndQuattor BEgrid wiki]
*[[Test_things| Test things]]
*[[Lemon_installation| Lemon installation]]
*[[QuattorPointers| Pointers]] to more in-depth information on quattor
*[[AddingMachineToCluster| Adding]] a new machine to the cluster
*[[AutomaticMachineTemplateGeneration| Automatic generation of hardware and profile templates for new workernodes]]
*[[InstallationBEgridClient0| Installation of a Quattor deployment server release 13.1]]
*[[InstallFilesNewOS| How to add a new OS to the Quattor Repository]]
*[[GenerateRPMFromATagInGithub| How to build an RPM from a tag in Github]]
*[[HowtoMigrateWNToCB9| How to migrate workernodes from CB8 to CB9]]
*[[WorkingInCB9| Working in CB9 (Quattor release >= 14.2)]]
*[[AideMemoire| FAQ - Aide-mémoire - Howtos]]
*[[BuildANewPysvnOnAiiServer| Howto build a new pysvn on a SL63 AII server]]
*[[QuattorFreeIPA| Quattor and FreeIPA]]
*[[NewRuncheck| Rewrite of the runcheck script in Perl]]
*[[HardDisksManagement| Hard disks management]]
*[[Aquilon| Aquilon]]

==== KVM virtualization ====
*[[VirtWithKVM| Virtualization of the new CREAM-CE on dom02 with KVM]]
*[[VirtWithKVM1| Installation of the new virtualization server dom04]]
*[[CreateVM| Easy creation of virtual machines]]
*[[MonitoringvHostswithGanglia| Monitoring the KVM vHosts with Ganglia]]

==== T2B Cloud ====
*[[MigrationToOpenNebula| Transforming the KVM hypervisors farm into an OpenNebula cloud]]
*[[WorkingInT2BCloud| Working in the T2B cloud]]

==== gUSE/WS-PGRADE portal ====
*[[PortalInstall| Portal installation]]
*[[PortalConfig| Portal configuration]]
*[[PortalOperations| Portal operations]]

==== Migration to EMI-3 ====
*[[MigrateBEgridToEMI3_part1| BEgrid facilities - Part 1]]
*[[MigrateBEgridToEMI3_part2| BEgrid facilities - Part 2]]

==== XEN ====
*[[Manage_XEN| Manage XEN]]
*[[XenQuattor| Xen and Quattor]]

==== CEPH ====
*[[UnderstandingCeph| Understanding Ceph]]
*[[InstallCephWithQuattor| Installing Ceph with Quattor]]
*[[ExperimentsWithCeph| Experiments with Ceph]]
*[[CephBasics| Operating a Ceph cluster]]

==== Logstash / Elasticsearch / Kibana (ELK) ====
machine: log10 | [http://log10.iihe.ac.be/index.html interface] | [http://log10.iihe.ac.be/HQ index manager]
* [[log_forwarding_with_quattor|Forwarding a log with rsyslog to logstash using quattor]]
* [[log_parsing_with_logstash|Parsing the logs with logstash]]

PortalOperations

2015-11-19T21:47:55Z

Stephane GERARD: /* How to stop/start the portal ? */

{{DISPLAYTITLE:Portal operations}}
== How to stop/start the portal ? ==
Login on guse as root and source the ~/.guserc file :
<pre>
source ~/.guserc
</pre>
To stop the portal :
<pre>
cd /usr/local/guse/apache-tomcat-6.0.39/bin
./shutdown.sh
</pre>
To start the portal :
<pre>
./startup.sh
</pre>
After a restart of the portal, you might need to start the Service Wizard by opening the following URL :
<pre>
http://< URL_install_backend>:8080/information
</pre>
In the popup menu Database Configuration the MySQL server must be redefined. Please do not forget to replace the string IP by the IP of the MySQL server.

== Log files ==
Tomcat logfiles are here :
<pre>
/usr/local/guse/apache-tomcat-6.0.39/logs/
</pre>
Liferay logfiles :
<pre>
/usr/local/guse/logs/
</pre>

{{TracNotice|{{PAGENAME}}}}

InstallCephWithQuattor

2015-11-17T16:29:48Z

Stephane GERARD: /* Before deploying Ceph */

== Overview ==
The component ncm-ceph is based on ceph-deploy. It is documented [http://quattor-core.readthedocs.org/en/latest/components/ceph/ here].

Prior to the installation, you must assign the roles to the machines of the future cluster. The possible roles are : MON, MDS, OSD, and DEPLOY. The machine with the DEPLOY role will be the one from which deployment will be done, and so, it is the machine on which the component ncm-ceph will be executed. You need at least one MON machine, but to avoid bottlenecks when the number of OSD increases, it's better to have 3 MON machines (always have an odd number of MON machines to be able to keep a quorum in case of failure of one MON machine). The MDS is only needed if you want to use CephFS. The OSD daemons will run on machines that will contains the storage disks (one OSD daemon per disk). When doing tests, you can put the MON daemons on OSD machines, but on large cluster in production, it's better to put the MON daemons on dedicated machines.

The layout and the parameters of the cluster are described in the template site/ceph.

== Before deploying Ceph ==
The deployment of the Ceph cluster is triggered by this line in the profile of the DEPLOY machine :
<pre>
include 'site/ceph';
</pre>
But before the deployment, your OSD machines must be prepared : the OSD disks must be partitioned and formatted correctly. In a OSD machine, at least one partition or one drive must be prepared to store the objects. To achieve this, you need to define an appropriate disk layout. You will find an example of such a disk layout in template site/filesystems/ceph_fs_test. Below, the intesting parts of this template :
<pre>
...
variable CEPH_FS = 'btrfs';

variable DISK_VOLUME_PARAMS = {
t = dict();
...
t['logvda'] = dict(
'size', -1,
'mountpoint', '/var/lib/ceph/log/vda4',
'fstype', CEPH_FS,
'type', 'partition',
'device', 'vda4',
'preserve', true,
'mkfsopts', CEPH_DISK_OPTIONS[CEPH_FS]['mkfsopts'],
'mountopts', CEPH_DISK_OPTIONS[CEPH_FS]['mountopts'],
);
t['osd'] = dict(
'size', -1,
'mountpoint', '/var/lib/ceph/osd/vdb',
'fstype', CEPH_FS,
'type', 'partition',
'device', 'vdb1',
'preserve', true,
'mkfsopts', CEPH_DISK_OPTIONS[CEPH_FS]['mkfsopts'],
'mountopts', CEPH_DISK_OPTIONS[CEPH_FS]['mountopts'],
);
t;
};

...

</pre>
From this extract, you can see that our OSD has two drives (vda and vdb). We choose to use the filesystem btrfs to store the objects (you should use ext4 of xfs for production cluster). The last partition of vda (vda4) will contain the journal of the filesystem, and the vdb disk will only contain one partition (vdb1) to store the data.

InstallCephWithQuattor

2015-11-17T16:23:00Z

Stephane GERARD: /* Before deploying Ceph */

InstallCephWithQuattor

2015-11-17T16:18:41Z

Stephane GERARD: /* Before deploying Ceph */

InstallCephWithQuattor

2015-11-17T16:14:42Z

Stephane GERARD: /* Overview */

InstallCephWithQuattor

2015-11-17T16:04:58Z

Stephane GERARD: /* Overview */

InstallCephWithQuattor

2015-11-17T15:43:56Z

Stephane GERARD: /* Overview */

== Overview ==
Prior to the installation, you must assign the roles to the machines of the cluster. The possible roles are : MON, MDS, OSD, and DEPLOY. You need at least one MON machine, but to avoid bottlenecks when the number of OSD increases, it's better to have 3 MON machines (always have an odd number of MON machines to be able to have a quorum). The MDS is only needed if you want to use CephFS. The OSD daemons will run on machines that will contains the storage disks.

The layout and the parameters of the cluster are described in the template site/ceph.

InstallCephWithQuattor

2015-11-17T15:37:26Z

Stephane GERARD: Created page with "== Overview == Prior to the installation, you must assign the roles to the machines of the cluster. The possible roles are : MON, MDS, OSD. You need at least one MON machine, ..."

== Overview ==
Prior to the installation, you must assign the roles to the machines of the cluster. The possible roles are : MON, MDS, OSD. You need at least one MON machine, but to avoid bottlenecks when the number of OSD increases, it's better to have 3 MON machines (always have an odd number of MON machines to be able to have a quorum). The MDS is only needed if you want to use CephFS.

AdminPage

2015-11-17T15:23:47Z

Stephane GERARD: /* CEPH */

{{DISPLAYTITLE:Page for Administrators}}
==== Management of the whole cluster ====
*[[elog]]
*[[ShutDownCluster| How to properly switch off the cluster]]
*[[PutClusterOn| How to properly put the cluster on]]
==== CMS Services ====
*[[Phedex]]
*[[Heartbeat]]
*[[LoadTest]]
*[[FroNTier]]
*[[ProdAgent]]
*[[GitForSiteConf| instructions to commit siteconf to git]]
==== Grid Configuration Issues ====
*[[UpdateCertificates| Update the certificates of all our machines]]
*[[CreamIssues| Issues with cream and how to solve them]]
*[[PBS_TMPDIR| PBS TMPDIR]]
*[[APEL]]
*[[BDII]]
*[[FTS]]
*[[SL4_x86_64_WNs| SL4 x86_64 WNs]]
*[[CE_oveloaded| CE overloaded]]
*[[RB]]
*[[IPMI]]
*[[CA_certificates| Upgrade CA certificates]]
*[[Shutdown| Shutting down the cluster]]
*[[Software_Area_Switch| Software Area Switch]]
*[[KernelUpdate| Kernel mandatory updates for critical vulnerabilities]]
*[[Argus| Argus server and glexec on the workernodes]]
*[[ApelGapPublishing| Apel gap publishing]]
==== Files section ====
*[[DCache| dCache]]
**[[DeleteObsoleteFiles| Find obsolete files]]
**[[DCacheAdminMode| dCache admin mode]]
**[[FindSizeOnDcache| find size of a directory on dcache]]
**[[DcachePoolConfig1912| dCache Pool Postinstallation steps]]
**[[DCacheMaxMovers| Adapt max mover]]
**[[pnfsScripts| scripts to see on what pools files in a directory reside and to move them to other pools]]
*[[OlPnfsFiles| Procedure for removal of old user files on pnfs]]
*[[GetLostFiles| Retrieve lost files from datasets]]
*[[StorageConsistency| Storage Consistency]]

==== Status and Monitoring ====
*[[ReservedWNs| List of reserved WNs]]
*[[Todo| Todo-list]]
*[[Monitoring]]
*[[Plans-Schedule| Plans/Schedule]]
*[[Grid_Troubleshooting_link| Grid Troubleshooting link]]
*[[Incident_reports| Incident Reports]]
*[[Dissapeared_software| How to put the software back]]
*[[Bad_WN| What to do when a WN sends a "bad_wn.pl" email to grid_admin ?]]
*[[Nagios_installation| Nagios Installation at IIHE]]
*[[Restart_DCache| How to restart DCache ]]

==== Info ====
*[[General_info| General info]]
*[[Installing_CMSSW| Installing CMSSW]]
*[[Installing_CRAB| Installing CRAB]]
*[[System_Benchmarks| System Benchmarks]]
*[[T2BTrac| T2B Trac config info]]
*[[HardWare| Hardware information]]
*[[NetworkSetup| Network Setup]]
*[[SetupMonitoringControlerSunfireV20z| Setup Monitoring of LSI Disk Controler on Sunfire V20z Server]]
*[[LDAP_UCL_IIHE| LDAP authentication system for the replication between UCL and IIHE sites]]
*[[GridAdminSurvivalGuide| IIHE Grid-admin survival guide]]
*[[Solaris| Solaris 10]]
*[[SolarisSSD| Adding an SSD card and configuring RAID, zpools, filesystems and shares on the new Solaris fileserver]]
*[[LinuxAdminTricks| Linux tricks for admins]]
*[[CrabLocalPbsSubmission| How to implement local PBS submission with CRAB ?]]
*[[AddNewUserFromUCLToLDAP| How to create an account for a CMS user from UCL ?]]
*[[OSErrata| Deploying OS errata]]
*[[BenchmarkHEPSPEC06| Howto benchmark a node with HEPSPEC06]]
*[[Installing_dcache_pool| Install a new Dcache pool]]
*[[BackupUsersHomeDirs| Backup of the users home dirs on Jefke]]
*[[MonWebServicesMigration| Migration of mon and its Web services]]
*[[HOWTORestartNagiosTest| HOWTO restart a nagios test manually]]
*[[CompileAndInstallRoot| Compile and install ROOT]]
*[[CleanCreamdb| Clean creamdb]]
*Reboot campaign for the workernodes :
**[[KernelUpdate| Reboot after a kernel update]]
**[[UpgradeWNstoSL5.5| Reboot after an OS upgrade]]
*[[ManageAllAdminScriptsWithSVN| Central management of all the admin scripts with SVN]]
*[[ConfigProxyCvmfs| Configuration of a proxy for CVMFS]]
**[[RecoverCvmfs| How to recover CVMFS]]
*[[TestNFSPerformance| How to test NFS Performance]]
*[[TetexNotAvailableInSL6| Alternatives to Tetex]]
*[[NewMethodUpdateKernelWorkernodes| A new easy method to update kernel on the workernodes]]
*[[AutomaticMailSendingFromCluster| About automatic mail sending from the cluster]]
*[[T2BTracAccess| T2B Trac access configuration]]
*[[WorkingWithRHEL7| Surviving to RHEL7]]
*[[CCMWithKerberos| Experimental : Securing profiles with Kerberos]]
*[[MigrateToMediaWiki| Migration of T2B Wiki from Trac to MediaWiki]]

==== Quattor ====
*[http://quattor.begrid.be/trac/centralised-begrid-v5/wiki/BEgridAndQuattor BEgrid wiki]
*[[Test_things| Test things]]
*[[Lemon_installation| Lemon installation]]
*[[QuattorPointers| Pointers]] to more in-depth information on quattor
*[[AddingMachineToCluster| Adding]] a new machine to the cluster
*[[AutomaticMachineTemplateGeneration| Automatic generation of hardware and profile templates for new workernodes]]
*[[InstallationBEgridClient0| Installation of a Quattor deployment server release 13.1]]
*[[InstallFilesNewOS| How to add a new OS to the Quattor Repository]]
*[[GenerateRPMFromATagInGithub| How to build an RPM from a tag in Github]]
*[[HowtoMigrateWNToCB9| How to migrate workernodes from CB8 to CB9]]
*[[WorkingInCB9| Working in CB9 (Quattor release >= 14.2)]]
*[[AideMemoire| FAQ - Aide-mémoire - Howtos]]
*[[BuildANewPysvnOnAiiServer| Howto build a new pysvn on a SL63 AII server]]
*[[QuattorFreeIPA| Quattor and FreeIPA]]
*[[NewRuncheck| Rewrite of the runcheck script in Perl]]
*[[HardDisksManagement| Hard disks management]]
*[[Aquilon| Aquilon]]

==== KVM virtualization ====
*[[VirtWithKVM| Virtualization of the new CREAM-CE on dom02 with KVM]]
*[[VirtWithKVM1| Installation of the new virtualization server dom04]]
*[[CreateVM| Easy creation of virtual machines]]
*[[MonitoringvHostswithGanglia| Monitoring the KVM vHosts with Ganglia]]

==== T2B Cloud ====
*[[MigrationToOpenNebula| Transforming the KVM hypervisors farm into an OpenNebula cloud]]
*[[WorkingInT2BCloud| Working in the T2B cloud]]

==== gUSE/WS-PGRADE portal ====
*[[PortalInstall| Portal installation]]
*[[PortalConfig| Portal configuration]]
*[[PortalOperations| Portal operations]]

==== Migration to EMI-3 ====
*[[MigrateBEgridToEMI3_part1| BEgrid facilities - Part 1]]
*[[MigrateBEgridToEMI3_part2| BEgrid facilities - Part 2]]

==== XEN ====
*[[Manage_XEN| Manage XEN]]
*[[XenQuattor| Xen and Quattor]]

==== CEPH ====
*[[UnderstandingCeph| Understanding Ceph]]
*[[InstallCephWithQuattor| Installing Ceph with Quattor]]
*[[ExperimentsWithCeph| Experiments with Ceph]]
*[[CephBasics| Operating a Ceph cluster]]

{{TracNotice|{{PAGENAME}}}}

CephBasics

2015-11-04T11:28:38Z

Stephane GERARD:

{{DISPLAYTITLE:Operating a Ceph cluster}}
== Where to operate ? ==
All operations should be done on the ceph-admin machine. In our experimental testbed, it is cephq1.wn.iihe.ac.be.
== Check Ceph cluster status ==
The command :
<pre>
ceph -s
</pre>
shows the actual status of the Ceph cluster :
<pre>
[root@cephq1 ~]# ceph -s
cluster 82766e04-585b-49a6-a0ac-c13d9ffd0a7d
health HEALTH_OK
monmap e1: 3 mons at {cephq2=192.168.41.2:6789/0,cephq3=192.168.41.3:6789/0,cephq4=192.168.41.4:6789/0}
election epoch 8, quorum 0,1,2 cephq2,cephq3,cephq4
osdmap e78: 6 osds: 6 up, 6 in
pgmap v1293: 192 pgs, 2 pools, 0 bytes data, 0 objects
27920 kB used, 4021 GB / 4106 GB avail
192 active+clean

</pre>
The following command displays a real-time summary of the status of the cluster, and major events :
<pre>
ceph -w
</pre>

== Remove OSDs ==
When you want to remove a machine that contains OSDs (for example : decommissioning of an old equipment out of warranty), there is manual procedure to follow in order to do things in a clean way and to avoid problems :
<ol>
<li> Identify the OSDs hosted by the machine with the command :
<pre>
ceph osd tree
</pre>
</li>
<li> Take the OSDs out of the cluster :
Before you remove an OSD, it is usually up and in. You need to take it out of the cluster so that Ceph can begin rebalancing and copying its data to other OSDs :
<pre>
ceph osd out {osd-num}
</pre>
Repeat this operation for all the OSDs on the machine.
</li>
<li> Monitor the data migration :
Once you have taken the OSDs out of the cluster, Ceph will begin rebalancing the cluster by migrating placement groups out of the OSDs you've removed. You must follow this process with the following command :
<pre>
ceph -w
</pre>
You should see the placement group states change from active+clean to active, some degraded objects, and finally active+clean when migration completes.
</li>
<li> Stopping the OSDs daemons
After you take an OSD out of the cluster, it may still be running. That is, the OSD may be up and out. You must stop your OSD before you remove it from the configuration :
<pre>
ssh {osd-host}
/etc/init.d/ceph stop osd.{osd-num}
</pre>
(Repeat the last command for all the OSDs on the machine.)
As a result, a "ceph -s" should show the OSDs as down.
</li>
<li> Removing the OSDs :
<ol>
<li>Remove OSDs from crush map :
<pre>
ceph osd crush remove osd.{osd-num}
</pre>
</li>
<li>Remove the OSD authentication key :
<pre>
ceph auth del osd.{osd-num}
</pre>
</li>
<li>Remove the OSDs
<pre>
ceph osd rm {osd-num}
</pre>
</li>
</ol>
</li>
</ol>

CephBasics

2015-11-03T22:42:05Z

Stephane GERARD: /* Remove OSDs */

{{DISPLAYTITLE:Operating a Ceph cluster}}
== Check Ceph cluster status ==
The command :
<pre>
ceph -s
</pre>
shows the actual status of the Ceph cluster :
<pre>
[root@cephq1 ~]# ceph -s
cluster 82766e04-585b-49a6-a0ac-c13d9ffd0a7d
health HEALTH_OK
monmap e1: 3 mons at {cephq2=192.168.41.2:6789/0,cephq3=192.168.41.3:6789/0,cephq4=192.168.41.4:6789/0}
election epoch 8, quorum 0,1,2 cephq2,cephq3,cephq4
osdmap e78: 6 osds: 6 up, 6 in
pgmap v1293: 192 pgs, 2 pools, 0 bytes data, 0 objects
27920 kB used, 4021 GB / 4106 GB avail
192 active+clean

</pre>
The following command displays a real-time summary of the status of the cluster, and major events :
<pre>
ceph -w
</pre>

== Remove OSDs ==
When you want to remove a machine that contains OSDs (for example : decommissioning of an old equipment out of warranty), there is manual procedure to follow in order to do things in a clean way and to avoid problems :
<ol>
<li> Identify the OSDs hosted by the machine with the command :
<pre>
ceph osd tree
</pre>
</li>
<li> Take the OSDs out of the cluster :
Before you remove an OSD, it is usually up and in. You need to take it out of the cluster so that Ceph can begin rebalancing and copying its data to other OSDs :
<pre>
ceph osd out {osd-num}
</pre>
Repeat this operation for all the OSDs on the machine.
</li>
<li> Monitor the data migration :
Once you have taken the OSDs out of the cluster, Ceph will begin rebalancing the cluster by migrating placement groups out of the OSDs you've removed. You must follow this process with the following command :
<pre>
ceph -w
</pre>
You should see the placement group states change from active+clean to active, some degraded objects, and finally active+clean when migration completes.
</li>
<li> Stopping the OSDs daemons
After you take an OSD out of the cluster, it may still be running. That is, the OSD may be up and out. You must stop your OSD before you remove it from the configuration :
<pre>
ssh {osd-host}
/etc/init.d/ceph stop osd.{osd-num}
</pre>
(Repeat the last command for all the OSDs on the machine.)
As a result, a "ceph -s" should show the OSDs as down.
</li>
<li> Removing the OSDs :
<ol>
<li>Remove OSDs from crush map :
<pre>
ceph osd crush remove osd.{osd-num}
</pre>
</li>
<li>Remove the OSD authentication key :
<pre>
ceph auth del osd.{osd-num}
</pre>
</li>
<li>Remove the OSDs
<pre>
ceph osd rm {osd-num}
</pre>
</li>
</ol>
</li>
</ol>

CephBasics

2015-11-03T22:40:37Z

Stephane GERARD: /* Remove OSDs */

CephBasics

2015-11-03T22:31:37Z

Stephane GERARD: /* Remove OSDs */

CephBasics

2015-11-03T22:29:58Z

Stephane GERARD: /* Remove OSDs */

{{DISPLAYTITLE:Operating a Ceph cluster}}
== Check Ceph cluster status ==
The command :
<pre>
ceph -s
</pre>
shows the actual status of the Ceph cluster :
<pre>
[root@cephq1 ~]# ceph -s
cluster 82766e04-585b-49a6-a0ac-c13d9ffd0a7d
health HEALTH_OK
monmap e1: 3 mons at {cephq2=192.168.41.2:6789/0,cephq3=192.168.41.3:6789/0,cephq4=192.168.41.4:6789/0}
election epoch 8, quorum 0,1,2 cephq2,cephq3,cephq4
osdmap e78: 6 osds: 6 up, 6 in
pgmap v1293: 192 pgs, 2 pools, 0 bytes data, 0 objects
27920 kB used, 4021 GB / 4106 GB avail
192 active+clean

</pre>
The following command displays a real-time summary of the status of the cluster, and major events :
<pre>
ceph -w
</pre>

== Remove OSDs ==
When you want to remove a machine that contains OSDs (for example : decommissioning of an old equipment out of warranty), there is manual procedure to follow in order to do things in a clean way and to avoid problems :
<ol>
<li> Identify the OSDs hosted by the machine with the command :
<pre>
ceph osd tree
</pre>
</li>
<li> Take the OSDs out of the cluster :
Before you remove an OSD, it is usually up and in. You need to take it out of the cluster so that Ceph can begin rebalancing and copying its data to other OSDs :
<pre>
ceph osd out {osd-num}
</pre>
Repeat this operation for all the OSDs on the machine.
</li>
<li> Monitor the data migration :
Once you have taken the OSDs out of the cluster, Ceph will begin rebalancing the cluster by migrating placement groups out of the OSDs you've removed. You must follow this process with the following command :
<pre>
ceph -w
</pre>
You should see the placement group states change from active+clean to active, some degraded objects, and finally active+clean when migration completes.
</li>
<li> Stopping the OSDs daemons
After you take an OSD out of the cluster, it may still be running. That is, the OSD may be up and out. You must stop your OSD before you remove it from the configuration :
<pre>
ssh {osd-host}
/etc/init.d/ceph stop osd.{osd-num}
</pre>
(Repeat the last command for all the OSDs on the machine.)
As a result, a "ceph -s" should show the OSDs as down.
</li>
<li> Removing the OSDs
<li>Remove OSDs from crush map :
<pre>
ceph osd crush remove osd.{osd-num}
</pre>
</li>
<li>Remove the OSD authentication key :
<pre>
ceph auth del osd.{osd-num}
</pre>
</li>
</li>
</ol>

CephBasics

2015-11-03T22:27:59Z

Stephane GERARD: /* Remove OSDs */

{{DISPLAYTITLE:Operating a Ceph cluster}}
== Check Ceph cluster status ==
The command :
<pre>
ceph -s
</pre>
shows the actual status of the Ceph cluster :
<pre>
[root@cephq1 ~]# ceph -s
cluster 82766e04-585b-49a6-a0ac-c13d9ffd0a7d
health HEALTH_OK
monmap e1: 3 mons at {cephq2=192.168.41.2:6789/0,cephq3=192.168.41.3:6789/0,cephq4=192.168.41.4:6789/0}
election epoch 8, quorum 0,1,2 cephq2,cephq3,cephq4
osdmap e78: 6 osds: 6 up, 6 in
pgmap v1293: 192 pgs, 2 pools, 0 bytes data, 0 objects
27920 kB used, 4021 GB / 4106 GB avail
192 active+clean

</pre>
The following command displays a real-time summary of the status of the cluster, and major events :
<pre>
ceph -w
</pre>

== Remove OSDs ==
When you want to remove a machine that contains OSDs (for example : decommissioning of an old equipment out of warranty), there is manual procedure to follow in order to do things in a clean way and to avoid problems :
<ol>
<li> Identify the OSDs hosted by the machine with the command :
<pre>
ceph osd tree
</pre>
</li>
<li> Take the OSDs out of the cluster :
Before you remove an OSD, it is usually up and in. You need to take it out of the cluster so that Ceph can begin rebalancing and copying its data to other OSDs :
<pre>
ceph osd out {osd-num}
</pre>
Repeat this operation for all the OSDs on the machine.
</li>
<li> Monitor the data migration :
Once you have taken the OSDs out of the cluster, Ceph will begin rebalancing the cluster by migrating placement groups out of the OSDs you've removed. You must follow this process with the following command :
<pre>
ceph -w
</pre>
You should see the placement group states change from active+clean to active, some degraded objects, and finally active+clean when migration completes.
</li>
<li> Stopping the OSDs daemons
After you take an OSD out of the cluster, it may still be running. That is, the OSD may be up and out. You must stop your OSD before you remove it from the configuration :
<pre>
ssh {osd-host}
/etc/init.d/ceph stop osd.{osd-num}
</pre>
(Repeat the last command for all the OSDs on the machine.)
As a result, a "ceph -s" should show the OSDs as down.
</li>
<li> Removing the OSDs
##Remove OSDs from crush map :
<pre>
ceph osd crush remove osd.{osd-num}
</pre>
##Remove the OSD authentication key :
<pre>
ceph auth del osd.{osd-num}
</pre>
</li>
</ol>