View source for To Restart The T2 After Power Down


When All the systems have been powered off, a clear procedure needs to be followed to restart all the systems. <br>
As an improper order of restarting the servers would be useless, many of the servers will not restart automatically after an unscheduled power down. Intervention is always needed in this case.

=== 1 ===
Put Bridge back online. The name of this PC is bridge3, but it is not in the DNS. <br>
It's IP number is 193.190.247.129<br>
There are a lot of interfaces on this PC:<br>
ifconfig gives: bond0  br0 -> 193.190.247.129 <br>
                br1 <br>
                eth[1-5]
=== 2 ===
Put ccq back online. You need to start it manually from the front. <br>
Log in and check the virtual hosts:

<pre>
xm list -> Domain 0
           ccq3        -b----
</pre>
check whether you can ssh to ccq3


=== 3 ===
Start up fs. See if the virtual host mon.iihe.ac.be is running (via xm list)

=== 4 ===
restart gridce and gliterb who are virtual hosts on dom01<br>
if gridce or gliterb are not present, execute the following:

<pre>
xm create gridce.iihe.ac.be
xm create gliterb.iihe.ac.be
</pre>
IMPORTANT: on gridce do: <br>
cp /etc/hosts.equiv2 /etc/hosts.equiv<br>


log into these virtual machines via ssh and see if /grid_mnt/pooluser is mounted.

=== 5 ===
look if you see maite via the web (http://maite.iihe.ac.be:2288 <br>
id pnfs mounted here?

=== 6 === 
start up frontier from the front panel.

=== 7 ===
Also log into the pools. look if they are all up and seen by the network (do an ssh to the different pools. For the moment, these are available:

<pre>
behar
behar2
behar3
behar4
behar6
behar020
behar021
behar022
behar023
behar024
</pre>

In case of doubt, they are labeled.
=== 8 === 
due to a wrong reboot order still present (wn and m0, m1) it is possible that /pnfs/iihe is not correctly mounted on these machines. <br>
You can mount is easily via "mount /pnfs/iihe", but since there are a lot of wn's, this is impractical. Therefore, there are scripts that login to the different machines and execute a command on it.

<pre>
cd distrib_scripts
./distrib_exec_list <list_file> "<command>"
</pre>
=== 9 ===
It is also important to see whether m0 and m1 are reachable from the lab. The access is done via tuncc.iihe.ac.be <br>
On this machien, the routing should be fine. the way back is done via ccq. here look at  /etc/rc.local and you might need to execute

<pre>
/root/route.sh
</pre>
 
'''Remark''' <br>

Sometime you will have to rerun '/root/tun.sh cc' on ccq to make sure the tunnel is correctly set.



{{TracNotice|{{PAGENAME}}}}