Deploying a new Ceph Octopus cluster

Before you start

Very important remark if you reinstall a ceph cluster: make sure to power off the nodes from a previous cluster deployment, otherwise the mon nodes will spawn a mon on the first machine to reach the quorum and the boostrap will fail! (The other nodes are not aware that you reinstall a new cluster.)

Another important remark: for all the Ceph commands, you must type them in the ceph shell that you launch with:

# cephadm shell

Yet another remark: when you run the command 'hostname' on the machines of our cluster, you get the FQDN of the machine, ie the name followed by the domain name. Ceph will always rely on this, meaning that when you need to specify the name of a machine in a Ceph command, you must always use the FQDN, and not the short name. That explains why we have to add the flag

--allow-fqdn-hostname

in the ceph bootstrap command below.

Bootstrap a new cluster

• Reinstall ceph1 that will be the ceph adm machine, ie the machine that will be used to bootstrap the ceph cluster and deploy all the other ceph node. The object template need contain the line:

include 'config/ceph/adm';

• Once the machine is quattor-installed, log in to it and do the following:

# mkdir -p /etc/ceph
# cephadm bootstrap --mon-ip *<ceph1-ip>* --allow-fqdn-hostname

Take note of the "admin" password that is generated, you'll need it to connect to the Ceph Dashboard.

Add other hosts to the cluster

• The other nodes of the Ceph cluster must be installed with Quattor with the following include in their object template:

include 'config/ceph/nodes';

• Copy the ssh pubkey of the ceph adm in the authorized_keys of other hosts: do it with Quattor (see config/ceph/nodes)

• Tell Ceph that new nodes are part of the cluster:

# ceph orch host add <new_host>

For example:

# ceph orch host add ceph2.wn.iihe.ac.be
# ceph orch host add ceph3.wn.iihe.ac.be