AutomaticMachineTemplateGeneration
Goal
Up to now, to generate automatically hardware and profile templates for new machines (usually workernodes), we were using the Python script described here. Due to new naming conventions, we had to add new features to it. (To be precise : the initial script was re-written in Perl, and then the new features were added.)
Where to find the script ?
It is available in our local SVN script repository : http://mon.iihe.ac.be/scripting/ccq/generate_machine_templates/hardware_template_generator.pl
General syntax
usage: ./hardware_template_generator.pl OPTIONS
OPTIONS :
--template: name of the hardware template example (mandatory)
--desc: name of file with the descriptions
FORMAT: (NAME)? (MAC_ADDRESS)+ (IP_ADDRESS)? (SERIAL)? (LOCATION)? (TEMPLATE_NAME)?
--ip_gen: ipaddress to start the ip generation from
--name_gen: the naming generator
--serial_gen: the serialnumber generator
--location_gen: the location generator
--templatename_gen: the hardware template name generator
--domain: domain to use for nslookup (must begin with a dot; default is localhost domain)
--object: will create the object templates if the name of the machine can be determined: value is the name of the machine-type (eg wn).
--object_prefix: the name of the object template is made of this prefix and the fqdn (default prefix: profile_ ).
--os: OS description string (to be used in the os_version_db)
--wn_cpu: the number of jobs/cpu (to be used with variable WN_CPUS)
--outdir: output basedir (default /tmp/quattor)
--indir: input dir (default )
--help: print this
--debug: verbose output
Values that change for every machine (such as location, serial,...), can be specified "in extenso" in the description file, or can be generated in an incremental way thanks to the generators given in options.
To avoid ambiguities, we adopt the following convention : what is specified by the description file overrides the generators.
More details about generators
The generators must be surrounded by double-quotes, and the possible syntaxes are the following :
#python
[<string>]{<start_value>,<nb_values>,<step>[,<jump>]}*
or
[<string>]{<start_index>,<list>,<nb_values>,<step>[,<jump>]}*
or where
<start_value> : integer or string : the first element of the list
<start_index> : integer : index of the starting point in the list
<nb_value> : integer : number of elements in the list
<step> : integer (can be negative) : specifies the increment step
<jump> : integer : indicates that the increment will only be applied every <jump> element(s)
<list> : syntax : (<string>|<string>|...|<string>) : to use if you want to run over a pre-defined list of values instead of generating them by incrementation
Examples of generators :
soleil{1,2,5} will give : soleil1 soleil3 soleil5 soleil7 soleil9
soleil{1,5,1,2} will give : soleil1 soleil1 soleil2 soleil2 soleil3
soleil{a,5,1} will give : soleila soleilb soleilc soleild soleile
sn_{1,(a|b|c|d),5,1} will give : sn_a sn_b sn_c sn_d sn_a
sn_{1,(a|b|c|d),5,-2} will give : sn_a sn_c sn_a sn_c sn_a
The star (*) at the end of the two possible syntaxes indicates that the sequence [<string>]{...} can be repeated, as can be seen in the following example :
s{1,6,1,2}{1,(a|b),6,1} will give : s1a,s1b,s2a,s2b,s3a,s3b
If you want to play a bit and get more trained with these generators, you can use the script gen_incremented_string_list_v2.pl. For example :
./gen_incremented_string_list_v2.pl "sn_{10,8,1,4}{1,(a|b|c|d),8}"
will give the following output :
sn_10a sn_10a sn_10a sn_10a sn_11a sn_11a sn_11a sn_11a
Example
We have bought 7 new HP 2U boxes. Each box contains 4 nodes (24 cores per node), and each node has 2 network interfaces. The fqdn's of the boxes have already been registered in the local DNS. They will be named node21-X.wn (with X in the range 1...28). The serial number of a node is the serial number of the box followed by a letter (a,b,c,d) giving the location of the node in the box. Boxes are numbered starting from 1110. The mac addresses are in a file mac_lst, and the hardware template is hw.tpl. The hardware templates will be named after the serial number. For the location, we give the rack number, followed by the position in the rack, starting from 40. Here is the command to launch the bulk generation of hardware and profile templates :
./hardware_template_generator.pl --template=hw.tpl --desc=mac_lst --name_gen="node21-{1,28,1}.wn" --serial_gen="sn_{1110,28,1,4}{1,(a|b|c|d),28}" --location_gen="rack3_u{40,28,-1,2}"
--templatename_gen="hp_{1110,28,1,4}{1,(a|b|c|d),28,1}" --object=wn --os="sl550-x86_64" --wn_cpu=24
If you want to check the result of this command, here is what you should get for the first two boxes, under the form of a mapping table :
Node name IP address Serial num. Location Template name node21-1.wn.iihe.ac.be 192.168.21.1 sn_1110a rack3_u40 hp_1110a node21-2.wn.iihe.ac.be 192.168.21.2 sn_1110b rack3_u40 hp_1110b node21-3.wn.iihe.ac.be 192.168.21.3 sn_1110c rack3_u39 hp_1110c node21-4.wn.iihe.ac.be 192.168.21.4 sn_1110d rack3_u39 hp_1110d node21-5.wn.iihe.ac.be 192.168.21.5 sn_1111a rack3_u38 hp_1111a node21-6.wn.iihe.ac.be 192.168.21.6 sn_1111b rack3_u38 hp_1111b node21-7.wn.iihe.ac.be 192.168.21.7 sn_1111c rack3_u37 hp_1111c node21-8.wn.iihe.ac.be 192.168.21.8 sn_1111d rack3_u37 hp_1111d