AutomaticMachineTemplateGeneration

Goal

Up to now, to generate automatically hardware and profile templates for new machines (usually workernodes), we were using the Python script described here. Due to new naming conventions, we had to add new features to it. (To be precise : the initial script was re-written in Perl, and then the new features were added.)

Where to find the script ?

It is available in our local SVN script repository : http://mon.iihe.ac.be/scripting/ccq/generate_machine_templates/hardware_template_generator.pl

General syntax

usage: ./hardware_template_generator.pl OPTIONS

    OPTIONS :

        --template: name of the hardware template example (mandatory)
        --desc: name of file with the descriptions
                FORMAT: (NAME)? (MAC_ADDRESS)+ (IP_ADDRESS)? (SERIAL)? (LOCATION)? (TEMPLATE_NAME)?

        --ip_gen: ipaddress to start the ip generation from
        --name_gen: the naming generator
        --serial_gen: the serialnumber generator
        --location_gen: the location generator
        --templatename_gen: the hardware template name generator

        --domain: domain to use for nslookup (must begin with a dot; default is localhost domain)

        --object: will create the object templates if the name of the machine can be determined: value is the name of the machine-type (eg wn).
        --object_prefix: the name of the object template is made of this prefix and the fqdn (default prefix: profile_ ).

        --os: OS description string (to be used in the os_version_db)
        --wn_cpu: the number of jobs/cpu (to be used with variable WN_CPUS)

        --outdir: output basedir (default /tmp/quattor)
        --indir: input dir (default )

        --help: print this
        --debug: verbose output

Values that change for every machine (such as location, serial,...), can be specified "in extenso" in the description file, or can be generated in an incremental way thanks to the generators given in options.
To avoid ambiguities, we adopt the following convention : what is specified by the description file overrides the generators.

More details about generators

The generators must be surrounded by double-quotes, and the possible syntaxes are the following :

#python
    [<string>]{<start_value>,<nb_values>,<step>[,<jump>]}*

or

    [<string>]{<start_index>,<list>,<nb_values>,<step>[,<jump>]}*

or where

    <start_value>  : integer or string : the first element of the list
    <start_index>  : integer : index of the starting point in the list
    <nb_value>     : integer : number of elements in the list
    <step>         : integer (can be negative) : specifies the increment step
    <jump>         : integer : indicates that the increment will only be applied every <jump> element(s)
    <list>         : syntax : (<string>|<string>|...|<string>) : to use if you want to run over a pre-defined list of values instead of generating them by incrementation

Examples of generators :

    soleil{1,2,5}               will give : soleil1 soleil3 soleil5 soleil7 soleil9
    soleil{1,5,1,2}             will give : soleil1 soleil1 soleil2 soleil2 soleil3
    soleil{a,5,1}               will give : soleila soleilb soleilc soleild soleile
    sn_{1,(a|b|c|d),5,1}        will give : sn_a sn_b sn_c sn_d sn_a
    sn_{1,(a|b|c|d),5,-2}       will give : sn_a sn_c sn_a sn_c sn_a

The star (*) at the end of the two possible syntaxes indicates that the sequence [<string>]{...} can be repeated, as can be seen in the following example :

    s{1,6,1,2}{1,(a|b),6,1}           will give : s1a,s1b,s2a,s2b,s3a,s3b

If you want to play a bit and get more trained with these generators, you can use the script gen_incremented_string_list_v2.pl. For example :

./gen_incremented_string_list_v2.pl "sn_{10,8,1,4}{1,(a|b|c|d),8}"

will give the following output :

sn_10a
sn_10a
sn_10a
sn_10a
sn_11a
sn_11a
sn_11a
sn_11a

Example

We have bought 7 new HP 2U boxes. Each box contains 4 nodes (24 cores per node), and each node has 2 network interfaces. The fqdn's of the boxes have already been registered in the local DNS. They will be named node21-X.wn (with X in the range 1...28). The serial number of a node is the serial number of the box followed by a letter (a,b,c,d) giving the location of the node in the box. Boxes are numbered starting from 1110. The mac addresses are in a file mac_lst, and the hardware template is hw.tpl. The hardware templates will be named after the serial number. For the location, we give the rack number, followed by the position in the rack, starting from 40. Here is the command to launch the bulk generation of hardware and profile templates :

./hardware_template_generator.pl --template=hw.tpl --desc=mac_lst --name_gen="node21-{1,28,1}.wn" --serial_gen="sn_{1110,28,1,4}{1,(a|b|c|d),28}" --location_gen="rack3_u{40,28,-1,2}" 
--templatename_gen="hp_{1110,28,1,4}{1,(a|b|c|d),28,1}" --object=wn --os="sl550-x86_64" --wn_cpu=24

If you want to check the result of this command, here is what you should get for the first two boxes, under the form of a mapping table :

Node name                IP address      Serial num. Location     Template name
node21-1.wn.iihe.ac.be   192.168.21.1    sn_1110a    rack3_u40    hp_1110a
node21-2.wn.iihe.ac.be   192.168.21.2    sn_1110b    rack3_u40    hp_1110b
node21-3.wn.iihe.ac.be   192.168.21.3    sn_1110c    rack3_u39    hp_1110c
node21-4.wn.iihe.ac.be   192.168.21.4    sn_1110d    rack3_u39    hp_1110d
node21-5.wn.iihe.ac.be   192.168.21.5    sn_1111a    rack3_u38    hp_1111a
node21-6.wn.iihe.ac.be   192.168.21.6    sn_1111b    rack3_u38    hp_1111b
node21-7.wn.iihe.ac.be   192.168.21.7    sn_1111c    rack3_u37    hp_1111c
node21-8.wn.iihe.ac.be   192.168.21.8    sn_1111d    rack3_u37    hp_1111d