wiki:Usage/UserDocumentation

Version 10 (modified by sil, 10 years ago) (diff)

(re)moved docs about datastructure

User Documentation

CMT can be used with different user interfaces:

Web Interface
An easy-to-use, self explainable, interface based on Django Admin. You only need a webbrowser to use this interface.
Command Line Interface (CLI)
A less-easy-to-use interface which could be used to make mass-updates. The CLI offers the user some powerful functions which aren't possible on the web-interface. To use this interface you need to install CMT on the computer you're using.

Please keep in mind: Both interfaces are lacking of an undo-function, so be careful when you're about to make any changes in the data you've stored in CMT.

Web-interface

The web-interface can be opened in any regular web-browser. Ask your administrator for its address. Once you open the web-interface, a login-page should appear where you have to issue your credentials.

From there you'll be able to add objects to the database, explore objects in the database, change objects in the database, or delete objects from the database. It's meant as a intuitive easy-to-use interface, with advanced Create-Read-Update-Delete (CRUD) functionalities. It makes you able to quickly explore the data stored in CMT, or act on the data. The most important entities you can act on are:

Clusters
A cluster could be described as a collection of hardware, connected to each other through one or more networks.
Racks
A rack is a cabinet that houses hardware like servers and switches. It has multiple slots to mount hardware. These slots identify the physical location of your hardware.
Hardware
Hardware units like (virtual) servers, switches, PDU's, etc. It could be 'any piece of hardware with a powercable attached to it'.
Networks
Networks defined by their (IP) address space. The addresses in the address space of a network each are related to an interface.
Interfaces
Network interfaces for input/output, thus any kind of card or (virtual) adapter which can have a network-cable plugged into it, and has a hardware address (like a MAC-address). Most obvious example is a regular 100/1000Mbit NIC, but it could be something like an InfiniBand interface card, or an out-of-band management adapter. Many network interfaces are related to a single piece of hardware, and a single network.

Command Line Interface

To use the Command Line Interface has to be installed on your system. If you issue the command cmt you'll get the usage info:

user@host$ cmt
Usage: cmt [options]

This is the commandline interface of the SARA package SARA CMT.

Options:
  -h, --help            show this help message and exit
  -n, --dry-run         This flag has to be given before -[aclmr]
  --script              
  -a ENTITY, --add=ENTITY
                        Add an object of the given ENTITY. Extra arguments
                        should be given like `set ASSIGNMENTS`, where
                        ASSIGNMENTS are formatted like `ATTR=VAL`
  -c ENTITY, --change=ENTITY
                        Change value(s) of object(s) of the given ENTITY.
                        Arguments should be given like `get QUERY set
                        ASSIGNMENTS`, where both QUERY and ASSIGNMENTS consist
                        out of one or more terms formatted like `ATTR=VAL`.
                        Only objects matching QUERY will be changed according
                        to the given ASSIGNMENTS
  -g TEMPLATE, --generate=TEMPLATE
                        Render the given TEMPLATE, and store the result if
                        specified
  -l ENTITY, --list=ENTITY
                        List object(s) of the given ENTITY. Arguments should
                        be given like `get QUERY`, where QUERY consists out of
                        one or more terms formatted like `ATTR=VAL`
  -r ENTITY, --remove=ENTITY
                        Remove object(s) of the given ENTITY. Arguments should
                        be given like `get QUERY`, where QUERY consists out of
                        one or more terms formatted like `ATTR=VAL`. Only
                        objects matching QUERY will be listed
  -v, --verbose         

More information about the options can be read in the following subsections. For each option we'll give one or more use cases as an example.

CRUD options

Just like the web-interface, the CLI can be used for CRUD-functionalities (Create, Read, Update, Delete) too. However, the implementation of the CLI is much more powerful, since it can be used to do things with multiple objects at once, and can be done from shell-scripts.

Add object: -a, --add

Use this option to add a new object of a given entity to the database. You have to specify the entity and assign values to its fields by providing assignments as extra arguments. The syntax is as follows:

cmt --add <entity> <assignments>

Use case

You want to add a new server, and its interfaces

As an example we'll add a server (PowerEdge M610) to the 1st position in rack 1 of the Lisa-cluster. The server will be used as a test- and computenode, and has the following interfaces:

Interface network MAC-address
Gigabit lisa admin 00:00:00:00:00:01
InfiniBand lisa infiniband 00:00:00:00:00:02
iDRAC6 lisa consoles 00:00:00:00:00:03
user@host$ cmt --add hardwareunit cluster=Lisa role=computenode role=testnode specifications='PowerEdge M610' rack=1 first_slot=1

user@host$ cmt --add interface host=r1n1 network='lisa admin' iftype='Gigabit' hwaddress='00:00:00:00:00:01'

user@host$ cmt --add interface host=r1n1 network='lisa infiniband' iftype='InfiniBand' hwaddress='00:00:00:00:00:02'

user@host$ cmt --add interface host=r1n1 network='lisa consoles' iftype='iDRAC6' hwaddress='00:00:00:00:00:03'

List object(s): -l, --list

Use this option to list existing objects in the database. You have to specify the entity, and filter objects you want to list by providing a query. The syntax is as follows:

cmt_sara --list <entity> [get] <query>

Use case

If you need to figure out where a machine -- in this case test3 -- is physically hosted:

When an engineer has to fix a node you need to tell him the location of the node. This information shall be listed by issuing the following command:

user@host$ cmt --list hardwareunit label=test3

The output of this command looks like:

 .---[  test3  ]---
 : cluster        : LDAP/CUA
 : address        : Amsterdam - Science Park 121
 : room           : S142A
 : rack           : rack AD38
 : first_slot     : 23
 : specifications : PowerEdge 1950 (Dell bv)
 : roles          : administration
 : in_support     : True
 '---

Change object(s): -c, --change

Use this option to change fields of one or more objects in the database. You have to specify the entity, filter objects you want to change by providing a query, and assign new values to their fields by providing assignments as extra arguments. The syntax is as follows:

sara_cmt --change <entity> [get] <query> set <assignments>

Use case

You have to change an existing address:

user@host$ cmt --change address get address='Kruislaan 415' set address='Science Park 121' postalcode='1098 XG'

Or maybe a motherboard has been replaced, so the MAC-addresses of the onboard interfaces have been changed:

Interface Old MAC New MAC
Gigabit 00:23:AE:FD:C9:00 00:26:B9:F9:95:34
iDRAC6 00:23:AE:FB:C7:DD A4:BA:DB:00:31:F6
user@host$ cmt --change interface get hwaddress=00:23:AE:FD:C9:00 set hwaddress=00:26:B9:F9:95:34

user@host$ cmt --change interface get hwaddress=00:23:AE:FB:C7:DD set hwaddress=A4:BA:DB:00:31:F6

In both commands you'll query for the interface with the old MAC-address, and overwrite the value of the hwaddress-field with the new MAC-address.

Since you know in which machine the motherboard has been replaced -- in this case r34n26 -- it might be easier to issue a command like:

user@host$ cmt --change interface get host=r34n26 iftype=Gigabit set hwaddress=00:26:B9:F9:95:34

user@host$ cmt --change interface get host=r34n26 iftype=iDRAC6 set hwaddress=A4:BA:DB:00:31:F6

This will only work when the machine contains a single Gigabit-interface, as well as a single iDRAC6-interface. In this case you don't need to know the old MAC-addresses.

You want to add the service-tags to new Dell-hardware:

For example new chassis in rack 2, 13, and 36:

user@host$ for rack in 2 13 36
> do
>   for cmc in a b
>   do
>     svctag=`sararac -c getsvctag dr-r${rack}cmc-${cmc} | sed -rn 's/^Chassis.*([A-Z0-9]{7})$/\1/p'`
>     echo saving warranty tag ${svctag} to r${rack}cmc-${cmc}
>     cmt --script --change hardwareunit get label=r${rack}cmc-${cmc} set warranty_tag=${svctag}
>   done
> done

And all hardware in these chassis:

user@host$ for rack in 2 13 36
> do
>   for node in `seq 1 32`
>   do
>     svctag=`sararac -c getsvctag dr-r${rack}n${node} | sed -rn 's/^.*([A-Z0-9]{7})$/\1/p'`
>     echo saving warranty tag ${svctag} to r${rack}n${node}
>     cmt --script --change hardwareunit get rack=${rack} first_slot=${node} set warranty_tag=${svctag}
>   done
> done

Remove object(s): -r, --remove

Use this option to remove objects from the database. You have to specify the entity, and filter objects you want to remove by providing a query. The syntax is as follows:

sara_cmt --remove <entity> [get] <query>

Use case

You have removed a server from a rack:

For example the first node in rack 2:

user@host$ cmt --remove hardwareunit rack=2 first_slot=1
INFO     remove 233 :: Found 1 objects matching query: r2n1
Are you sure? [Yn]

As you can see there will be asked for confirmation by default.

Other options

Dry-run: `-n, --dry-run

Use this option to suppress (CRUD-)actions that make changes to the database. Please note, this option has to be given before CRUD-options. So, use it like:

user@host$ cmt --dry-run --{add,list,change,remove} ENTITY ...

This option can become handy when you first want to test a command that'll have a big impact.

Use case

You're replacing all compute-nodes in a couple of racks (2-5), and want to test your command to remove the old hardware from CMT:

user@host$ cmt --dry-run --remove hardwareunit rack={2,3,4,5} role=computenode

Generate a config-file: -g, --generate

Render a template to generate a config-file or a report. Use the following syntax:

sara_cmt --generate <templatename>

By default your templates can be found in your (virtual) environment in de directory etc/cmt/templates.

Use case

When you want to generate the cfrun_hosts file of the Clearspeed cluster:

user@host$ cmt --generate cfrun_hosts.clearspeed

Which generates the cfrun_hosts file and stores the following content at /usr/sara/etc/cfengine/rc.hosts.clearspeed:

#
# This file is automagicly generated by CMT
#
#             SARA - Computing and Networking Services
# Date      : Tue, 15 Jun 2010
# Version   : CMT CLI 1.0
# Generated : Thu, 15 Jul 2010 09:20:27 +0200
#
# SVN:
#       $Id:$
#       $URL:$
#


outputdir=/data/cfengine/output/clearspeed
maxchild=8
login1
gb-r50n1.irc.sara.nl
gb-r50n2.irc.sara.nl
gb-r50n3.irc.sara.nl
gb-r50n4.irc.sara.nl
gb-r50n5.irc.sara.nl
gb-r50n6.irc.sara.nl
gb-r50n7.irc.sara.nl
gb-r50n8.irc.sara.nl
gb-r51n1.irc.sara.nl
gb-r51n2.irc.sara.nl
gb-r51n3.irc.sara.nl
gb-r51n4.irc.sara.nl
gb-r51n5.irc.sara.nl
gb-r51n6.irc.sara.nl
gb-r51n7.irc.sara.nl
gb-r51n8.irc.sara.nl

CMT Templates

CMT offers a powerful feature, powered by templates. With these templates you're able to let CMT generate various kind of files, like configuration files or reports about your clusters.

The syntax of the templates in CMT is based on Django templates, therefore the documentation of the Django template language can be useful as a reference. Django templates are usually meant for web pages, but since we use it for our configuration files some functionalities have been added for CMT-specific needs. By extending the Django template language, it has been made possible to:

  • Define an epilogue script
  • Pre-load datasets from database
  • Set a path to write the output to

To make use of these functionalities you have to start your template with

{% load cmt_client %}

and store the file in your template-directory as <name>.cmt

Epilogue script

Some configuration files need to be reloaded by the program when they are modified. Or maybe some other things should be done to activate the new configuration. This is where an epilogue script comes in. You can write the epilogue script in the template itself, between the epilogue and endepilogue tag:

{% epilogue %}
# this epilogue-script will restart the DHCP server
/etc/init.d/dhcp3-server restart
{% endepilogue %}

Pre-load datasets from database

You may define collections of objects to use in your templates. These collections can be defined with a filter, and assigning it to a variable name to use.

# syntax for the use-tag is:
#  use <entity> with <filter> as <variable name>
{% use network with 'name=lisa consoles' as net_consoles %}
{% use interface with 'network__name=lisa consoles' as ifaces_consoles %}

In the example two collections are assigned to a variable:

  • net_consoles will contain the network with the name lisa consoles
  • ifaces_consoles will contain all interfaces in that network

Now these collections can be used in the template, to build a part of a DHCP config-file for example:

group { # Start Console network
    option domain-name "{{net_consoles.domain}}";

    {% for iface in ifaces_consoles %}
    {% if iface.hwaddress != None %}
    host {{iface.fqdn}} {
        hardware ethernet {{iface.hwaddress}};
        fixed-address {{iface.ip}};
        option host-name "{{iface.fqdn}}";
    }
    {% endif %}
    {% endfor %}
} # End Console network

Set a path to write the output to

To save the result of your template, you can store the output to a file. This can be done by using the store-tag, and give it a path:

# syntax for the store-tag is:
#  store <outputpath>  as output
{% store /etc/dhcp/dhcpd.conf as output %}

In this example the result will be written to /etc/dhcp/dhcp.conf.