Getting started with Keen

What we will build

Our first program is a collector of 27 lines. It asks a website for its page, reads the response code and the response time, then judges those two numbers. We will write it line by line, and stop on each one.

When it runs, the probe’s page shows the OK state, the response code (200, for instance) and the response time in milliseconds. If the site returns an error, answers too slowly or stops answering, the state changes and a sentence says why.

You do not need to know Keen, or how to program. Each notion will be explained when we need it.

Why Keen

Keen is the language of banshee’s collectors. It looks like PHP cut down to the essentials, without the spellings that invite confusion. banshee built it for four reasons.

• A probe can be read. Anyone can read it and understand it, easily.
• No dependency on your servers. With scripts, each check brings its own prerequisites, to install and maintain on every machine. A Keen collector brings none.
• Full control and one way of working. Everything a probe does to its target is written in plain sight in its code. Every author writes the same way, so collectors can be shared across the community with confidence.
• Less risk. Fewer security holes and fewer abandoned servers: the kind of monitoring everyone dreads comes to an end.

Five words before we start

First, the vocabulary. These five words come back everywhere.

The master is banshee’s central server, the one you open in your browser. It keeps the collectors, the configuration and the results.

The poller is a small program installed as close as possible to what you monitor. It runs the checks. It is the poller that opens the connection to the master.

A host is a machine, a device or a site that you monitor. It has its own page in banshee’s inventory.

A collector is the recipe of a check. It is a program written in Keen and kept on the master. It says how to query something, the target (here, a website), and which values to take from it.

A probe is a collector placed on one particular host, with its settings. The collector is the recipe, the probe is the dish prepared for that host.

Where to write the code

You need a running banshee master and an account allowed to write collectors, that is, with write permission on the root / of the tree. For the final trial, you also need a poller connected to that master, a host that it collects and the address of a site to check. An HTTP check needs neither a password nor a key.

Open the Collectors screen (Collecteurs) and click “New collector” (Nouveau collecteur). Type the name web/https/status, leave the code empty and confirm. A name is a path whose segments are separated by /. Folders exist only through the names they contain: web and web/https appear by themselves.

Open the collector’s page, click “Edit” (Éditer) and choose the “Code” file. This is where we will write the code. Under the text, the “Problems” area (Problèmes) reads the code again after each pause in your typing. When everything is correct, it shows “No problems.” (Aucun problème.).

Step 1: ask the site for its page

Let’s start with the simplest gesture: asking the site for its page. Your browser does exactly that when you open an address. This request is called a GET request.

Type these lines in the “Code” file:

need('protocol:http');

// Ask the site for its page.
$answer = http.get('https://www.example/');

The first line, need, announces that the collector uses the HTTP protocol. The master refuses an http.get call without it. We will see in step 7 what it brings.

Then comes a comment. Everything after //, up to the end of the line, is ignored by banshee. It is there only for the person who will read the code later. Get into the habit of writing them.

The last line does the work. http.get is a function, an action that has a name. This name has two pieces joined around a dot, with no space: http is the domain, the protocol used, and get is the method, what to do. Almost all of Keen’s functions are written this way.

Between the parentheses, we give the function what it needs: the address of the site. What we give a function is called an argument. An address is text, and text is written between apostrophes. In step 3, we replace this address with a setting.

On the left, $answer = stores the response in a variable. A variable is a box that has a name. The name starts with $, followed by lowercase letters, digits or _. There is no need to create the box in advance: it exists as soon as we store something in it with =.

The semicolon ends the instruction. Each one needs it.

What does $answer hold? An array, a series of boxes that each have a label. The one that http.get returns has four:

• status: the site’s response code, a number. 200 means all is well, 404 means the page does not exist.
• headers: the headers of the response.
• body: the text of the page.
• time: the response time, in milliseconds.

If the site does not answer at all, http.get stops the collector by itself. We have nothing to write for that. Step 4 shows what the probe says about it.

Step 2: return the response code

For now, the response stays locked inside the collector. Let’s bring the response code out, so that banshee can display it and judge it.

A collector hands over its results as returned values. Returning one takes two gestures: announce it at the head of the code, then return it at the right moment. Add the first line under the need line, before the comment, and the second at the end, under the $answer line:

output('code', 'number', array('label' => 'Response code'));

value('code', $answer['status']);

output announces the value. It is a declaration, and declarations always come before the rest of the code: the master refuses a declaration placed lower down.

'code' is the name of the value, which will be used to find it again. 'number' is its sort, here a number. The other possible sort is 'text'.

The last argument, array('label' => 'Response code'), is an array like the one in the response, but written by us. It holds a single pair, which reads “label => content”. The label label holds the caption that the screen shows in front of the value.

value('code', $answer['status']) returns the value: first the name announced above, then the value itself.

$answer['status'] fetches from the array the box labeled status. We write the name of the variable, then the label between square brackets and apostrophes.

An announced value must be returned once, and only once, on each run. A run is one complete execution of the collector. Otherwise the collector stops, and step 4 says in which state.

Here is the complete code at this point:

need('protocol:http');

output('code', 'number', array('label' => 'Response code'));

// Ask the site for its page.
$answer = http.get('https://www.example/');
value('code', $answer['status']);

Step 3: make the address a setting

An address written in the code forces us to make one collector per site. It is better if each probe carries its own: a single collector then serves as many sites as you like.

Add this block to the declarations, above the output lines, then replace the $answer line with the second line:

$url = variable('web:url', 'text', array(
    'label' => 'Address to check',
    'help'  => 'For example https://www.example/.',
));

$answer = http.get($url);

variable declares a setting that the probe provides. On the probe, it becomes a field to fill in. The collector’s page lists these settings under the title “Arguments” (Arguments).

$url = stores in a variable the value of this setting for the probe’s host. On each run, $url therefore holds the address that the administrator typed.

'web:url' is the name of the setting. It has at least two parts, separated by :, in lowercase letters. 'text' is its form: a text field.

The last argument is an array of two pairs, written over several lines. In Keen, line breaks and spaces change nothing: they are there for reading. label is the text shown in front of the field, help the sentence that explains it.

In http.get($url), the argument is now the variable. A variable is written without apostrophes; with apostrophes, it would be the text $url.

This setting is required. A probe without an address cannot work, and banshee makes it say so: step 4 shows how.

Here is the complete code at this point:

need('protocol:http');

$url = variable('web:url', 'text', array(
    'label' => 'Address to check',
    'help'  => 'For example https://www.example/.',
));

output('code', 'number', array('label' => 'Response code'));

// Ask the site for its page.
$answer = http.get($url);
value('code', $answer['status']);

Step 4: judge the response code

So far, the collector measures. Let’s teach it to say whether the measurement is good.

That is the job of a watch: a rule of judgment that the collector proposes, for instance “the response code must be 200”. The collector does not judge by itself. On the probe, the administrator ticks the watches they want, and the poller judges each result.

A watch has thresholds, the limits at which it triggers. The code writes a proposed value, and the administrator can change it on the probe without reading the code.

The result of the judgment is a state, a word that sums up the probe’s situation. We meet them one by one.

Add this line under the output lines:

watch.expect('code', 'Wrong response', 'code', array('critical' => array(200)));

watch.expect declares a watch of the kind “Expected value”. It triggers when the value is none of those we expect.

It takes four arguments. The first, 'code', is its identifier: the probe remembers under this name whether the administrator ticked it. It can be identical to the name of the value, because they are two distinct things. The second, 'Wrong response', is the title that the administrator reads. The third, 'code' again, designates the watched value, the one that step 2 announced. The fourth is the array of options.

'critical' => array(200) says that only 200 is expected. array(200) is a list of a single box, here a number, written without apostrophes. critical is the critical level, the most serious. Any value missing from the list triggers it.

That 200 is a proposed threshold. The administrator can change it on the probe, or empty it. An empty threshold means that this level is not judged.

OK

The OK state means all is well: no ticked watch triggers. Here, the response code is 200.

CRITICAL

The CRITICAL state means critical. Here, the response code differs from 200. A redirect (301, for instance) also gives CRITICAL, because http.get does not follow it. So does a server error (500, for instance): the site did answer, and the collector receives that answer as a value.

UNREACHABLE

The UNREACHABLE state means that the site cannot be reached. It appears when the site does not answer at all: timeout, connection refused, unknown name. http.get then stops the collector itself, and the probe takes this state. No line of code is needed. No value is returned and no watch is judged.

MISCONFIGURED

The MISCONFIGURED state means badly configured. It appears when the collector cannot work because of a setting. That is the case of a probe without an address, since this setting is required. It is also the case of an announced value that is not returned, or returned twice.

Each state comes with a sentence that says why.

Here is the complete code at this point:

need('protocol:http');

$url = variable('web:url', 'text', array(
    'label' => 'Address to check',
    'help'  => 'For example https://www.example/.',
));

output('code', 'number', array('label' => 'Response code'));

watch.expect('code', 'Wrong response', 'code', array('critical' => array(200)));

// Ask the site for its page.
$answer = http.get($url);
value('code', $answer['status']);

Step 5: measure the response time

A site can answer 200 and take five seconds to do it. Let’s add the response time, with its watch.

Three lines arrive, one in each group: the first with the output lines, the second with the watch lines, the third at the end of the code.

output('response-time', 'number', array('label' => 'Response time', 'unit' => 'ms'));

watch.above('slow', 'Slow site', 'response-time', array('warning' => 500, 'critical' => 2000, 'for' => '3 runs'));

value('response-time', $answer['time']);

The output line announces a second value, on the model of step 2. The array of options now has two pairs, separated by a comma. 'unit' => 'ms' gives the unit, which the screen shows after the number. The hyphen in 'response-time' is allowed in the name of a value.

The value line returns that value. $answer['time'] reads the time box of the response, the duration in milliseconds.

watch.above declares a watch of the kind “High threshold”. It triggers when the value goes above a threshold. Its arguments follow the same order as in step 4: the identifier 'slow', the title 'Slow site', the watched value 'response-time', then the options.

'warning' => 500 sets the first threshold. Above 500, the state becomes WARNING, a level less serious than critical. The threshold is crossed strictly: exactly 500 triggers nothing. The number is in the unit of the watched value, here milliseconds.

'critical' => 2000 sets the second threshold. Above 2,000 milliseconds, the state becomes CRITICAL.

'for' => '3 runs' sets the duration. The text is written in English in the code: a number, a space, then runs. It asks for the threshold to be crossed on three runs in a row before the state changes. A single slow response triggers nothing.

A probe carries the most serious state of its watches. If the code is 200 but the time exceeds 2,000 ms for three runs, the probe is CRITICAL.

Here is the complete code at this point:

need('protocol:http');

$url = variable('web:url', 'text', array(
    'label' => 'Address to check',
    'help'  => 'For example https://www.example/.',
));

output('code', 'number', array('label' => 'Response code'));
output('response-time', 'number', array('label' => 'Response time', 'unit' => 'ms'));

watch.expect('code', 'Wrong response', 'code', array('critical' => array(200)));
watch.above('slow', 'Slow site', 'response-time', array('warning' => 500, 'critical' => 2000, 'for' => '3 runs'));

// Ask the site for its page.
$answer = http.get($url);
value('code', $answer['status']);
value('response-time', $answer['time']);

What each situation gives:

• code 200 and a response time of 500 ms or less: OK;
• response time above 500 ms for three runs: WARNING;
• response time above 2,000 ms for three runs: CRITICAL;
• code other than 200: CRITICAL;
• site that does not answer: UNREACHABLE;
• no address on the probe: MISCONFIGURED.

Step 6: look for a word in the page

So far, everything the collector judges goes through watches that the administrator sets. Let’s see how the code can decide by itself.

A site can answer 200 while displaying an error page. The response code does not reveal it, but the text of the page does: it is the body box of the array. We look for a word in that page and, if it is missing, declare the probe critical. The word is a second setting of the probe, optional, because not every site needs it.

Add the first block under the declaration of $url, and the second under the $answer line:

$word = variable('web:word', 'text', array(
    'label'    => 'Word to find in the page',
    'help'     => 'Optional. If the page does not contain this word, the probe is critical.',
    'required' => false,
));

if ($word != '' && !text.contains($answer['body'], $word)) {
    fail('critical', 'Word missing', 'The page does not contain the word ' . $word . '.');
}

The setting follows the form of step 3. The novelty is 'required' => false. A setting is required by default; this option makes it optional. false is a boolean, a value that is true or false. Its two spellings are true and false, without apostrophes. When the probe gives no word, the variable $word holds an empty text, ''.

if (…) { … } runs the code between braces only if the condition between parentheses is true. The braces are required, even for a single instruction.

The condition has two parts. $word != '' is the first. The != sign means “is different from”, and two apostrophes side by side write an empty text. It reads “a word has been filled in”.

&& means “and”: both parts must be true. The right side is not even read when the left side is false. Its counterpart is ||, “or”.

text.contains($answer['body'], $word) calls a function of the text domain. It returns true if the first text contains the second. Here, it looks for the word in the page.

The ! in front of it reverses the result. The whole condition therefore reads “a word is requested and the page does not contain it”.

fail stops the collector on the state we choose. The first argument is that state, written as is: 'critical', 'misconfigured' or 'unreachable'. The second is the summary of the message, the third its detail. A collector stopped by fail returns no value, and no watch is judged. The code alone decides this state: the administrator sets no threshold on it.

In the detail of the message, the dot . joins texts together. This operation is called concatenation. The dot is written with a space on each side, as in '…the word ' . $word . '.'.

Here is the complete code at this point:

need('protocol:http');

$url = variable('web:url', 'text', array(
    'label' => 'Address to check',
    'help'  => 'For example https://www.example/.',
));
$word = variable('web:word', 'text', array(
    'label'    => 'Word to find in the page',
    'help'     => 'Optional. If the page does not contain this word, the probe is critical.',
    'required' => false,
));

output('code', 'number', array('label' => 'Response code'));
output('response-time', 'number', array('label' => 'Response time', 'unit' => 'ms'));

watch.expect('code', 'Wrong response', 'code', array('critical' => array(200)));
watch.above('slow', 'Slow site', 'response-time', array('warning' => 500, 'critical' => 2000, 'for' => '3 runs'));

// Ask the site for its page.
$answer = http.get($url);

if ($word != '' && !text.contains($answer['body'], $word)) {
    fail('critical', 'Word missing', 'The page does not contain the word ' . $word . '.');
}

value('code', $answer['status']);
value('response-time', $answer['time']);

Step 7: set up the protocol

An HTTP request has its own settings, which banshee already knows. There is no need to declare them again with variable: the collector asks for them by name.

That is the job of the line written in step 1, at the very top of the code:

need('protocol:http');

need asks for settings that already exist in banshee. 'protocol:http' designates those of the HTTP protocol: the request’s timeout, the fingerprint of a pinned certificate, the name under which the request introduces itself to the site (the User-Agent) and the way the encrypted connection is judged (the TLS profile). They are optional, and each appears on the probe as a field.

A collector receives only the settings it declares: without this line, those fields would not exist on the probe, and the request would follow the default TLS profile instead of the one the administrator set. That is why the master refuses an http.get call without it. need is a declaration, like variable and output, so it stays in the header.

Our collector is finished. Here it is in full, with its 27 lines:

need('protocol:http');

$url = variable('web:url', 'text', array(
    'label' => 'Address to check',
    'help'  => 'For example https://www.example/.',
));
$word = variable('web:word', 'text', array(
    'label'    => 'Word to find in the page',
    'help'     => 'Optional. If the page does not contain this word, the probe is critical.',
    'required' => false,
));

output('code', 'number', array('label' => 'Response code'));
output('response-time', 'number', array('label' => 'Response time', 'unit' => 'ms'));

watch.expect('code', 'Wrong response', 'code', array('critical' => array(200)));
watch.above('slow', 'Slow site', 'response-time', array('warning' => 500, 'critical' => 2000, 'for' => '3 runs'));

// Ask the site for its page.
$answer = http.get($url);

if ($word != '' && !text.contains($answer['body'], $word)) {
    fail('critical', 'Word missing', 'The page does not contain the word ' . $word . '.');
}

value('code', $answer['status']);
value('response-time', $answer['time']);

Step 8: read the master’s faults

Everyone writes faults. Let’s cause a few on purpose, to learn to read them.

Keen is stricter than PHP, deliberately. The master reads the code again after each pause in your typing, and one last time when you save. It refuses anything that invites confusion, and code that has a fault cannot be saved. Each fault gives three pieces of information: the line, the column and the reason. The application shows the reasons in French; they are given here in English.

A function that does not exist

Add this line at the end of the code:

$f = file_get_contents('/etc/passwd');

The “Problems” area shows line 28, column 6 and the reason: The function “file_get_contents” does not exist in a collector. Keen knows only the functions on its list. None reads a file on the poller, launches a program or reaches the network, apart from the protocol functions such as http.get. Remove the line.

A spelling that Keen removes from PHP

Now add this line instead:

$speed = $answer['time'] > 500 ? 'slow' : 'normal';

The reason: “? :” does not exist in a collector: write an if / else. Keen removes a few PHP spellings that duplicate another spelling or read badly, and the fault always says what to write instead. Remove the line.

A badly spaced dot

$label = 'code '.$answer['status'];

The reason: A concatenation is written with a blank on each side of the dot. The dot has two meanings. Stuck to two names, it writes a function call, like http.get. Surrounded by blanks, it joins two texts. Remove the line.

A variable read before it is filled

value('code', $reply['status']);

The reason: “$reply” is not assigned anywhere before. In our code, the variable is called $answer; $reply never received a value. A variable must have received a value higher up in the code before it is read. Remove the line.

What Keen removes from PHP

Each removed spelling has its replacement:

• ? : is written if … else …;
• += and ++ are written $i = $i + 1;
• [1, 2] is written array(1, 2);
• null is written with a value, or '' for “nothing”;
• # and /* … */ are written //.

Two other rules also cause faults. A name is written in lowercase: $Answer is refused. And a declaration (need, variable, output, watch) placed after an instruction is refused.

Step 9: try the collector

It remains to run the collector for real. It takes four screens, in order.

In the editor, type an optional comment and click “Save” (Enregistrer). The master checks the code one last time. Each save creates a version, a numbered copy of the code that never changes. The latest one is the current version, the one that probes use. Nothing is erased: going back means designating an older version. Click “Close” (Fermer) to return to the collector’s page.

In the inventory, open the host and add a probe that uses the collector web/https/status. Fill in “Address to check”. Keep “Wrong response” and “Slow site” ticked, as they are by default, and set their thresholds if needed.

On the poller’s machine, an administrator has the poller take the collector with the --update-collectors option, followed by the collector’s name, for example --update-collectors=web/https/status*. The poller shows the changes and waits for confirmation.

Finally, open the probe’s page. As long as no result has arrived, it is in the PENDING state, waiting. Then it takes the state that the watches give. To see the other states, change a threshold on the probe, or give it an address that does not exist.

In short

The complete collector:

need('protocol:http');

$url = variable('web:url', 'text', array(
    'label' => 'Address to check',
    'help'  => 'For example https://www.example/.',
));
$word = variable('web:word', 'text', array(
    'label'    => 'Word to find in the page',
    'help'     => 'Optional. If the page does not contain this word, the probe is critical.',
    'required' => false,
));

output('code', 'number', array('label' => 'Response code'));
output('response-time', 'number', array('label' => 'Response time', 'unit' => 'ms'));

watch.expect('code', 'Wrong response', 'code', array('critical' => array(200)));
watch.above('slow', 'Slow site', 'response-time', array('warning' => 500, 'critical' => 2000, 'for' => '3 runs'));

// Ask the site for its page.
$answer = http.get($url);

if ($word != '' && !text.contains($answer['body'], $word)) {
    fail('critical', 'Word missing', 'The page does not contain the word ' . $word . '.');
}

value('code', $answer['status']);
value('response-time', $answer['time']);

What we learned by writing it:

• Keen code begins with its declarations, then comes the rest. Each instruction ends with a semicolon, and // opens a comment.
• A variable is written $name and receives a value with =. Values are texts, numbers, booleans and arrays.
• An array is written array(…), as a list or as label => content pairs, and is read with $a['label'].
• A function is written domain.method. Seven words, including value and output, are written alone.
• need, variable, output and watch.… declare what the collector asks for, returns and watches. value returns a value, and fail stops the collector on a state.
• if, comparisons, &&, || and ! are used to decide. The dot surrounded by spaces joins texts.
• A probe is OK, WARNING, CRITICAL, UNREACHABLE or MISCONFIGURED. The master reports each fault with its line, its column and its reason.

To go on, try another kind of watch, such as watch.below, or write a second collector on the same model. Keen can also query a device over SNMP with snmp.get, or over SSH with ssh.run. Common questions about Keen and collectors are in the FAQ, and the team answers the others through the contact form.