Toolkit

Overview Copied

This guide discusses the installation, set-up, and configuration of the Toolkit sampler. It is intended to provide different sample scripts of user scenarios and basic information to use this plug-in.

The Toolkit plug-in integrates output from custom commands and scripts into Geneos. This plug-in executes these commands or scripts on a periodic basis and converts their output into a dataview.

These custom commands can execute standard output application written in C/C++, Perl, TCL, Python, and scripts, such as VB, batch, and shell script.

Prerequisites Copied

The following requirements must be met prior to the configuration and setup of the Toolkit plug-in:

• Netprobe must have the appropriate access to execute the commands and scripts.
• Results of the commands and scripts must be to standard output (stdout).
• Results must be in the custom comma separated values (CSV) format.

User requirements Copied

To use this plug-in, you must meet the following requirements:

• Basic understanding of programming languages, such as C/C++, Perl, TCL, and shell script that can execute the custom commands.
• Use of batch scripting or PowerShell scripting in Windows, and other third-party scripting languages.

Introduction Copied

A typical dataview constructed using the Toolkit plug-in contains several headline variables followed by a detailed table.

The Toolkit plug-in supports several output formatting conventions that control how script output is interpreted, including headline rows, column headers, and multi-dataview output sections. For more information, see Toolkit output markup tags.

By default, the Toolkit plug-in uses commas (,) as cell delimiters. Other commonly used delimiters include semicolons (;), quotation marks (" or '), and slashes (/).

The following example shows sample Toolkit script output:

$ ./toolkit_test03.bash
id,name,status,Value
<!>AlertDetails,this is red alert
001,agila,up,97
002,lawin,down,85

In this example, ./toolkit_test03.bash is configured in the Sampler script field:

Given the sample script used in the example, the generic format to display in the dataview is:

column1,column2,column3,column4
<!>headlineName,headlineValue
row1,row1col2,row1col3,row1col4
row2,row2col2,row2col3,row2col4

The dataview of this sampler configuration displays in Active Console:

Warning

If the script fails by either returning no output or failing to run within the timeout period for five successive samples, then the Netprobe disables sampling for that instance of the toolkit. To re-enable the sampling, you can force a manual sample (using the Sample Now button), restart the plug-in through reconfiguration, or restart the Netprobe.

Toolkit output markup tags Copied

Toolkit scripts can use markup tags to control how output is published to dataviews. Use <!> for headline values, <h> for column headers, and <dataview> for multiple dataviews.

<!> headline values Copied

Use the <!> tag to publish headline values in a Toolkit dataview.

A line that begins with <!> is interpreted as a headline row instead of table data. Headline values appear above the dataview table.

When Toolkit processes the headline row, it removes the leading <!> tag and displays only the headline name and value in the dataview.

The sample dataview shown earlier includes two types of headline cells:

<h> column headers Copied

You can optionally use the <h> tag to mark a line explicitly as the column header row.

A line that begins with <h> is the column header row for the current dataview section. The <h> tag is case-insensitive. When Toolkit processes the line, it removes the leading <h> tag and displays only the column names in the dataview.

For example:

<h>Process,CPU,Memory
nginx,12,256
java,34,1024

This example produces a dataview with column headers Process, CPU, and Memory.

If a section has no <h> line, Toolkit uses the first line in that section as the column header row. If the first line starts with <!>, Active Console shows the <!> tag in the first column header instead of publishing a headline value:

If you use more than one <h> line in a dataview section, Active Console returns an error:

<dataview> dataview sections Copied

Use the <dataview> tag to publish multiple dataviews from a single Toolkit sampler script.

A line that begins with <dataview> starts a new dataview section. All output after that line belongs to that dataview until the script encounters another <dataview> line or the script ends.

The text after <dataview> on the same line becomes the dataview name. Toolkit ignores leading and trailing spaces. If you omit the dataview name, Toolkit uses the default dataview name.

Each dataview section uses an <h> line or the first line in that section as the column header row. Output that appears before the first <dataview> line is published to the default dataview.

For example:

<h>Process,CPU,Memory
nginx,12,256
java,34,1024

<dataview> Disk usage
Filesystem,Used,Available
/dev/sda1,40G,60G
/dev/sdb1,120G,80G

This example publishes:

• The default dataview, which contains process metrics and uses an <h> line for column headers.
• A second dataview named Disk usage, which uses the first line after <dataview> Disk usage as column headers.

Note

• If a dataview section has no rows, Toolkit does not publish the dataview. Empty dataview sections do not consume a dataview license allocation.
• If rowLimit is configured, the limit applies independently to each dataview section.
• Do not use the same dataview name more than once in a single script execution. If Toolkit detects duplicate dataview names, it publishes the affected dataview as an error dataview.
• If a later script execution no longer includes a previously published dataview, the Gateway removes that dataview. Rules, dashboards, and history views that reference the removed dataview might stop working as expected.

Commas within view cells Copied

Since the comma is the cell delimiter, it is not possible for the toolkit’s scripts to have these within the cell’s content.

To include an actual comma within the cell’s content, precede the comma with a backslash \.

Sample command script:

queue3, 7\,331, 45\,000,0.16, online

Expected output:

queue3 | 7,331 | 45,000 | 0.16 | online

Environment variables Copied

In addition to Geneos variables, user variables can be set in the Environment Variables and Secure Environment sections for the plug-in to utilise.

1. Go to Gateway Setup Editor > Sampler > Toolkit plug-in.
2. Click the Advanced tab.

Secure Environment Variables are encrypted using the AES 256 encryption. This is useful for storing passwords or any sensitive data.

To learn how to use these variables, see the Appendix section.

Display CSV file in the dataview Copied

Toolkit plug-in allows you to display the data in the Active Console dataview by using a command to display the file to a standard output.

To display the content of a .csv file:

1. In the Gateway Setup Editor, open the Toolkit sampler.

2. Enter the command script in the Sampler script following the directory path of the file. For example:

   /bin/cat /home/documents/toolkit.csv
   

3. Click Save current document to apply changes.

   On Unix, /bin/cat is the common program, while on Windows, use cmd.exe command interpreter to execute the type built-in. For example:

   C:\Windows\system32\cmd.exe /c type C:\Users\Administrator\Documents\toolkit.csv
   

Install and set up Copied

Ensure that you have read and can follow the system requirements prior to installation and set-up of this plug-in.

If you are using this plugin with Gateway Hub, you must create a user defined data schema. For instructions, see Create a data schema. This provides instructions on how to set up a basic configuration using a Toolkit plug-in:

1. To select the Toolkit plug-in, go to Gateway > Probes > Managed entity > Samplers.
2. In the Samplers section, select Toolkit on the Plugin menu.
3. In the Sampler script field, enter the command to execute the script.

Note

The Sampler script specifies the name of the command that the plug-in runs on a periodic basis. If there are any spaces in your path, you must enclose it in double quotation marks. Otherwise, the sampler fails to execute the script.

In this example, the command and script is entered in the Sampler script field:

./toolkit_test.bash

4. Click Save current document to save your changes.
5. Go to the Toolkit dataview in the Active Console to check if your configuration to monitor the path is working.

These Toolkit sampler configurations below show the different user scenarios that you can use to execute commands.

Use command argument Copied

To execute command using Command argument, follow these instructions:

1. In the Sampler script field, enter the command to execute the script. In this example, the command and script entered in the Sampler script field is:

   /bin/cat toolkit_command_args.txt
   

2. Click Save current document to save your changes.

3. Go to the Toolkit dataview in Active Console to check if your configuration to monitor the path is working.

Use PowerShell Copied

To execute command using Powershell script, follow these instructions:

1. In the Sampler script field, enter the command to execute the script. In this example, the command and script entered in the Sampler script field is:

   C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -noprofile -executionpolicy bypass -file "C:\Users\Administrator\Downloads\Test Scripts\output-select-lines.ps1"
   

2. Click Save current document to save your changes.

3. Go to the Toolkit dataview in Active Console to check if your configuration to monitor the path is working.

Use Windows BAT files Copied

To execute command using Windows bat script, follow these instruction:

1. In the Sampler script field, enter the command to execute the script. In this example, the command and script entered in the Sampler script field is:

   "C:\Users\Administrator\Downloads\Test Scripts\toolkit_pebl_col273.bat"
   

2. Click Save current document to save your changes.

3. Go to the Toolkit dataview in Active Console to check if your configuration to monitor the path is working.

Plugin configuration Copied

Note

If you are using this plugin with Gateway Hub, you must create a user defined data schema. For instructions, see Create a data schema.

The following parameters can be configured for this plug-in:

samplerScript Copied

Name of the command that the plug-in runs on a periodic basis. If the command contains spaces, then the entire command (but not any arguments) must be wrapped in double quotes, or the sampler will fail to execute the script.

For example, rather than specifying:

c:\monitoring\example path\example path\tkscript.bat a b c d

The command should be specified correctly as:

"c:\monitoring\example path\example path\tkscript.bat" a b c d

Absolute and relative paths to the command must be specified as well.

Note

The PATH environment variable is not used to locate the script or executable.

Mandatory: Yes

numberOfDataviews Copied

Controls how many dataview license allocations the Toolkit requests. Each allocation corresponds to one distinct Toolkit dataview that the sampler can publish.

Note

Changing the numberOfDataviews value changes how many license allocations the Gateway associates with the sampler. Set this value to match the number of Toolkit dataviews that you expect the script to produce.

Default: 1. If the value is missing, invalid, or not a positive integer, the Toolkit uses 1.

rowLimit Copied

Limits the number of rows displayed. The default rowLimit is 200, but can be set higher or lower in the configuration.

The rowLimit prevents the Netprobe from becoming overloaded if the script outputs a large amount of data. If the rowLimit is reached, the Netprobe will still read the data from the script but will not store it. This ensures any script relying on read requests will complete.

Note

• This is a limit on the number of lines read in from the script. The limit would include lines read in that became Headline Variables and Column Headers, as well as lines that became rows in the dataview.
• You can override this setting with a Gateway-wide row limit that you can configure in the Gateway Setup Editor under Operating environment > Maximum number of dataview rows.

scriptTimeout Copied

Changes the timeout value for a script. A script is allowed 60 seconds to execute. A script running longer than the timeout value will not display the desired results in the dataview. Use this setting to change the script timeout value for longer running scripts.

Default: 60 (in seconds)

terminateOnTimeOut Copied

Changes the behavior of the plug-in when a timeout occurs due to the script running longer than 60 seconds. The normal behavior is for the plug-in to kill the long running process. Setting this flag to false will change the behavior, and the plug-in will leave the process running. This behavior change can also be controlled through the use of an environment variable. Setting the environment TOOLKIT_TERMINATE_ON_TIME_OUT to the value of FALSE will have the same effect as setting the terminateOnTimeOut setting to false. The environment setting is overridden by the GSE setting.

Mandatory: No

Default: true (unless environment variable TOOLKIT_TERMINATE_ON_TIME_OUT is set to FALSE, then false)

environmentVariables Copied

Defines an optional set of environment variables that will be available to the script.

Mandatory: No

environmentVariables > variable Copied

Defines a single environment variable that will be available to the script.

Mandatory: No

environmentVariables > variable > name Copied

Name of the variable being defined.

Mandatory: Yes

environmentVariables > variable > value Copied

Value of the variable being defined.

Mandatory: Yes

secureEnvironmentVariables Copied

Defines an optional set of secure environment variables that are AES 256 encrypted and will be available to the script. Environment variables in this section will override any other environment variables of the same name defined elsewhere.

Mandatory: No

secureEnvironmentVariables > variable Copied

Defines a single secure environment variable that will be available to the script.

Mandatory: No

secureEnvironmentVariables > variable > name Copied

Name of the variable being defined.

Mandatory: Yes

secureEnvironmentVariables > variable > value Copied

Value of the variable being defined. This will be AES 256 encrypted or reference a variable that is AES 256 encrypted. References to other variable types are invalid and will result in an error and the variable being ignored.

Mandatory: Yes

script Copied

Defines a script that will be sent to the Netprobe and written out as a file for execution.

Mandatory: No

script > filename Copied

Name of the script that the Netprobe will write to. This must be in or under the Netprobe’s working directory. Absolute paths are not permitted in that paths must start with (./) or state the filename itself, denoting the Netprobe’s relative path. The subdirectories and filename in the path will then be created if they do not exist.

For example, it can be any of the following:

• ./<netprobe\_directory>/<filename>
• <netprobe\_directory>/<filename>
• ./filename
• filename

Mandatory: Yes

script > contents Copied

Defines the contents of the script.

Mandatory: Yes

showExecutionDuration Copied

ExecDuration Headline shows the toolkit script execution time during last sample.

Mandatory: No

Default: true

Appendices Copied

Use the secured environment variables Copied

This procedure has two steps:

1. Generate the key file and encrypted value.
2. Decrypt the encrypted value before the Toolkit executes the script.

Generate the encrypted password Copied

Note

This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org).

1. Generate and use a key file as follows:

   openssl enc -aes-256-cbc -k "Now is thewinter of our discontent" -P -md sha1 >key-file
   

   The key file is required for the encryption and decryption of AES 256 passwords.

2. Generate the encrypted value.

   ./active/gateway2.linux -aes256-encrypt 12345 -key-file key-file -skip-cache
   

   The 12345 isa user-specified password which is then encrypted. The Gateway produces the output in hexadecimal:

   Encoded text: +encs+69B1E12815FA83702F0016B0E7FBD33B
   

   A Gateway binary, such as gateway2.linux_64 is needed to create the AES 256 password. The -skip-cache is needed only the first time you wish to change the key file.

3. In the Toolkit sampler, create a secured environment variable.

4. Add the encrypted password to the secured environment variable. You can convert the sampler view to XML.

Decrypt the encrypted value Copied

1. Use the sample shell script below that decodes the hexadecimal or encrypted value:

   ./decode.sh +encs+69B1E12815FA83702F0016B0E7FBD33B
   

   decode.sh:

   #!/bin/bash
   hex=${1#*+encs+}
   salt=$(grep salt key-file)
   key=$(grep key key-file)
   iv=$(grep iv key-file)
   echo ${hex}| xxd -r -p | base64 | openssl enc -d -aes-256-cbc -S ${salt#*=} -K ${key#*=} -iv ${iv#*=} -a
   

2. To convert an environment variable, replace the ‘1’ in the second line with the name of the environment variable. All encrypted variables start with +encs+, so this removes the prefix prior to decoding.

   Note

   xxd is part of the vim-common package which you may have to install separately.

Sample scripts in Windows Copied

wgetenv3.exe

Running wgetenv3.exe reg.txt, for example, where the file reg.txt contains:

[H] logPixelsHEX hkey_current_config\software\fonts\logpixels    [HEX]
[H] logPixelsDEC hkey_current_config\software\fonts\logpixels [DEC]
[L] sysID HKEY_LOCAL_MACHINE\HARDWARE\DESCRIPTION\System\identifier [STR]
[L] sysBIOSDate HKEY_LOCAL_MACHINE\HARDWARE\DESCRIPTION\System\SystemBiosDate [STR]
[L] videoBIOSDate HKEY_LOCAL_MACHINE\HARDWARE\DESCRIPTION\System\VideoBiosDate [STR]
[L] videoBIOSVersion HKEY_LOCAL_MACHINE\HARDWARE\DESCRIPTION\System\VideoBiosVersion [STR]

generates the following input to a toolkit sampler:

name,value
<!>logPixelsHEX,60
<!>logPixelsDEC,96
sysID,AT/AT COMPATIBLE
sysBIOSDate,06/23/08
videoBIOSDate,03/26/09
videoBIOSVersion,HWEAASUS EN9400GT VBIOS Ver 62.94.71.00.AS01 HWEAASUS
EN9400GT VBIOS Ver 62.94.71.00.AS01 HWEAASUS EN9400GT VBIOS Ver
62.94.71.00.AS01 Version 62.94.71.00.13  Version 62.94.71.00.13  Version
62.94.71.00.13  Version 62.94.71.00.13

W3C XML 1.0 (Fourth Edition) Specification - http://www.w3.org/TR/2006/REC-xml-20060816