Slim Poller 2.0

UUID

It is now possible to enable node identification using UUID instead of using IP. This can be useful if your TCP packets have a non-unique outgoing IP address, such as when behind a NAT or if your nodes' incoming IP addresses and outgoing IP addresses differ. This is also useful for setting up multiple Slim Pollers in Kubernetes.

There are two new Merlin settings used to configure the UUID:

• ipc_uuid — set the UUID of a specific node. This is a top-level configuration in /opt/monitor/op5/merlin.conf.

• uuid — identify a connecting node with a given UUID. This must be set in the node configuration.

A UUID must have a length of 36 characters. To generate a well-formed UUID, you can use mon id generate.

The following are examples for a master and a poller UUID configuration:

....poller op5-slim-poller-ssh-65799d958f-h9wjt {        uuid = de5c4eb9-dc9e-4b53-831c-246d254ad39e        hostgroup = k8s_group        address = IP_ADDRESS        port = 15551}

log_level = info;use_syslog = 1;ipc_uuid = de5c4eb9-dc9e-4b53-831c-246d254ad39e...master master {  address = MASTER_IP  port = 15551  connect = no}

In the above case, the master identifies the poller with its UUID. However, the poller is using regular IP identification to identify connections from the master. This is common in a passive poller mode, where the master does not do active connections to the poller.

If you want to use UUID to identify both components, you would need to add ipc_uuid to the master merlin.conf file, and the corresponding uuid setting in the master node configuration on the poller.

File sync for mon oconf fetch

It is possible to sync files between OP5 Monitor nodes. For guidance, see How to Synchronize files and folders with Merlin in load balanced setup.

By default, files are pushed from one node to another. However, it is also possible to fetch files from remote. For example, a poller could be set up to fetch custom plugins from a master server.

To set up file fetching, first you must set up the node to fetch from a master. Then, you need to add a sync section.

Files from the sync section are only synced when using the --sync argument with the mon oconf fetch command. An example configuration can be seen below:

master master {  address = IP_ADDRESS  port = 15551  sync {    /opt/plugins/custom/  }  object_config {    fetch_name = poller    fetch = mon oconf fetch --sync master  }}

Files are only synced when a Naemon configuration change is done. If you need to trigger it in other situations, see mon oconf remote-fetch.

mon oconf remote-fetch

The command mon oconf remote-fetch will tell a remote node to do a fetch against the current node. This will only work if the remote node is correctly configured to fetch from the node that calls mon oconf remote-fetch.

This command can be useful if you want to manually trigger the poller to fetch a new file; for example, if you have added a new custom plugin.

It is possible to trigger a fetch on a specific node:

mon oconf remote-fetch poller-name

Or a type of node:

mon oconf remote-fetch type=poller

Command usage:

remote-fetch     [--type=<peer|poller> [<node>]       Tells a specific node to fetch split configuration from this node.       NOTE: A configuration variable called "fetch_name" is       required in the object_config section of merlin.cfg on the       remote node. The variable should be set to the name of the node,       as seen by the master.

Cluster update

cluster_update is a Merlin module setting which takes a command. The command is run when a node gets a signal from a master that its cluster configuration is invalid. Use this setting to create a script to automatically update the cluster configuration.

The following is used for the autoscaling functionality in the Slim Poller. You can create a custom script to use instead.

module {    cluster_update = /usr/local/bin/cluster_tools.py --update    log_file = stdout;    notifies = no}

Setup for autoscaling

In order achieve autoscaling, the following is set up:

• A container entry script that connects to a designated master, and registers on the cluster to all relevant masters and peers.

• The cluster_update module is set up to detect any changes to the cluster. A connection is established with the designated master, and the cluster configuration is updated.

• Slim Pollers are identified by UUID on the master, but not towards peers.

• The address of each Slim Poller must be an address that is reachable from within the Kubernetes cluster; for example, the pod IP. It is not necessary for this IP to be reachable from masters outside the Kubernetes cluster.

A number of environment variables are used to configure this, such as the master IP, poller hostgroup, and so on. For more information, see Setting environment variables.

Setting environment variables

For autoscaling to work correctly, you need to set up the following environment variables:

The following example shows a YAML file with the environment variables configured for autoscaling in Kubernetes:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
         env:         - name: MASTER_ADDRESS           value: "IP_ADDRESS_HERE"         - name: MASTER_NAME           value: "master"         - name: MASTER_PORT           value: "15551"         - name: POLLER_ADDRESS           valueFrom:             fieldRef:               fieldPath: status.podIP         - name: POLLER_NAME           valueFrom:             fieldRef:               fieldPath: metadata.name         - name: POLLER_HOSTGROUPS           value: "GROUP1, GROUP2"         - name: FILES_TO_SYNC           value: "/opt/plugins/custom/, /opt/plugins/check_nagios"

Installing SSH keys

In order for autoregistering to work correctly, the Slim Poller container must have SSH keys installed that are authorized at the master server. One way to achieve this is to create a new Docker image from the Slim Poller image.

Caution: The SSH key added in this image must also be added to /opt/monitor/.ssh/authorized_keys on the designated master server.

The following are example commands for installing the SSH:

FROM op5com/slim-poller_naemon-core:slim-poller-2.0-prereleaseCOPY --chown=monitor:root id_rsa /opt/monitor/.ssh/id_rsaCOPY --chown=monitor:root id_rsa.pub /opt/monitor/.ssh/authorized_keysRUN chmod 600 /opt/monitor/.ssh/id_rsaRUN chmod 644 /opt/monitor/.ssh/authorized_keys

Scaling to a higher number of replicas

After starting a Slim Poller deployment, you can scale up the replicas manually by using the kubectl command:

kubectl scale deployment.v1.apps/op5-slim-poller --replicas=2

You can also use Kubernetes autoscaling. For more information, see Horizontal Pod Autoscaler.

Preparation of master

For the Slim Poller 2.0 to work correctly, an update to the master is required. All peered masters must be set up accordingly:

1. Install OP5 Monitor version 8.2.5, as normal.

2. After installation, open the file, /etc/yum.repos.d/op5-release.repo and update it to have the following content:

   Copy

   op5-release.repo

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   [op5-monitor-slim-poller-2.0]name=op5 Monitor Slim Poller 2.0baseurl=http://repos.op5.com/el7/x86_64/monitor/slim-poller-2.0/updates/enabled=1gpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-op5##### op5 Monitor[op5-monitor-updates]name=op5 Monitor Updatesbaseurl=http://repos.op5.com/el$releasever/$basearch/monitor/2020.k/updatesenabled=1gpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-op5##### op5 Epel[op5-epel]name=op5 EPELbaseurl=http://repos.op5.com/epel/7/$basearchenabled=1gpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7

    

   Save the updates.

3. Execute yum clean all && yum update.

Before you can set up the Slim Poller, ensure that the pollers are authorized to connect to the master. To do so, set up the SSH keys on the masters. For guidance, see Installing SSH keys.

Docker image

The Docker image can be found on Docker Hub, and is named slim-poller-2.0-prerelease. To deploy the Docker image into Kubernetes, see the Example Kubernetes deployment file.

Before starting the Slim Poller in Kubernetes, ensure that the hostgroups, specified in the POLLER_HOSTGROUPS environment variable, exist on the master.