Cache Manager

Opsview Cache Manager is a fast, distributed in-memory key-value store, which can be used to cache session and other disposable data with a short shelf life. As the storage is only in memory, restarting the Cache Manager will potentially lose previously stored cache data and the components making use of this will then need to re-populate that data. The data is also shared between all nodes within the same Collector Cluster, meaning a check run on one node can access data previously stored on another.

The Cache Manager has an API available to client components with both HTTP and HTTPS endpoints. It also uses this API to communicate with other peers within the cluster to implement the distributed cache.

Package dependencies Copied

Service Dependencies Copied

Installation Copied

The component is deployed on Collectors by Opsview Deploy. The specific playbook is cache-manager-install.yaml.

Configuration Copied

Specific configuration options should be set in /opt/opsview/cachemanager/etc/cachemanager.yaml. Opsview Deploy will automatically configure Cache Manager peers within the cluster.

The following settings can be managed, but it is strongly recommended that they managed via Opsview-Deploy to ensure that they are synchronized within each cluster, otherwise the Cache Manager may not function correctly.

Setting Description
worker_timeout The time a worker remains dormant after an error before re-starting (default 30s).
max_cache_size The maximum size of the local cache (default = 1GB, 0B = no limit).
max_item_size The maximum size of an individual cache entry (default 10MB, 0B = no limit).
peer_refresh_time The time delay before peers marked as “down” will be rechecked.
peer_connection_timeout The timeout value when attempting communication with a peer.
timestamp_error_margin The max time difference allowable when validating an encrypted namespace.
cache_purge_time The time delay before expired items are actively purged from the cache.
http_server The settings for the HTTP endpoint (including namespace encryption).
https_server The settings for the HTTPS endpoint (including namespace encryption). The encryption must match that specified in the http_server. The client certificate, ca certificate and server certificate should also be specified here. (Generally automatically setup by Opsview Deploy)
registry The connection to Opsview Registry. (Generally automatically setup by Opsview Deploy)
peer_nodes The other Cache Manager nodes in the cluster. (Generally automatically setup by Opsview Deploy)

Management Copied

Start/Stop cachemanager Copied

# Once configured on the machine you need to run the following command to get it working
sudo /opt/opsview/watchdog/bin/opsview-monit reload
sudo /opt/opsview/watchdog/bin/opsview-monit start opsview-cachemanager
# To stop the cachemanager
sudo /opt/opsview/watchdog/bin/opsview-monit stop opsview-cachemanager

Validate cache manager status Copied

Get status of cache manager Copied

curl http://127.0.0.1:8182/status -s | python -m json.tool
{
    "available_peer_refs": [
        "1d20de92-12bd-4067-8563-8d63e83429cc"
    ],
    "cache_items": 0,
    "peers": [
        {
            "deactivated": 0,
            "ref": "1d20de92-12bd-4067-8563-8d63e83429cc"
        }
    ],
    "ref": "1d20de92-12bd-4067-8563-8d63e83429cc",
    "uptime": 4581
}

Logging Copied

Logs to syslog as per all other components.

Example of logging:

Oct 14 11:13:11 cdb-cachemanager journal: opsview : [NOTICE] Launching Opsview Cache-Manager daemon.
Oct 14 11:13:11 cdb-cachemanager journal: opsview.cachemanager.cachemanagerworker : [NOTICE] Worker started (PID=15807).

Using the Cache Manager Copied

How the Cache Manager works Copied

Opsview Deploy, by default, will install one Cache Manager per Collector. Each Cache Manager is then connected to the Executor and Scheduler via the Load Balancer. When plugins are executed, they make use of their local Cache Manager. The local Cache Manager can then communicate with any remote Cache Managers in the same cluster and share data where necessary.

Using the Cache Manager

API Methods Copied

The Cache Manager is based on two main API methods, get_data and set_data.

Method Description
set_data Input a piece of data into the Cache Manager given a key, namespace and TTL
get_data Retrieve data from the Cache Manager given a key, namespace and maximum wait time.

The Cache Manager assigns data to a unique combination of **key **and namespace.

The namespace ensures naming collisions are avoided and potentially sensitive data cannot be read by other unauthorized plugins.

The key is the identifier for the data in the namespace specified. Data will be valid for the time specified by the TTL (default is 15 minutes).

When getting data, Cache Manager will wait a **maximum **amount of time for “The Lock”.

What is The Lock? Copied

What happens if a Cache Manager goes offline? Copied

It depends on where the data was stored initially and how many Cache Manager’s you have in your clustered environment.

Please note, the Cache Manager is **not **a synchronised data store between collectors. You will only keep the data if both Cache Managers made a call for the same data, prior to the secondary Cache Manager shutting down.

Building plugins that use Cache Manager Copied

Cache Manager is now fully integrated with plugnpy. You can import plugnpy into your Python monitoring script and make full use of all the Cache Manager’s features.

plugnpy comes with an HTTP client which is able to connect to the opsview-cachemanager component. This allows the plugins to use the Cache Manager to store temporary data into memory which can be consumed by other service checks which require the same data.

The module consists of two classes, namely CacheManagerClient and CacheManagerUtils, which provide easy to use interfaces to communicate with the opsview-cachemanager.

CacheManagerClient Copied

A simple client to set or get cached data from the Cache Manager.

The Cache Manager client requires the namespace of the plugin and the host IP address and port number of the Cache Manager to be supplied. These are provided to the plugin as opsview-executor encrypted environment variables, or you can just specify them manually.

Optionally, when creating the client, the concurrency, connection_timeout and network_timeout parameters can be specified to modify the number of concurrent HTTP connections allowed (default: 1), the number of seconds before the HTTP connection times out (default: 30) and the number of seconds before the data read times out (default: 30), respectively.

client = CacheManagerClient(host, port, namespace, concurrency=1, connection_timeout=30,
                            network_timeout=30)

Once a Cache Manager client has been created, the get_data and set_data methods can be used to get and set data respectively.

 client.set_data(key, data, ttl=900)
  
# The get_data method can be called with the key parameter to retrieve data stored under the specified key.  #
  
client.get_data(key, max_wait_time=30)

CacheManagerUtils Copied

To simplify calls to the Cache Manager, plugnpy provides a helper utility method get_via_cachemanager, this will create the Cache Manager client and call the get_data and set_data methods as required.

This method expects five parameters:

 def api_call(string):
  return string[::-1]
 
CacheManagerUtils.get_via_cachemanager(no_cachemanager, 'my_key', api_call, 'hello')

Debugging Copied

To see what is going on inside the Cache Manager as you are making calls, you must start it in debug mode.

# Stop the Cache Manager
sudo /opt/opsview/watchdog/bin/opsview-monit stop opsview-cachemanager
  
# Start the Cache Manager in debug mode using -d flag
sudo -iu opsview /opt/opsview/cachemanager/venv/bin/cachemanagerlauncher -d
  
Oct 14 11:13:11 cdb-cachemanager journal: opsview : [NOTICE] Launching Opsview Cache-Manager daemon.Oct 14 11:13:11 cdb-cachemanager journal: opsview.cachemanager.cachemanagerworker : [NOTICE] Worker started (PID=15807).
["Opsview On-premises"] ["User Guide"]

Was this topic helpful?