Results Exporter

Opsview Results Exporter exports results to services such as Splunk, syslog, local files, and arbitrary HTTP endpoints.

Known issue

When logging data to Rsyslog via TCP, the first message has the correct priority, but all subsequent messages use the user.notice priority.

Installation Copied

Like other Opsview components, the Results Exporter is installed and managed via Opsview Deploy. It can be installed on the Orchestrator system or any infrastructure systems making up part of the Orchestrator (not in a Cluster).

The installation of Results Exporter is controlled by the opsview_module_results_exporter Opsview Deploy variable. By default, it is set to False. To enable it, set opsview_module_results_exporter to True in /opt/opsview/deploy/etc/user_results_exporter.yml.

If you are coming from an older version of Opsview Monitor, you may be missing user_results_exporter.yml. To get a copy of this in the correct location, run the following in a shell on the box with Opsview Deploy:

cp -n /opt/opsview/deploy/etc/examples/user_results_exporter-example.yml /opt/opsview/deploy/etc/user_results_exporter.yml

Configuration Copied

The opsview-results-exporter component is configurable via the user_results_exporter.yml file under /opt/opsview/deploy/etc/.

Credentials

Ensure that your component configuration has the correct Message Queue encoder key, password, database password, and registry password during installation. These values were generated when you deployed Opsview and can be found in /opt/opsview/deploy/etc/user_secrets.yml.

For instructions on how to add your own outputs, please refer to the examples described in Exporting Results.

Migration from manually installed Results Exporter Copied

The automated installation and configuration of the Opsview Results Exporter has been added to Opsview Deploy. Previously, the component and its configuration files had to be installed manually.

By running the check_deploy playbook on an environment that has a manually installed Results Exporter, Opsview Deploy will inform you that it will back up any existing Results Exporter configuration and then overwrite it with the configuration in the user_results_exporter file.

To continue using the existing Results Exporter configuration, the customized fields in /opt/opsview/resultsexporter/etc/resultsexporter.yaml must be carried across to user_results_exporter.yml. To get a copy of this in the correct location, run the following in a shell on the box with Opsview Deploy:

cp -n /opt/opsview/deploy/etc/examples/user_results_exporter-example.yml /opt/opsview/deploy/etc/user_results_exporter.yml

By default, the installation of the Deploy Managed exporter is disabled. To enable it, set the opsview_module_results_exporter to True in the user_results_exporter.yml file.

The following table shows which old fields from the resultsexporter.yaml file map to new fields in the user_results_exporter.yml file. The Message Queue, Database, and Registry configuration do not need to be copied over, as they are managed solely by Opsview Deploy.

For example, the following excerpt from the resultsexporter.yml file:

outputs:
  syslog:
    local_syslog_server:
      fields:
        - hostname
        - stdout
        - perf_data_raw

This would be rewritten as follows and placed in the user_results_exporter.yml file to continue using it:

opsview_results_exporter_outputs:
  syslog:
    local_syslog_server:
      fields:
        - hostname
        - stdout
        - perf_data_raw

Advanced Configuration Copied

Worker settings Copied

The number of parallel component workers and the timeout before component worker restart (if encountering errors) can be set as follows:

opsview_results_exporter_worker_timeout: <secs>       # default 30
opsview_results_exporter_initial_worker_count: <num>  # default to 0, which will spawn as many workers as the machines CPU has logical processors

Please note the following considerations:

• On a multi-core system, the more workers in use, the lower the load and elapsed time will be, with an upper bound on the number of cores available in your system.
• The more workers in use, the greater the overall component memory usage will be used.
• The memory usage of workers is multiplicative with the number of outputs configured.

Data cache settings Copied

The Results Exporter can cache Opsview data if the dal_fetchall or dal_fetchone field operations are in use. For more information, see Exporting Results.

You can override the opsview_results_exporter_data_access_cache section in the user_results_exporter file to have full control over the cache settings:

opsview_results_exporter_data_access_cache:
  - <data model type>:        # e.g. RuntimeHosts
      cache_items: <yes|no>   # whether to cache data for the model
      max_items: <num>        # if caching, the maximum number of items to cache (if field unset, defaults to no limit)
      max_item_size: <num>    # if caching, the maximum size of any cached item (if field unset, defaults to no limit)
      pre_load: <yes|no>      # whether to pre-load data on component start up (default no)
      refresh_on: reload      # set if the cache should be cleared for the model when Apply Changes completes

Please note the following considerations:

• The maximum number of items refers to the number of individual queries (for example, individual field operation executions after message fields are populated) that can be cached at once. For example, if a query was configured to run using a message field and it was ran against 2 messages with different values for that field, it would count as 2 separate cache items.
• The maximum size of each item refers to the number of data model results the cache will accept. For example, if a query matches 2 RuntimeHosts in the system, it would be a cache item of size 2.
• Setting a maximum number or size on items will reduce the memory usage of the component, but it will increase the load and elapsed time to process individual messages.
• Enabling preloading will increase load and elapsed time on component startup (and when Apply Changes occurs, if refreshing on reload is set), but it will decrease load and elapsed time during processing.
• Using message filters to decrease the number of messages processed will decrease the overall load and elapsed time to process all messages.

By default, the component uses the following cache configuration:

opsview_results_exporter_data_access_cache:
  - RuntimeHashtags:
      cache_items: no
  - RuntimeHosts:
      cache_items: yes
      max_items: 8_000
      max_item_size: 1
      pre_load: yes
      refresh_on: reload
  - RuntimeServicechecks:
      cache_items: yes
      max_items: 32_000
      max_item_size: 1
      pre_load: yes
      refresh_on: reload

The component will generate a warning if it encounters an unexpected model type.

Data from unsupported model ... could not be queried from cache

If intentionally not caching a model type, then add a cache_items: no entry for the model to the cache settings to suppress the logging. This will require overriding the entire section and then adding your new entries.

Logging Settings Copied

The Results Exporter logging settings can be set by overriding the following section:

opsview_results_exporter_config_overrides:
  logging:
    sensitive_trace: <yes|no>   # enable low-level trace logging (default no)
    data_access_cache:
      collisions: <yes|no>      # whether to log data cache collisions (default no)
    loggers:
      opsview:
        level: NOTICE           # overall component log level

The sensitive_trace will only have an effect if the component is also set at the DEBUG log level.

Service administration Copied

As root, start, stop and restart the service using:

/opt/opsview/watchdog/bin/opsview-monit <start|stop|restart> opsview-resultsexporter