Geneos ["Geneos"]
["Geneos > Netprobe"]["User Guide"]

Collection Agent setup

Overview

This guide outlines the step-by-step procedures you need to start running the Collection Agent in a standard environment.

Caution: Make sure to read Upgrade to Collection Agent 3.x, which outlines the breaking changes that may have an impact on your upgrade, before you upgrade to the latest version of Geneos and Collection Agent.

For information on running the Collection Agent in an orchestrated environment, see Overview in Prerequisites.

Note: Beginning Geneos 5.1, the Collection Agent is included in the Netprobe binaries for Windows and generic Linux platforms.

Prerequisites

System requirements

The following requirements must be met on the machine where the Netprobe and Collection Agent are installed:

  • Linux operating system (CentOS, RHEL, or SUSE), or
  • Windows operating system

Java requirements

  • Java 11 or higher.

  • A minimum Java heap size of 512MB is recommended for Collection Agent.

    If the agent is performing garbage collection too frequently as the volume increases, a larger heap size should be allocated.

  • Once you have Java installed, configure the environment variables required by the Netprobe so that it can locate the required resource files. For guidance, see Configure the Java environment.

Caution: Java installation and environment configuration is a common source of errors for users setting up Java-based components and plug-ins. You should read Configure the Java environment to help you understand your Java installation.

Geneos components requirements

  • Geneos Netprobe version 5.1.0 or higher. Please read Upgrade to Collection Agent 3.x for information on how to properly upgrade to the latest version of Collection Agent.

  • Ensure that your Gateway and Netprobe are running.

Caution: If you are using a Gateway version 5.2, then any Netprobe running in normal or floating mode must also be version 5.2. Otherwise, the Collection Agent will not start.

Setup and configuration

To set up and configure your Collection Agent, do the following.

  1. Configure your Collection Agent depending on how you want to run it. You can run the Collection Agent in two ways:

    Regardless of how you run your Collection Agent, it will always run as a Java process.

  2. Create Dynamic Entities

  3. Add Dynamic Entities in your Netprobe

Run Collection Agent via Netprobe

You can run the Collection Agent with Netprobes running in the normal, floating, or self-announcing modes on the same host.

To run the Netprobe and the Collection Agent on different hosts, you must Run Collection Agent as a stand-alone Java process.

To run the Collection Agent via Netprobe, follow these steps:

  1. In Gateway Setup Editor, click Dynamic entitiesCollection agent parameters, and then click New Collection agent parameters.

  2. Select Managed in the Collection Agent dropdown list.

  3. Choose your preferred YAML type in the Yaml dropdown list and configure your Collection Agent parameters.

     

    YAML type Configure your Collection Agent in
    Bundled YAML

    the collection_agent/collection-agent.yml packaged with the Netprobe binary.

    Custom YAML the specified YAML file that can have any name. The file can be in any folder but if you are using a relative path, then it should be relative to the working directory.
    Managed YAML the Gateway Setup Editor. For more information, see Managed Collection Agent in Dynamic Entities.
       

    Note: If you are starting the Collection Agent using the Netprobe, all the ports configured in the Gateway Setup Editor will automatically apply to the running Collection Agent.

  4. Configure your plugins.

    YAML type Configure your plugins by
    Bundled YAML

    adding the parameters of the desired plugins in the collection_agent/collection-agent.yml packaged with the Netprobe binary.

    Custom YAML

    adding the parameters of the desired plugins in the specified YAML file in Step 3.

    Managed YAML adding the parameters in the Dynamic EntitiesCollectors section in the Gateway Setup Editor. For more configuration information, see Collectors in Dynamic Entities.
       

    Note: If you need to set up a new plugin that is not yet included in the package, you can download it from the Resources page and add it to the collection_agent directory.

Self-Announcing Netprobe

For Self-Announcing Netprobes, you can start the Collection Agent by adding the following configuration in the selfAnnounce settings in the Netprobe setup file:

<dynamicEntities>
	<collectionAgentParameters>CollectionAgentParams</collectionAgentParameters>
	<mappingType>DynamicEntitiesMapping</mappingType>
</dynamicEntities>
					

For more information, see Self-announce settings in Netprobe setup.

Note: You still have to Create Dynamic Entities and make sure of the following:

  • The value inside the collectionAgentParameters tag in the XML above is the name of your Collection agent parameters in Gateway Setup Editor.

  • The value inside the mappingType tag in the XML above is the name of your Mapping type in Gateway Setup Editor.

Run Collection Agent as a stand-alone Java process

Running the Collection Agent as a stand-alone Java process allows you to run the Netprobe and the Collection Agent on different hosts.

If you wish to run the Netprobe and the Collection Agent on the same host, you must Run Collection Agent via Netprobe.

To run Collection Agent as a stand-alone Java process, follow these steps:

  1. Download the Collection Agent jar file, collection-agent-<version>-exec.jar, from the Resources page.
  2. Create a configuration file in YAML format, for example, collection-agent.yml.

    Ensure the following:

    • The TCP reporter fields are configured:

      • hostname: - host where the Netprobe is running.
      • port: - a unique port number.
    • The environment variables in the collection-agent.yml are defined on your host.

    plugin-directory: ${env:CA_PLUGIN_DIR}
    monitoring:
      health-check:
        listen-port: ${env:HEALTH_CHECK_PORT}
      metrics:
        dimensions:
          app_name: collection-agent
    reporters:
      - type: tcp
        name: tcp
        hostname: localhost
        port: ${env:TCP_REPORTER_PORT}
    collectors:
      - name: statsd
        type: plugin
        class-name: StatsdServer
    workflow:
      store-directory: .
      metrics:
        reporter: tcp
      events:
        reporter: tcp

    For more information on configuring the Collection Agent, see Collection Agent configuration reference.

  3. In Gateway Setup Editor, click Dynamic entitiesCollection agent parameters, and then click New Collection agent parameters.

  4. Select Unmanaged in the Collection Agent dropdown list.

  5. Input the port value from your YAML file in the Reporter port field.

  6. If you need to set up a new plugin, download it from the Resources page and put it in the same directory as collection-agent-<version>-exec.jar.

  7. Run the Collection Agent using the following command:

    java -jar collection-agent-<version>-exec.jar collection-agent.yml
  8. To stop the Collection Agent, stop the Java process you started in Step 7.

    Note: The only way to start or to stop the Collection Agent as a stand-alone Java process is to run the Java command in Step 8 and to stop the Java process in the terminal, respectively.

Create Dynamic Entities

Netprobes receive datapoints from Collection Agents. Each datapoint can have a different set of dimensions, attributes, and properties. You should create mappings to construct a Geneos hierarchy from datapoints. This Geneos hierarchy determines how data is displayed in your Active Console. You should also create a mapping type, which allows you to easily deploy the same configuration on multiple probes.

  1. Open Gateway Setup Editor.

  2. Click Dynamic entitiesMapping, and then click New Mapping.

  3. Enter a mapping name in the Name field.

  4. In Options, choose custom or builtin. For more information, see Options in Dynamic Entities.

    Note: A built-in Collection Agent V3 mapping is packaged with the Netprobe, which automatically sets up the default mappings needed to display the Collection Agent self-monitoring metrics.

  5. Click Dynamic entitiesMapping types, and then click New Mapping type.

  6. If you configured a managed Collection Agent with a managed YAML type in the previous steps, then in the Collectors field, click Add new to add the name of the collectors you created in Run Collection Agent via Netprobe.

  7. In the Mappings field, click Add new to add the name of the mapping created in Step 4.

  8. To monitor the logs collected by the Collection Agent plugin, you can add an FKM plugin sampler in the Samplers field. The FKM plugin consumes these logs as streams. To configure your FKM plugin to monitor these streams, see files > file > source > stream in File Keyword Monitor configuration.

Add Dynamic Entities in your Netprobe

  1. Open Gateway Setup Editor.

  2. In Probes, select your Netprobe and go to the Dynamic Entities tab.

  3. Choose a mapping type in the Mapping type dropdown list.

  4. Choose a collection agent parameter in the Collection agent parameters dropdown list.

    Note: If mappings are defined in both the Mapping type and in the Collector, then the Netprobe will apply the mappings defined in the Mapping type first then the mappings defined in the Collectors.

Collection Agent self-monitoring Metrics

If you started the Collection Agent with the built-in Collection Agent V3 mapping, then the Collection Agent self-monitoring metrics are displayed in your Active Console.

Dynamic Entities Health

The Dynamic Entities Health plugin monitors the performance of Collection Agent and Netprobe communication. It tracks the state of all metrics received by a Netprobe from the Collection Agent, and all Dynamic Entities associated with a Netprobe.

For guidance on how to set up the Dynamic Entities Health sampler, see Dynamic Entities Health.

Collection Agent log file

The Collection Agent uses a separate log file from the Netprobe. The log settings for Collection Agent use Logback and are stored in logback.xml.

By default, the Collection Agent log file is stored in the collection-agent.log of the working directory. If you are running a managed Collection Agent and you configured the Log directory parameter in GSE, then the collection-agent.log will be in that specified location.

Connection and lifecycle

When run with the Netprobe, Collection Agent runs as a Java process alongside the Netprobe process.

Common failure modes

The Collection Agent will not start in the following scenarios:

  • JAVA_HOME is not set in the Netprobe's environment. For more information, see Configure the Java environment.
  • Collection Agent resource files are missing, or are not the same version packaged in the collection_agent directory:
    • collection-agent-3.x.x-exec.jar
    • collection-agent.yml
  • The following Collection Agent ports are not available:
    • Health port
    • Reporter port
  • StatsD UDP port is not available

    Note: The StatsD plugin uses the default which is specified in the collection-agent.yml. If two Netprobes are running with one Collection Agent, then the Address bind already in use. error message is displayed in the Collection Agent log file.

Netprobe connection

The following describes the behaviour of the Netprobe connection depending on the state of the Collection Agent:

  • Netprobe restarts the managed Collection Agent if the latter goes down, or cannot be detected after having been detected before.
  • Netprobe will attempt to restart an unreachable Collection Agent three times. If on the third time, the Netprobe still cannot detect the Collection Agent, then it no longer attempts to restart the Collection Agent.
  • If the Netprobe restarts, it resumes its connection with the managed Collection Agent.

Note: A Collection Agent started independently of the Netprobe is not managed by the Netprobe.

Health checks

The Netprobe periodically pings the managed Collection Agent through the defined health check port.

The health checks are governed by the following variables:

Name Description
CA_HEALTHCHECK_INTERVAL

Time in seconds between pings from the Netprobe to a managed Collection Agent.

Default value: 20

CA_MAX_HEALTHCHECK_ATTEMPTS

Maximum number of attempts that the Netprobe makes to contact a managed Collection Agent.

Default value: 3

CA_HEALTHCHECK_TIMEOUT

Time in seconds that the Netprobe waits for a response from the Collection Agent during a health check.

Default value: 5

CA_MAX_RESTARTS

Maximum number of attempts that the Netprobe makes to restart an unresponsive managed Collection Agent.

If the Netprobe fails to communicate with the managed Collection Agent after the maximum number of attempts, then the Netprobe stops monitoring the managed Collection Agent.

Default value: 3