Kubernetes Collection Agent plug-in

Overview

The Kubernetes Collection Agent plug-in collects logs, metrics and events from OpenShift and Kubernetes.

Configuration reference

Below is an example YAML file which may require some changes for your project’s configuration:

collectors:
  - type: plugin
    name: kube-metrics
    class-name: KubernetesMetricsCollector
    
    # Events module configuration
    events:
    
      # Whether events collection is enabled.  Defaults to true.
      enabled: true
      
      # List of namespaces to watch events from. In case of empty list or not present, defaults to 
      # all namespaces.
      namespaces:
        - geneos
        - collection-demo
      
      # Name of the data point.  Defaults to "kubernetes_event".
      data-point-name: kubernetes_event
    
    # Metrics module configuration
    metrics:
    
      # Whether metrics collection is enabled.  Defaults to true.
      enabled: true
      
      # Number of milliseconds between reporting intervals.  Defaults to 10 seconds.
      reporting-interval: 10000
      
  - type: plugin
    name: kube-logs
    class-name: KubernetesLogCollector
    
    # Container log directory, typically /var/log/containers.
    log-directory: /tmp/logs
    
    # Directory where the collector will save position files for each container log.
    # Must have read/write privileges to this directory.
    persistence-directory: /tmp/persistence
    
    # Whether to read newly discovered log files from the beginning of the file.
    # If false, only lines written to the log after the collector starts will be read.
    # Defaults to false.
    read-from-beginning: false
    
    # Number of worker threads (i.e. concurrent log readers).  Increasing this may improve 
    # performance, especially if there are several very active log files.
    # Defaults to 5.
    worker-threads: 5
    
    # Number of milliseconds to wait before pausing a worker that is blocking other workers from running.
    # Defaults to 30 seconds.
    long-running-worker-threshold: 30000

    # Number of milliseconds between log processing intervals, i.e. how long to wait before checking
    # if a log has new data to read. 
    # Defaults to 5 seconds.
    processing-interval: 5000
    
    # Glob patterns to include specific logs in log-directory for processing.
    # Defaults to undefined (include all).
    includes:
      - "*namespace1*"
      - "*namespace2*"
    
    # Glob patterns to exclude specific logs in log-directory from processing.
    # If a log file matches both an include and exclude, the exclusion will take precedence.
    # Defaults to undefined (no exclusions).
    excludes:
      - "*namespace3*"  

workflow:
  common:
    processors:
      - type: plugin
        name: kube-enricher
        class-name: KubernetesEnricher
        
        # Name of the enriched dimension that represents the application name.  Defaults to 'kubernetes_app_name'. 
        app-dimension: kubernetes_app_name	

Load an include file

Load an include file

Example include files are provided for Gateway configuration. To load an include file into the Gateway Setup Editor:

1. Open the Gateway Setup Editor.
2. In the Navigation panel, click Includes to create a new file.
3. Enter the location of the file to include in the Location field.
4. Update the Priority field. This can be any value except 1. If you input a priority of 1, the Gateway Setup Editor returns an error.
5. Expand the file location in the Include section.
6. Select Click to load.
7. Click Yes to load the new include file and save your setup.

The example includes file for the Kubernetes Collection Agent plug-in is provided in the downloaded binaries at /include/kubernetes.xml.

Collected metrics

All the metrics are collected from the Summary API of each node.

Node metrics

Note: Filesystem metrics for a node represent the root filesystem whose volume_name dimension is fs by default.

Pod metrics

Container metrics