Advanced configuration

This page covers advanced setting. A default installation of Obcerv does not require changes to the settings described here.

Note: For sample scenarios in installing the Obcerv instance, see Parameters file for the Obcerv instance.

Resource settings

For each Obcerv workload there is a resources parameter that defines the resource requests and limits for the pods in that workload. We recommend setting these parameters for all applications, otherwise some pods may consume all available resources on your Kubernetes cluster.

The provided configuration examples have baseline resources and limits set for all workloads. They can be altered as needed, for example:

kafka:
  resources:
    requests:
      memory: "1Gi"
      cpu: "500m"
    limits:
      memory: "2Gi"
      cpu: "1"

Timescale retention settings

To adjust metric data retention, update the timescale.retention parameter. The values are in the format of <number><time>, such as: 5d or 45m. The time format options are:

- m  (minutes)
- h  (hours)
- d  (days)
- mo (months)
- y  (years)
timescale:
  retention:
    metrics:
      chunkSize: 3h
      retention: 60d
      compressAfter: 3h
  ...

Kafka retention settings

The kafka.defaultRetentionMillis parameter controls how long data is retained in Kafka topics, which defaults to 21600000 (6 hours).

Timescale and Kafka on reserved nodes

For larger deployments, you may want to run Timescale and/or Kafka on reserved Kubernetes nodes. This can be achieved using Kubernetes labels and taints on the nodes and tolerations and nodeSelector on the workloads.

The following example shows how to deploy Timescale on reserved nodes:

  1. Add a label to the Timescale nodes in the manifest or using kubectl:

    instancegroup: timescale-nodes
    
  2. Add a taint to the Timescale nodes in the manifest or using kubectl:

    dedicated=timescale-nodes:NoSchedule
    
  3. Set the following in your parameters file:

     timescale:
       ha:
         enabled: true
         clusterSize: 3
       maxConnections: 100
       dataDiskSize: 100Gi
       walDiskSize: 50Gi
       resources:
         requests:
           memory: "12Gi"
           cpu: "3"
         limits:
           memory: "12Gi"
           cpu: "12"
       compressAfter: 1h
       retention:
         entity_attributes:
           chunkSize: 2d
           retention: 2y
         metrics:
           chunkSize: 20m
           retention: 60d
    
  4. (Optional) In order for Obcerv to collect pod logs from the reserved nodes, the following tolerations are required so that the logs agent can run on these nodes:

    collection:
      daemonSet:
        tolerations:
        # must match the tainted node setting 
        - key: dedicated
          operator: Equal
          value: timescale-nodes
          effect: NoSchedule
        - key: dedicated
          operator: Equal
          value: kafka-nodes
          effect: NoSchedule
    
["Obcerv"] ["User Guide", "Technical Reference"]

Was this topic helpful?