Configuration reference

Important

This information refers to the previous helm install method for the ITRS Analytics Platform. If you are looking to install using the more streamlined Kubernetes Off-the-Shelf (KOTS) method, see the updated deployment options.

ITRS provides sample configuration files for installing an ITRS Analytics instance. You can download sample scenarios in Sample configuration files.

Advanced configuration settings are all optional.

You are not required to change the settings described here for a default installation of ITRS Analytics.

Resource settings Copied

For each ITRS Analytics workload there is a resources parameter that defines the resource requests and limits for the pods in that workload. We recommend setting these parameters for all applications, otherwise some pods may consume all available resources on your Kubernetes cluster.

The provided configuration examples have baseline resources and limits set for all workloads. They can be altered as needed, for example:

kafka:
  resources:
    requests:
      memory: "1Gi"
      cpu: "500m"
    limits:
      memory: "2Gi"
      cpu: "1"

ClickHouse volume sizing Copied

The current sample configuration files for fresh installs use clickhouse rather than timescale. Each ClickHouse workload has its own disk settings under clickhouse.platform, clickhouse.metrics, clickhouse.logs, and clickhouse.traces.

clickhouse:
  traces:
    diskSize: "80Gi"
  metrics:
    diskSize: "500Gi"
    diskCount: 4
    storageClass: "gp3-clickhouse"
  platform:
    diskSize: "100Gi"
    storageClass: "gp3-clickhouse"
  logs:
    diskSize: "500Gi"
    storageClass: "gp3-clickhouse"
  keeper:
    replicas: 3

Use diskSize to set the size of the persistent volume for a ClickHouse workload. Each ClickHouse workload also supports diskCount, which can be used to distribute data across multiple disks. In practice, this is most likely to be used for traces, metrics, and logs, but it is available for all ClickHouse workloads. You can also set storageClass per workload when ClickHouse data needs a different storage class from defaultStorageClass.

When clickhouse.metrics.diskCount is greater than 1, calculate the total metrics disk capacity per replica using the following formula:

clickhouse.metrics.diskSize * clickhouse.metrics.diskCount

The clickhouse.keeper.replicas setting controls the size of the ClickHouse Keeper cluster used by replicated ClickHouse deployments.

Note

The nginx sample configuration files still include commented timescale and loki sections for upgrades from pre-2.18 deployments. These are upgrade-only examples and are not used for fresh installs.

Kafka retention settings Copied

The kafka.defaultRetentionMillis parameter controls how long data is retained in Kafka topics, which defaults to 21600000 (6 hours).

ClickHouse on reserved nodes Copied

For larger deployments, you may want to run ClickHouse workloads on reserved Kubernetes nodes. The current medium and large sample configurations do this by using Kubernetes labels and taints on the nodes and nodeSelector and tolerations on the ClickHouse workloads.

  1. Add a label to the ClickHouse nodes in the manifest or by using kubectl:

    instancegroup: clickhouse-nodes

  2. Add a taint to the ClickHouse nodes in the manifest or by using kubectl:

    dedicated=clickhouse-nodes:NoSchedule

  3. Set the following for each ClickHouse workload that should run on the reserved nodes:

    clickhouse:
  metrics:
    nodeSelector:
      instancegroup: clickhouse-nodes
    tolerations:
      - key: dedicated
        operator: Equal
        value: clickhouse-nodes
        effect: NoSchedule

Apply the same pattern to clickhouse.platform, clickhouse.logs, and clickhouse.traces if those workloads should also run on the reserved nodes.

Ingestion services Copied

ITRS Analytics supports ingesting data via two gRPC endpoints, which are both serviced from the same external host name (as defined by ingestion.externalHostname) and port (443).

The current sample configuration files tune ingestion by setting ingestion and ingestion.traces resources and producer settings, for example:

ingestion:
  externalHostname: "iax-ingestion.mydomain.internal"
  ingress:
    className: "nginx"
    annotations:
      nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
  traces:
    sampler:
      expectedSpanRate: 20000
    producerProperties:
      buffer.memory: 67108864

If your deployment ingests OpenTelemetry traces, review the ingestion.traces settings in the sample configuration that most closely matches your expected load. The larger sample configurations also override ingestion.traces.resources to account for higher span rates.

OpenTelemetry ingestion service Copied

While raw traces are currently not stored, the OpenTelemetry ingestion service in the ITRS Analytics platform captures the following metrics (where <span> is the span name):

When pointing your instrumentation to the ITRS Analytics OpenTelemetry API, make sure to:

For more information, refer to the instrumentation guide from Open Telemetry.

Kubernetes log collection Copied

ITRS Analytics is configured to collect logs for Kubernetes pods and containers in the configured namespaces, or the ITRS Analytics namespace by default. This behavior is controlled by the following setting:

collection:
  logs:
    enabled: true
    # Restrict collection to specific namespaces. If empty, only the ITRS Analytics installation namespace
    # will be collected. To collect all, specify "*".
    namespaces: []

Note

For the log collection process to function properly, the obcerv-operator Kubernetes service account must have appropriate permissions to create resources with read and write access to hostPath volumes. The process for granting these permissions may vary depending on your Kubernetes distribution.
["ITRS Analytics"] ["User Guide", "Technical Reference"]

Was this topic helpful?