---
title: "Grafana Data Source"
description: "Integrate ITRS Analytics with Grafana. Configure the data source and query metrics and logs in your dashboards."
url: "https://docs.itrsgroup.com/docs/itrs-analytics/app/grafana-datasource/3.4.0/user-guide/grafana/index.html"
---

**Note**

 The Grafana data source version 3.4.1 requires ITRS Analytics API Gateway version 2.12.0 or later for proper functionality. 


Grafana is an open source visualization tool which can be used to visualize data from ITRS Analytics. If you have existing Grafana installs, you can seamlessly integrate data from ITRS Analytics into your current dashboards.

You can also create visualizations using [ITRS Analytics Dashboards](https://docs.itrsgroup.com/docs/itrs-analytics/app/dashboards/current/user-guide/about-dashboards-app/index.html).

## Prerequisites

  
**Note**


Starting with Grafana data source version 3.0.2, the new ITRS Analytics data source plugin replaces the previous ITRS Group Obcerv plugin. The old ITRS Group Obcerv plugin is no longer supported.



- Grafana (version 12.x or higher) is installed and running. For more information, refer to the [Grafana installation documentation](https://grafana.com/docs/grafana/latest/setup-grafana/installation/).

- The Grafana and ITRS Analytics Keycloak integration is configured and functioning correctly. For guidance, see [How to configure Grafana to authenticate users via ITRS Analytics](https://docs.itrsgroup.com/docs/itrs-analytics/latest/faqs/how-to-configure-grafana-authentication/index.html).

- The [API Gateway](https://docs.itrsgroup.com/docs/itrs-analytics/app/api-gateway/current/user-guide/api-gateway/index.html) is deployed and exposes the Public API v2. 

  - Metric queries require API Gateway 2.7.0 or later, and log queries require API Gateway 2.10.0 or later.
  - API Gateway 2.12.0 or later is required to support filtered lookups for chained query variables, value suggestions in the metrics **Filter** field, and dashboard template variables in the **Dimensions** (**Key** and **Value**) fields of advanced log queries.

- Log data is ingested into ITRS Analytics for the entities and time range you want to inspect.

## Configure the data source

To download and install the plugin, choose one of the following options:

- From the [Grafana Plugins](https://grafana.com/grafana/plugins/itrs-analytics-datasource/) website
- By manually downloading the Grafana data source from the [ITRS Resources](https://resources.itrsgroup.com/downloads#package-grafana-data-source) website

Once you have logged into your instance of Grafana, you must configure the ITRS Analytics data source.

To add a new data source:

1. Go to **Administration > Data sources** in the Grafana navigation panel.
2. Click **Add data source**, search for `ITRS Analytics` and select it.
3. Specify the data source settings. The following settings are available:

    ![ITRS Analytics Data sources](https://docscdn.itrsgroup.com/itrs-analytics-grafana-datasource/images/iax-grafana-connect-to-data-source.png)

    | Field         | Description                                                                |
    | --------------------- | -------------------------------------------------------------------------- |
    | Name | Specify a display name for the data source. |
    | ITRS Analytics API    | Host and port of the ITRS Analytics REST API, for example, `https://local.itrslab.com/obcerv-app-api-gateway` |
    | Skip tls verification | If you want to skip the TLS verification.          |

    - You can also click on the **Default** switch on the top of the screen if you want it to appear as default datasource.

4. Click **Save & test** to save the configuration.

    For more information, see the official Grafana documentation to add a data source [here](https://grafana.com/docs/grafana/latest/datasources/add-a-data-source/).

## Query the data source

To visualize data from ITRS Analytics, create a panel in a new or existing Grafana dashboard.

1. Open a dashboard or create a new dashboard.
2. Click **New > New dashboard**, and then click **Panel**.
3. Click **Configure visualization** from the main screen or **Configure** from the panel configuration window on the right.
4. Under **Queries**, set the data source to **ITRS Analytics**.
5. Set **Mode** to **Metrics** or **Logs** and specify the query parameters for that mode.


**Note**

 A dashboard can combine metrics and logs from ITRS Analytics. Use separate panels for metrics and logs so each panel uses the visualization type that fits the data (for example, time series for metrics and logs for log lines). Within a panel, you can add multiple queries by clicking **Add query** to define additional **Metrics** or **Logs** queries, each with its own parameters. 


### Query metrics

Set **Mode** to **Metrics** and specify the query parameters:

  ![ITRS Analytics query editor image](https://docscdn.itrsgroup.com/itrs-analytics-grafana-datasource/images/iax-grafana-query-data-source-3-3.png)

  | Field           | Description                                                                                                                                                       |
  | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
  | Alias           | Displayed in the legend to identify a time series. |
  | Filter          | Filters will be available for auto-completion when composing a query, they are in the form `attribute/dimension="value"`, for example `pod="collection-agent-sx6cv"`. |
  | Metric          | Name of the metric. Metrics will be available for auto-completion. |
  | Group by        | Dimensions or attributes to group by the results. |
  | Buckets         | Used to populate the `Buckets` in `BucketRequest`. |
  | Bucket function | Used to populate the `Function` in `BucketRequest`. |
  | Forecast        | If enabled, returns the forecast of the selected metric. This only works if the filter returns only one metric. |

To use status metrics, select the metric and select **All fields** to display non-numeric fields:

![Status metrics](https://docscdn.itrsgroup.com/itrs-analytics-grafana-datasource/images/iax-grafana-all-fields-3-3.png)

### Query logs

The data source can also return log lines from ITRS Analytics. Use **Logs** mode to search ingested logs and display them in Grafana's **Logs** panel.

To create a logs panel:

1. Add or edit a panel and select the **Logs** visualization type.
2. Under **Queries**, select **ITRS Analytics** as the data source.
3. Set **Mode** to **Logs**.
4. Choose whether you want to run a **Simple** or **Advanced** search. Complete the required fields and run the query.
5. Use the dashboard time picker to set the range of log entries to retrieve.

#### Simple search

![Log query simple search](https://docscdn.itrsgroup.com/itrs-analytics-grafana-datasource/images/iax-grafana-log-query-simple-search-3-3.png)

Simple search is the fastest way to get started when you know roughly which log file you need but not its full entity context.

1. Set **Search** to **Simple**.
2. Click **Log name** and type a keyword to filter the list of known log sources.
3. Select a log source from the dropdown.

The data source discovers log sources by calling the API endpoint `POST /api/v2/logs/log-sources` for the current dashboard time range (up to 10,000 sources per request). Options are filtered by the text you type against the actual log name.

Each option shows:

- **Label** — the log source name (for example `output.log`).
- **Description** — the log namespace and a summary of entity dimensions (for example `itrsgroup.com/c2/kubernetes-plugin | pod=my-pod, namespace=itrs`).

When you select a source, the data source stores the log name, namespace, and entity dimensions from the API response and runs the query automatically. Clearing the selection removes those values.

#### Advanced search

![Log query advanced search](https://docscdn.itrsgroup.com/itrs-analytics-grafana-datasource/images/iax-grafana-log-query-advanced-search-3-3.png)

Advanced search lets you specify the log source manually and refine which log lines are returned.

1. Set **Search** to **Advanced**.
2. Enter **Log name** and **Log namespace**.
3. Add one or more **Dimensions** rows:
   - **Key** — type to search dimension names. Suggestions come from `POST /api/v2/lookup/names` (dimensions only). Select a name from the list.
   - **Value** — type to search values for the selected key, or enter a dashboard template variable with a `$` prefix (for example `$managedEntity` or `$host`). Suggestions come from `POST /api/v2/lookup/values`. When the value contains `$`, the platform lookup is skipped and the variable is resolved when the query runs.
   - Click **Add dimension** for additional key/value pairs, or **Remove** to delete a row.
4. Optionally refine the query using the fields below:

    - **Message contains** — one substring per line. Log lines match if the message contains **any** of these substrings (OR logic).
    - **Trace id** — return only log lines with this trace ID.
    - **Span id** — return only log lines with this span ID.
    - **Max lines** — maximum number of log lines to return. Leave empty to use the API default (1000 lines). The API accepts up to 10,000.
    - **Oldest first** — when enabled, results are sorted oldest-to-newest. When disabled (default), results are newest first.
5. Click **Run query**.

Fields marked with * are required before the data source sends a log query. If any required value is missing, the panel returns no log lines. The data source sends the completed query to `POST /api/v2/logs/log-data`.

#### Log line results

![Log results](https://docscdn.itrsgroup.com/itrs-analytics-grafana-datasource/images/iax-grafana-log-results-3-3.png)

Each returned log line includes:

| Field | Description |
| ----- | ----------- |
| timestamp | When the log entry was recorded. |
| body | The log message text. |
| severity | Log severity as a lowercase label (for example `error`, `warning`, `info`). |
| labels | Entity dimensions, log namespace, log name, and any log attributes returned by the API. |

If a log line includes a trace ID, Grafana can use it for trace correlation in supported visualizations.

## Templates and variables

You can define variables in Grafana that allow you to dynamically change the data displayed on your dashboards. Variable types include **Query**, **Textbox**, **Constant**, **Custom**, **Data source**, and **Interval**. Use the ITRS Analytics data source to populate variables and reference dashboard variables in panel queries.

Dashboard template variables apply to:

- **Metrics** — **Filter** expression values (for example `pod=$pod`).
- **Advanced** search in log queries — **Dimensions** values (for example Key `managedEntity`, Value `$managedEntity`).
- **Query** variables — JSON lookups that load dimension or attribute values from ITRS Analytics.

You can use common Grafana dashboard variable types in those fields by referencing them with a `$` prefix.


**Note**

 You can enable **Multi-value** selection on only one dashboard variable when using the ITRS Analytics data source. If that variable has several values selected, the data source runs a separate query for each value and combines the results. 


### Use variables in metrics and logs queries

In a metrics panel, compose a regular **Filter** and enter the variable with a `$` prefix. For example, `pod=$pod` or `container=$containerVar`. When the dashboard variable changes, the filter is resolved at query time and the panel refreshes.

In an advanced logs search, select the dimension **Key** (for example, `managedEntity` or `host`) and enter the dashboard variable in the **Value** field (for example, `$managedEntity` or `$host`). Resolved dimension values are sent to `POST /api/v2/logs/log-data` when the query runs. Changing the dashboard variable refreshes the log panel without re-editing the dimension fields.

### Add a datasource variable

If your Grafana instance has more than one ITRS Analytics data source (for example, different environments), you can add a **Datasource** variable so users pick which one a panel uses. Create a datasource variable that lists only ITRS Analytics data sources, then set the panel’s data source to that variable (for example `$datasource`). Changing the variable switches the panel to the selected data source.

### Add a query variable

To add a query variable that loads values from ITRS Analytics:

1. Go to the **Dashboards** and click **Edit** to switch to edit mode.
2. Open the **Dashboard settings**.
3. Click **Variables > Add variable**.
4. Set **Variable type** to **Query**, then configure the variable using the following options:

    ![variable configuration image](https://docscdn.itrsgroup.com/itrs-analytics-grafana-datasource/images/iax-grafana-variables-3-3.png)

    | Field           | Description                                                                                                                                                          |
    | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
    | Name            | Variable name. Refer to this name in panels and other variables with a `$` prefix.                                                          |
    | Label           | Label of the control displayed on the dashboard.                                                                                                                        |
    | Data source     | Specify `ITRS Analytics`. |
    | Query           | JSON query used to populate the variable options. See [Grafana query syntax](#grafana-query-syntax). |

5. (Optional) Enable **Multi-value** if you want to choose more than one value for the variable. Only one multi-value variable is supported per dashboard that uses this data source.
6. To see a preview of the values, click **Run query**.

### Grafana query syntax

The syntax for a dimension query variable is JSON with the form:

```json
{ "name": "<dimension>" }
```

For example:

```json
{ "name": "container" }
```

For an attribute, include the `namespace` parameter:

```json
{ "namespace": "<namespace>", "name": "<attribute>" }
```

For example:

```json
{ "namespace": "itrsgroup.com/geneos", "name": "source_gateway" }
```


**Caution**


If you are using API Gateway 2.12.x and 2.13.x, an attribute query variable returns no values when the `namespace` parameter is omitted.



### Chained query variables

Query variables can be filtered by another variable, dimension, or attribute. The data source applies the filter when it looks up values (`POST /api/v2/lookup/values`), so dependent variables only list values that match the selected parent. The syntax is:

```json
{ "name": "<dimension-or-attribute>", "filter": [{ "name": "<dimension-or-attribute>", "value": "$variableName1" }] }
```

The `filter` is a list of objects with the fields `name` and `value`. The `name` must be an attribute or dimension name; the `value` can be a specific value or the name of another variable with the `$` prefix. For example, if a variable named `kubecontainer` exists, then the query could be:

```json
{ "name": "pod", "filter": [{ "name": "container", "value": "$kubecontainer" }] }
```

This allows variable chaining so the values of some variables can be used to filter other dependent variables. For example, you can list probes under a selected Gateway by filtering probe values on the selected Gateway dimension.

