--- title: "Configuring Dynamic Thresholds" description: "" url: "https://docs.itrsgroup.com/docs/opsview/6.12.6/enterprise/dynamic-thresholds/configuring-dynamic-thresholds/index.html" --- Before configuring Dynamic Thresholds, ensure that the [prerequisites](https://docs.itrsgroup.com/docs/opsview/6.12.6/enterprise/dynamic-thresholds/dynamic-thresholds/index.html#prerequisites) have been met. ## Exporting data to ITRS Analytics Exporting data to ITRS Analytics requires configuring the Results Exporter in Opsview. ## Create an ITRS Analytics User To create and configure users in ITRS Analytics, follow the instructions in [Role-based access control (RBAC)](https://docs.itrsgroup.com/docs/itrs-analytics/platform/current/install/advanced-keycloak/keycloak-rbac/index.html) page from the ITRS Analytics documentation. To import Dynamic Thresholds from an ITRS Analytics instance, the ITRS Analytics Importer process user must be configured with the credentials of an ITRS Analytics user having appropriate permissions. For on-premises ITRS Analytics instances, creating a dedicated user for this purpose is recommended. In this example, a user named `dynamic_thresholds` is created. 1. Log in to the ITRS Analytics instance's **Keycloak** page as administrator. 2. Switch to the `obcerv` realm by selecting it from the dropdown menu. 3. Create a new user with the username `dynamic_thresholds`. 3. Add a password to the `dynamic_thresholds` user by navigating to **User > Credentials > Set Password**. Ensure the **Temporary** option is not selected. 4. Add the `user` role to the `dynamic_thresholds` user by navigating to **User > Role Mappings > Assign Roles**. You may need to filter by realm roles to see the `user` role. **Note** To keep the system secure, it is recommended to create a new policy in ITRS Analytics's Keycloak to allow the `dynamic_thresholds` user access to only the necessary entities. 5. Navigate to **Clients > obcerv-platform > Authorization > Policies**, and then create the following client policies of user type. See [examples in ITRS Analytics](https://docs.itrsgroup.com/docs/itrs-analytics/platform/current/install/advanced-keycloak/keycloak-rbac/index.html#worked-example) documentation for reference. ``` Name: allow-ov-dynamic-thresholds Users: dynamic_thresholds Logic: Positive ``` ``` Name: deny-ov-dynamic-thresholds Users: dynamic_thresholds Logic: Negative ``` 6. Navigate to the **Authorization > Scopes** tab, and then create the following scope: ``` Name: hierarchy = [host, service] Display Name: Opsview Dynamic Thresholds rules ``` 7. Navigate to the **Authorization > Resources** tab, and then open the `entities` resource and add the newly created authorization scope to the `Authorization scopes` field. You may need to add a display name to save changes. 8. Create a new permission to deny a user the read scope for the resource. Navigate to the **Authorization > Permissions** tab, and then click **Create scope-based permission** to provide the following: ``` Name: deny-entities-read-scope Resource: entities Authorization scopes: read Policies: deny-ov-dynamic-thresholds Decision Strategy: Unanimous ``` 9. At this point, the user has no available scopes and cannot view any metrics from the query service. So, you must add a permission to allow the authorization scope created above. Return to the **Authorization > Permissions** tab, and then click **Create Permission > Create scope-based permission** to provide the following: ``` Name: allow-entities-for-ov-dynamic-thresholds Resource: entities Authorization scopes: hierarchy = [host, service] Policies: allow-ov-dynamic-thresholds Decision Strategy: Unanimous ``` ## Configure backend The Dynamic Thresholds backend is configured by Opsview Deploy. While several options are available for backend configuration, the following must be specified in the user variables file at `/opt/opsview/deploy/etc/user_vars.yml`. ```yaml # Hostname of the Analytics server to connect to for importing dynamic thresholds opsview_dynamic_thresholds_analytics_host: analyticsinstance.domain.com # Username of the Analytics user used to connect to the Analytics server opsview_dynamic_thresholds_analytics_username: dynamic_thresholds # Password of the Analytics user used to connect to the Analytics server opsview_dynamic_thresholds_analytics_password: amFtaWVsdWNrZXR0NzA3 ``` Optionally, you can enable or disable server certificate checking. This defaults to true, and it uses the system CA certificate store to verify the ITRS Analytics server certificate: ``` # Whether to check the CA certificate of the server opsview_dynamic_thresholds_obcerv_verify_cert: false ``` Further optional configuration options are documented in `/opt/opsview/deploy/etc/examples/user_variables-example.yml`. Once the configuration is added, run the `sys-install.yml` playbook with Opsview Deploy to apply the changes. ```shell cd /opt/opsview/deploy ./bin/opsview-deploy ./lib/playbooks/sys-install.yml ``` ### Retrieve ITRS Analytics ingestion credentials You need to retrieve the ITRS Analytics ingestion password and ingestion certificate to configure the Results Exporter to export data to ITRS Analytics successfully. #### Ingestion host and certificate Access the ITRS Analytics UI and navigate to **Admin > Certificate**. You can extract the ingestion host from the present URL by removing the scheme and port sections; for example, `my.host.com` is the ingestion host taken from the URL `https://my.host.com:123`. The ingestion certificate contents can then be copied and saved to a file on your orchestrator system. This file must be readable by the `opsview` user and in a valid PEM format. #### Ingestion password You can retrieve the initial password for the `ingestion-api` user by running the following `kubectl` command against the ITRS Analytics cluster. ``` kubectl -n get secret iam-initial-credentials -o jsonpath='{.data.obcerv-ingestion-api}' | base64 -d ``` If the password has been changed, you can reset it in the Keycloak UI: 1. Log in to the ITRS Analytics instance's **Keycloak** page as an administrator. 2. Switch to the `obcerv` realm by selecting it from the dropdown menu. 3. Navigate to the **Users** page and select the `ingestion-api` user. 4. Select the **Credentials** tab and click **Reset Password**. 5. Enter the new password. Ensure the **Temporary** option is not selected. 6. Use the updated password in the Results Exporter configuration. ### Configure Results Exporter in Opsview Dynamic Thresholds relies on metric timeseries from an [ITRS Analytics instance](https://docs.itrsgroup.com/docs/itrs-analytics/platform/current/install/install-overview/index.html). To export this data to an ITRS Analytics system, you must configure the [Results Exporter](https://docs.itrsgroup.com/docs/opsview/6.12.6/administration/distributing-functionality/results-exporter/index.html) component. To export data to an ITRS Analytics instance, add the following configuration to your Results Exporter configuration file at `/opt/opsview/deploy/etc/user_results_exporter.yml`. If `opsview_results_exporter_outputs` is already defined, then add the `analytics` output to the existing configuration. ```yaml opsview_dynamic_thresholds_ingestion_host: analyticsinstance.domain.com opsview_dynamic_thresholds_ingestion_password: your-ingestion-api-password opsview_results_exporter_outputs: analytics: dynamic_threshold_metrics: type: dynamic_thresholds filter: '*' parameters: host: '{{ opsview_dynamic_thresholds_ingestion_host }}' port: 443 cert_path: /path/to/analytics/ingestion/cert.pem username: ingestion-api password: '{{ opsview_dynamic_thresholds_ingestion_password }}' ``` Once the configuration is added, run the `results-exporter-install.yml` playbook with Opsview Deploy to apply the changes. ```shell cd /opt/opsview/deploy ./bin/opsview-deploy ./lib/playbooks/results-exporter-install.yml ``` ## Configure Dynamic Thresholds for Service Checks After integrating Opsview with ITRS Analytics and setting up the Results Exporter, you can configure and manage Dynamic Thresholds for service checks through the UI. The Dynamic Thresholds page is only visible to users with `ADMINACCESS` permission, and it only displays host-services the current Opsview user has access to. For more information, see [Adding a New Role](https://docs.itrsgroup.com/docs/opsview/6.12.6/configuration/users-and-roles/roles/index.html#adding-a-role). Watch the following product tour to learn how to enable Dynamic Thresholds, verify plugin arguments, and test that your service checks are properly configured. https://app.storylane.io/demo/hix6peeygkdk **Note** When the service check runs, the `$WARNINGDT;DEFAULTVALUE;HARDLIMIT$` and `$CRITICALDT;DEFAULTVALUE;HARDLIMIT$` macros are replaced with the current threshold values if Dynamic Thresholds has been correctly configured. If Dynamic Thresholds is not configured properly or applied to the check, the default value will be used; the value is never allowed to exceed the hard limit. ## Limitations - Dynamic Thresholds are only available for service checks that return a single metric value. - Dynamic Thresholds currently only support service checks that calculate thresholds as `VALUE > THRESHOLD_VALUE`. Less-than thresholds are not supported. - Dynamic Thresholds only support service checks that include the Dynamic Thresholds macros: `$WARNINGDT` or `$CRITICALDT` as values for warning and critical threshold arguments. - Currently only a subset of the bundled Service Checks support Dynamic Thresholds. - Dynamic Threshold macros are written in the format `$WARNINGDT;DEFAULTVALUE$`, where `DEFAULTVALUE` is the default threshold value if Dynamic Thresholds are not available/not configured. - Optionally, you can also set a hard limit for Dynamic Threshold macros. This is defined as `$WARNINGDT;DEFAULTVALUE;HARDLIMIT$`, where `HARDLIMIT` will cap any calculated dynamic thresholds at its value. Note that hard limits only apply to checks with Dynamic Thresholds enabled and do not affect default values. - Threshold prediction accuracy will improve based on the length of time data has been exported to ITRS Analytics. While one week of exported data is recommended before using Dynamic Thresholds, optimal accuracy is achieved with up to four weeks of data. - For newly exported host-service metric data, Dynamic Thresholds may not be available for a brief period until sufficient data is collected. ## Troubleshooting To help resolve any issues you may encounter during the configuration, refer to the [Dynamic Thresholds issues](https://docs.itrsgroup.com/docs/opsview/6.12.6/troubleshooting/dynamic-thresholds-issues/index.html) troubleshooting documentation.