Configure language models

Beta

The ITRS Agentic SRE app is currently available as a beta release. Features, behavior, and documented procedures are subject to change without prior notice during ongoing development. For information about major and minor updates, see the Agentic SRE app release notes.

Use Admin > AI > LLMs to add, test, and manage the language models that the Agentic SRE app can use for chat, retrieval, and evaluation.

Use this page when you need to:

• add a new model provider for chat or evaluation
• update model settings such as endpoint, model name, or credentials
• verify that a configured model is ready before making it available to users
• review which models are available for selection in chat
• send one model’s requests through an HTTP proxy

Important

Starting with Agentic SRE app version 0.5.0, provider API keys and tokens are expected to be stored in the Secrets app instead of directly in the LLM configuration. If you upgrade from an earlier beta release, review any previously configured LLMs and recreate or update them to use API Key (Secret).

Add an LLM Copied

Admin access

The procedures on this page are for users with access to Admin > AI. The AI chat is not usable until at least one LLM is available for selection. For day-to-day chat, see Use Agentic SRE for operations and support in ITRS Analytics.

1. In the Web Console, go to Admin > AI > LLMs.

2. Click Add LLM to create a new entry, or copy an existing LLM configuration.

3. Select the provider that you want to use.

4. Enter the required provider settings, such as model name, endpoint or base URL, credentials, and any provider-specific options.

5. In API Key (Secret), select the secret that stores the provider credential.

6. If no secrets are available, create the required secret first in the Secrets app, then return to the LLM configuration and select it. See Store provider credentials in the Secrets app.

7. Use the masked credential fields and reveal control only when you need to verify an entered value.

8. Click Test to verify that the model connection works as expected. The control shows progress followed by a success or failure icon. Point to the control to review the response or failure reason, and click it again when you want to rerun the test.

   

9. Save the LLM.

10. Open the AI chat and select the new LLM from the prompt toolbar.

Before you save a model, confirm the following:

• the provider is correct
• the model name matches the model that you intend to use
• the endpoint or base URL is valid for that provider
• the credential is stored as a secret reference
• the Test action succeeds

Depending on your deployment, available providers can include managed services and self-hosted options.

After you save a model, you can confirm that chat can use it from Review readiness, audit, and exports.

Configure a Google GenAI model Copied

Google GenAI is available as an LLM provider. When adding a Google GenAI model, configure the following required settings:

• API Key (Secret) — The secret that stores the Google GenAI API key
• Model — The Gemini model that the app uses
• Base URL — The Google GenAI service endpoint

You can also configure Temperature, Max Tokens, Top P, and Top K. Leave optional settings unchanged to use the provider defaults.

For supported Gemini models, use either Thinking Level or Thinking Budget to limit how much reasoning the model performs. Do not configure both settings for the same LLM. Leave both settings unset to use the model default.

Configure reasoning for an OpenAI model Copied

For OpenAI models that support reasoning, use Reasoning Effort to control how much reasoning the model performs. The available values depend on the selected model. Models that do not support this setting do not show the field.

Route an LLM through an HTTP proxy Copied

Use the Proxy section when this model can be reached only through an HTTP proxy. The proxy applies to that LLM only. Another LLM can connect directly. Calls that stay inside the cluster are not sent through the proxy.

The section is hidden for a model that runs inside the cluster. A proxy cannot be set on that model.

1. In Admin > AI > LLMs, open the LLM.

2. In Proxy, enter the following:

   • Proxy URL — http://host:port. The port is required. Leave this blank to connect directly. Use http:// even when the model endpoint is https://. The app opens a tunnel through the proxy, and the connection to the provider runs inside that tunnel. A proxy that accepts only TLS is not supported. Do not put a username or password in the URL, and do not add a path.
   • Proxy Username — Only when the proxy requires a login.
   • Proxy Password (Secret) — The secret in the Secrets app that stores the proxy password.

3. Click Test. The test uses the proxy values currently on the form.

4. Save the LLM. Saving applies the proxy. You do not need to restart the app.

If the proxy rejects the login, the test and the chat report that the proxy rejected its credentials (HTTP 407). After you rotate the proxy password in the Secrets app, save the LLM again so the model is rebuilt with the new password. A rejected proxy login does not by itself cause the app to re-read that password.

If the proxy re-signs TLS certificates, its certificate authority must be trusted by the Agentic SRE service. That is a deployment task. It is not a field on the LLM form.

Store provider credentials in the Secrets app Copied

Beta

The Secrets app and the Agentic SRE app are both currently available as beta releases.

If your deployment includes the Secrets app, use it to store provider credentials and reference them from the Agentic SRE app. For how to enable the Secrets app with Agentic SRE, see Enable Agentic SRE with a license entitlement.

1. In the Secrets app, create a secret for the provider credential that the LLM will use.
2. Assign an owner role that matches the users who are allowed to use that secret.
3. Save the secret.
4. In Admin > AI > LLMs, open the LLM configuration.
5. In API Key (Secret), select the stored secret instead of entering the credential value directly.
6. Save the LLM configuration and test the connection.

This keeps provider credentials in one managed location and avoids storing provider API keys directly in day-to-day LLM configuration.

After you rotate an API key in the Secrets app, a call that the provider rejects causes the app to read the new secret. A replacement that the provider does not reject is picked up on the next lookup after the cache interval, which is 15 minutes by default. For a proxy password, save the LLM again after you rotate the secret. See Route an LLM through an HTTP proxy.