Use Agentic SRE to investigate with entity context
Beta
The ITRS Agentic SRE app is currently available as a beta release. Features, behavior, and documented procedures are subject to change without prior notice during ongoing development. Starting with Agentic SRE app version 0.6.0, your license must include the Agentic SRE entitlement, and the app must be enabled in the KOTS Admin Console. For details, see Enable Agentic SRE with a license entitlement. For information about major and minor updates, see the Agentic SRE app release notes.
About the ITRS Agentic SRE app Copied
The ITRS Agentic SRE app is an AI-assisted capability within the ITRS Analytics Web Console. It helps users investigate operational conditions, ask product-related questions, and work from monitored entity context within a single interface.
The app provides an AI chat experience for operational and support workflows. Administrators prepare that chat from Admin > AI, where they configure LLMs, review service readiness, and review evaluation results before making models available for wider use.
Investigate and support Copied
Use these topics when you are investigating, asking for product guidance, or creating a dashboard from the chat:
- Explore telemetry with Insights — Show logs, metric charts, a severity timeline, or a trace waterfall for the question you asked.
- Create a dashboard from chat — Preview a dashboard, see which metrics were chosen, and save it only when you confirm.
This page covers how to open the chat, choose an agent, add entity context, review an answer, send feedback, and return to an earlier conversation.
Tasks for admins Copied
Use these topics when you have access to Admin > AI. Available sections can vary by deployment and assigned permissions.
- Configure language models — Add and test an LLM, store the provider credential in the Secrets app, and route a model through an HTTP proxy.
- Review readiness, audit, and exports — Check whether chat is ready, review recorded conversations, and export a conversation record.
- Evaluate model answers — Compare model output and review grader scores before wider use.
The Agentic SRE app provides the following capabilities for day-to-day use:
- Specialized agent modes such as
SRE,Insights,RCA, andSupport, each intended for a different operational or support task. - Entity-aware investigations that can find relevant monitored entities and help keep a conversation focused on selected services, hosts, or components.
- Exploratory telemetry answers in
Insightsthat show the records you asked for: a metric chart with severity bands, a severity timeline, a log grid, or a trace waterfall. - Follow-up questions that stay on the entity already in the conversation, and a time range taken from the period you name in the question.
- Inline entity references that let users add one or more monitored entities to a prompt from recently visited entities, entity cards, or the
@menu. - Structured responses that can include reasoning steps, supporting sources, tool activity, and generated outputs to help users review the response in context.
- Times in answers and telemetry views shown in the date format, time format, and time zone from your ITRS Analytics user settings.
- Response feedback controls that let users approve or disapprove an answer and optionally explain why an answer was not useful.
- Chat history controls that let users reopen, rename, archive, and restore previous conversations when chat history is enabled.
- Entity-focused dashboard workflows that preview a dashboard, explain which metrics were chosen, and save it only when you confirm.
Note
Available agent modes and admin features can vary by deployment, user permissions, and enabled capabilities.
Prerequisites Copied
Important
Starting with Agentic SRE app version 0.5.0, provider API keys and tokens are expected to be stored in the Secrets app instead of directly in the LLM configuration. If you upgrade from an earlier beta release, review any previously configured LLMs and recreate or update them to use API Key (Secret). For the procedure, see Configure language models.
Before using the Agentic SRE app, ensure that the following requirements are met:
- Your license includes the Agentic SRE entitlement, and the app is enabled in the KOTS Admin Console. For details, see Enable Agentic SRE with a license entitlement.
- ITRS Analytics Platform 2.17.0 or later and Web Console 3.13.0 or later. Agentic SRE app version 0.7.0 installs from the same package on Platform 2.18 and newer.
- At least one LLM is configured in Admin > AI > LLMs. The AI chat is not usable until an LLM is available for selection. See Configure language models.
- The Secrets app must be available in your deployment so provider credentials can be stored and referenced securely.
- If you want to use administrative features such as LLMs, Eval, or Audit, you must have access to Admin > AI.
If no LLM is configured, an administrator must add a provider in LLMs before you use the AI chat.
Use case scenarios Copied
Use the Agentic SRE app when you want faster guidance without switching between multiple tools or documentation sources.
Note
UseSREwhen you want a general entry point for the Agentic SRE app. TheSREagent acts as an orchestrator that can handle different types of requests and route them to the appropriate specialist workflow when needed.
Investigate what is happening in your environment Copied
Use Insights when you want either a status summary or the telemetry behind a question. A status question, such as whether an entity is healthy, returns a verdict and an entity tile. A question that asks to see logs, errors, traces, a metric over a period, or when something started returns the records themselves. For how those answers are shown, see Explore telemetry with Insights.
Typical tasks include:
- Starting with a single agent for operational, support, and investigation queries.
- Summarizing what changed in the last few minutes for one or more selected entities or services.
- Requesting an overview of the whole environment by stating the intended scope.
- Reviewing related signals, metrics, and surrounding context before escalation.
- Reading log lines, a metric chart, a severity timeline, or a trace waterfall for a named entity.
- Building a first-pass explanation of what is happening in the environment.
If Insights cannot resolve the entity named in a request, refine the entity reference or try the request again instead of relying on a platform-wide summary.
In most cases, you can send all tasks through SRE and let it direct the request to the most relevant workflow. This is useful when you need to move from an alert to a working hypothesis quickly, or when you are not sure which specialist agent to choose first.
Perform root cause analysis Copied
Use RCA when you already have an incident, service, or entity in scope and want a more structured investigation. Insights shows the evidence. RCA is the mode that can commit to a likely cause.
Typical tasks include:
- Narrowing the likely cause of a service issue by following related telemetry.
- Comparing current symptoms with the surrounding operational context.
- Focusing the investigation on the most relevant evidence first.
RCA reads the time the entity’s severity changed and treats that instant as the start of the problem. A time range in your question is when you started looking. It is not treated as the moment the problem began. If that start time cannot be determined, the answer says so and does not claim that a change came before or after the start.
It reads log lines from each relevant log source around that start time. A short deploy, release, or configuration line is included even when another source has many more lines. A change that happens shortly before the problem starts, and that explains the memory, latency, or error symptoms that follow, is treated as the likely cause. The resource symptom is treated as how the change showed up. The first suggested step is to undo that change, such as a rollback or a configuration revert, before tuning the symptom.
Kubernetes evidence is gathered only when the entity is identified as a Kubernetes object. A service with no Kubernetes identity, including a Geneos service that has none, is not investigated with Kubernetes steps.
Evidence already shown earlier in the conversation is used. A follow-up such as “Why is it red?” stays on the entity from the previous turn. A question about how to configure the product is still answered from documentation.
For example, you might ask, “What changed in the last 30 minutes for this service?” after selecting the affected entity in Entity Viewer.
Get product and setup guidance Copied
Use Support when you need help with product features, setup tasks, or product documentation.
Typical tasks include:
- Asking how to complete a configuration task in ITRS Analytics.
- Requesting product guidance for onboarding or troubleshooting.
- Searching across supported documentation sources without leaving the Web Console.
For example, you can ask for help creating a configuration template or for guided steps to complete a setup task.
You can also start these requests in SRE and allow it to route the query to the appropriate support-oriented workflow.
Create dashboards from AI results Copied
Use the chat when you want a dashboard for a selected entity or a named scope. The result is a preview until you save it. The reply names the metrics that were chosen and why, and you confirm with Save or Discard. For the layout, the preview lifecycle, how to export the dashboard JSON, and known limitations in the Dashboards app, see Create a dashboard from chat.
Entity cards in a response provide separate actions for opening the related entity context or adding the entity to a prompt.
How to use the AI chat in the ITRS Analytics Web Console Copied
Follow these steps to start using the AI chat in the Web Console:
- In the ITRS Analytics Web Console, open the AI chat from the header.
- In the prompt toolbar, choose the agent mode that best matches your task.
- Select the LLM that you want to use.
- Enter your prompt.
- If you are already working in Entity Viewer, select the relevant entity to make it available among the recently visited entities in AI chat.
- Add each relevant entity to the prompt before sending it.
Use the agent modes as follows:
SRE— Use this as the primary agent for most requests. It can handle general operational questions and broad troubleshooting, and it can route the request to the appropriate specialist workflow when needed.Insights— Use this for entity summaries, exploratory telemetry, change investigation, and explicitly requested system-wide overviews. See Explore telemetry with Insights.RCA— Use this for focused incident investigation and root cause analysis.Support— Use this for product how-to guidance, setup help, and documentation-based questions.
Add entity context to a prompt Copied
Add entity references when you want an answer to focus on specific monitored entities. Entity references appear as chips within the prompt, so you can review the scope before sending the request.
To add entity context:
-
Open or select an entity in the Web Console. Recently visited entities appear above the prompt.
-
Add an entity by using one of the following methods:
- select Add to chat on the entity chip
- open Contexts to apply and add one or all recently visited entities
- enter
@in the prompt and select an entity from the filtered list - select Add to chat on an entity card in an AI response
-
Repeat the process to include more than one entity.
-
Review the entity chips within the prompt, then send the request.
You can select an entity chip to open the related entity.
When you reopen a conversation from chat history, entity references in earlier prompts are shown as chips again, and charts, grids, timelines, and entity tiles remain in the answer.
Answer a question from the chat Copied
When the chat needs you to choose, it shows numbered options. Click an option, or type its number in the prompt. The number is the position of the option, starting at 1. This applies to choices such as which entity you meant, and to Save and Discard on a dashboard preview.
Review the AI response Copied
After you send a prompt, review the response details before acting on the recommendation:
- Check the main answer for the recommended next step.
- Review a metric chart, severity timeline, log grid, or trace waterfall when the answer includes one. See Explore telemetry with Insights.
- Review reasoning steps when you want to understand how the answer was built.
- Review sources when the response includes documentation links or cited references.
- Review tool activity and generated outputs when the app provides structured results.
- Click entity cards to continue the investigation from the related entity context.
- Compare times in the answer with your user settings. The chat formats them in your date format, time format, and time zone, and appends the zone abbreviation.
Provide feedback on an AI response Copied
If response feedback is enabled, use the thumbs-up or thumbs-down controls below an AI response to record whether it was useful.
- Select thumbs up to approve the response
- Select thumbs down to disapprove the response and optionally explain what was wrong
- Select the active control again to remove your feedback
Submitted feedback remains associated with the response and is available to administrators in Audit. See Review AI activity in Audit.
Use chat history if it is enabled Copied
If chat history is enabled in your deployment, you can reopen earlier conversations from the history panel.
Use chat history to:
- Start a new chat from the history panel.
- Resume a previous investigation.
- Rename a conversation so it is easier to find later.
- Archive conversations that you no longer need in the active list.
- Open the archived list and restore a conversation when you need it again.
- See how many conversations are in the list you are viewing. The panel shows how many are loaded out of the total for that list.
- Export a conversation from its menu when you have permission to export your own conversations. See Export a conversation.
Chat history is your own earlier work. Administrators use Audit when they need tenant-wide activity, including conversations from other users. See Review readiness, audit, and exports.