--- title: "Geneos Monitoring Report" description: "" url: "https://docs.itrsgroup.com/docs/geneos/7.11.0/administration/geneos_monitoring_report/index.html" --- **Important** Starting with Geneos version 7.7.0, the provided `monitoring_report_to_excel.py` script will not work with data from Gateways running versions prior to 7.7.0. Similarly, data from Geneos version 7.7.0 onwards cannot be used with the original monitoring report script that was supplied with Geneos versions earlier than 7.7.0. ## Introduction The Geneos Gateway provides a Geneos command that will generate a report of what the Gateway is monitoring. This report is presented in the form of a JSON or YAML document, with example scripts to [convert into an Excel workbook](#generating-excel-reports). The report includes the following: | Report information| Specific details | |-------|-------| | Details of all the probes that are running on the Gateway **Note** This includes any self-announcing probes that are connected to the Gateway at the time the report is run. | - Probe name - Probe hostname - Probe port - OS version of the server the probe resides upon | | Details of all the managed entities that are running on the Gateway | - Managed entity name - All attributes defined on the managed entity | | Details of each sampler that are running on the Gateway | - Sampler name - Sampler type - Sampler group - Sampler interval of the sampler - Active times that are applied to the sampler and affect when it samples - Dataviews created by the sampler - Any rules that are configured to run on the sampler or any descendant of the sampler (dataview, headline cells, or table cells). These rules are identified by name (rule group, rule name, and rule default name). Any sampler variables that are referenced in the rules will also be listed along with their values. - Any rules configured on a sampler or any descendant that have associated KBA links configured on the targets where those rules are applied. This shows a count of how many cells the KBA sets apply to, out of the total number of cells where each rule is evaluated. | | Details of all the rules that are available on the Gateway | - Rule name - Rule group - Rule default name - Rule logic - Rule actions - Rule active times | | Details of all the active times that are available on the Gateway | - Active Time name - Active Time description | | Details of all configured KBAs and URLs | - KBA set name - Label name - URL | ## Generating a Gateway report using a Geneos Command The report can be generated by running the Gateway command `/REPORTING:monitoring`. - The command is available on the Gateway directory component and the Gateway sampler that is provided as part of the Gateway self-monitoring. - In the Active Console and in the Entity Viewer, the command is called `Monitoring`. This can be found under the `Reporting` menu. ![Monitoring command](https://docscdn.itrsgroup.com/geneos/images/gw-monitoring-report-ac-7-11.png) This can be run in all the ways that a Geneos command is available: - From the Active Console - From the [Entity Viewer App in the Web Console](https://docs.itrsgroup.com/docs/itrs-analytics/app/entities/current/user-guide/commands/index.html) - Using a [Geneos Task](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/monitoring-and-alerts/geneos_commands_tr/index.html#task-commands) - Scheduled using [scheduled commands](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/monitoring-and-alerts/geneos_commands_tr/index.html#scheduled-commands) - Via the commands [REST API](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/monitoring-and-alerts/geneos_commands_tr/index.html#rest-service) The results of the command are stored in a file that can be found in the directory specified in the command arguments. The file can either be in JSON (`form monitoring_report_.json`) or YAML (`monitoring_report_.yaml`) format. **Note** While YAML is easier for humans to read, JSON is easier to manipulate to generate other data. JSON is ideal for maintaining data types and interacting with APIs or web services, while YAML is useful for creating configuration settings or adding comments in your data because of its user-friendly and readable format. ![Gateway Monitoring Report Arguments](https://docscdn.itrsgroup.com/geneos/images/gw-monitoring-report-arguments-7-11.png) ### Adding Custom Details Certain information that needs to be captured for the monitoring report can be obtained from: - Headlines and cells in sampler dataviews. - Values in the **resolved sampler setup** (the sampler configuration after variables have been substituted). The monitoring report command allows you to provide report configuration in YAML format. This configuration instructs the report to provide detailed information for certain samplers. **Note** - You can provide a list of details entries. - Each details entry has a filter that identifies which samplers it applies to, a `type` that controls the evaluation scope, and the information to extract (`columns`, `headlines`, and optionally `configExtracts`). - The same plugin can appear in more than one details entry. Use different `name` and `type` values when you need both sampler-wide setup and per-row or per-dataview data. ```yaml # Report YAML that defines the extra detailed information to obtain for the report report: # Indicates whether sampler rules in the report will contain only the rules that have been evaluated. showEvaluatedRulesOnly: false # Set to true to include per-rule KBA match counts in sampler rule data. showRuleKbaMatches: false # Array of details - each of which is extracted for each sampler that matches the filters and plugin details: # Name of the details which is used to wrap the data when placed in the report # Details will be placed alongside other information for each sampler that the filters apply to. - name: fkm # Plugin name of the samplers that these details will be collected for plugin: fkm # Filters to use to limit which samplers to collect this data for filters: # Type of the filter (hostname, probe, managedEntity, sampler, samplerType, dataview, attribute) - type: attribute # Name of the attribute to match against (required only for attribute filters) name: environment # List of regular expressions to match against the value provided by the type. If any one of the # regular expressions match then the details will be extracted regex: - prod - uat # Details extraction type. Controls how often columns, headlines, and configExtracts are evaluated: # row - once per dataview row # dataview - once per matching dataview # sampler - once per matching sampler (no dataview or row iteration) type: row # List of columns. Cells in these columns will be extracted from the dataview and placed in the report. # This data is only extracted when type is `row`. columns: - name - fileOwner - filename # List of headlines. These headline cells will be extracted from the dataview and placed in the report. headlines: - filesMonitored ``` #### Details type Each details entry must set `type` to one of: | `type` | Evaluation scope | Typical use | |--------|------------------|-------------| | `row` | Once per dataview row | Runtime table cells and setup values correlated to a row | | `dataview` | Once per matching dataview | Headlines and setup values that apply to a whole dataview | | `sampler` | Once per matching sampler | Setup values that apply to the whole sampler (for example, a script path or port list) | Unknown `type` values are rejected. #### Extracting dataview cells and headlines - `columns` — Table cell values; used when `type` is `row`. - `headlines` — Headline cell values; used when `type` is `row` or `dataview`. Extracted column and headline values are written to `additionalInfo` in the report output. #### Extracting sampler setup with configExtracts Use `configExtracts` when the information you need is in the resolved sampler configuration, but isn't available or reliable as a dataview cell or headline. Each rule is evaluated against that sampler's resolved setup XML. Extracted values are written to `additionalInfo` under the rule `name`. Existing reports that omit `configExtracts` are unchanged. Each `configExtracts` entry supports: | Field | Required | Description | |-------|----------|-------------| | `name` | Yes | Output field name in `additionalInfo`. | | `xpath` | One of `xpath` or `wpath` | Single-value extraction. A string, or an ordered list of XPath strings. The first expression that yields a usable value is used. | | `wpath` | One of `xpath` or `wpath` | Multi-value extraction. A single XPath that can return zero or more values. | | `join` | No | Delimiter used only with `wpath`. Defaults to `", "` when omitted. | | `mode` | No | `text` (default) or `exists`. `exists` is valid only with `xpath`. | Rules: - A rule must specify exactly one of `xpath` or `wpath`. Specifying both, or neither, is invalid and the report request is rejected. - `mode: text` (default) returns the string value of the matched node. Empty text isn't treated as a match, so the next `xpath` in the list is tried. If none of them match, the field is set to `not matched`. - `mode: exists` returns `"true"` if any listed `xpath` matches at least one node; otherwise, it returns `"false"`. Text content is ignored. Use this mode to detect optional empty elements. - `wpath` joins all matched values with `join` (or `", "`). If nothing matches, the field is empty. - Sensitive setup values (for example, passwords) are masked in the report output. XPath expressions are evaluated only within the resolved sampler setup for that sampler. Don't navigate through `` wrapper nodes; after variable resolution, those wrappers aren't a reliable anchor for extraction. ##### Placeholders XPath and `wpath` strings can include placeholders. Values are substituted and escaped for use in XPath string contexts. | Placeholder | Available when | Description | |-------------|----------------|-------------| | `{{samplerName}}` | `row`, `dataview`, `sampler` | Current sampler name. | | `{{dataviewName}}` | `row`, `dataview` | Current dataview name. Empty for `type: sampler`. | | `{{rowname}}` | `row` | Current row name. Empty for `type: dataview` or `type: sampler`. | | `{{parameter:}}` | All types | Value of the named managed entity attribute. | | `{{runtimeParameter:}}` | `row` | Value of the named row runtime parameter (for example, FKM `configIndex`). | If a placeholder doesn't apply for the current `type`, or the named value isn't set, it resolves to an empty string. **Note** For FKM, `{{runtimeParameter:configIndex}}` is converted from the probe's 0-based index to a 1-based XPath position, so expressions such as `plugin/fkm/files/file[{{runtimeParameter:configIndex}}]/...` select the correct configured `` entry. ##### Limitations - Row-to-setup linking without `configIndex` — Reliable correlation of a dataview row to a specific entry in the resolved sampler setup depends on a row runtime parameter such as FKM `configIndex`. Plugins that don't expose `configIndex` (or an equivalent runtime parameter) can't be reported in a way that reliably links each dataview row to its matching resolved setup entry. You can still extract sampler-wide or dataview-wide setup with `type: sampler` or `type: dataview`, or attempt name-based matching with `{{rowname}}` where the configured value and the runtime row name are identical. - FTM filename wildcards — FTM paths that use wildcards or tokens in the configured filename (for example, ``) can't be matched reliably to runtime rows. The resolved setup keeps the wildcard form, but the dataview row name shows the expanded value. As a result, `xpath` expressions that compare against `{{rowname}}` don't extract those entries. - Gateway SQL tables and dataviews — Gateway SQL source tables and output views are configured separately. `configExtracts` can report table definitions at sampler scope and per-view SQL at dataview scope, but you can't reliably link source tables to their output dataviews in the report. ##### Example: TCP-Links sampler and runtime rows Use a `type: sampler` entry for setup that applies once per sampler, and a separate `type: row` entry for runtime table cells: ```yaml - plugin: tcp-links name: tcp-links-setup type: sampler filters: - type: managedEntity regex: - ME - type: sampler regex: - TCPLinks configExtracts: - name: localPorts wpath: plugin/tcp-links/localPorts/port join: ", " - name: remotePorts wpath: plugin/tcp-links/remotePorts/port join: " / " - plugin: tcp-links name: tcp-links-connections type: row filters: - type: managedEntity regex: - ME - type: sampler regex: - TCPLinks columns: - name - localAddress - remoteAddress - state - type ``` ##### Example: Gateway SQL tables and per-view SQL Use two details entries for the same plugin: one sampler-scoped and one dataview-scoped. For the restriction on linking source tables to output dataviews, see [Limitations](#limitations). ```yaml - plugin: gateway-sql name: gwsql-tables type: sampler filters: - type: sampler regex: [GwSql] configExtracts: - name: sourceTables wpath: plugin/Gateway-sql/tables/dataview/tableName join: ", " - plugin: gateway-sql name: gwsql-views type: dataview filters: - type: sampler regex: [GwSql] configExtracts: - name: viewSql xpath: plugin/Gateway-sql/views/view[name="{{dataviewName}}"]/sql ``` ##### Example: State Tracker path correlated to a row ```yaml - plugin: statetracker name: statetracker-job-rows type: row filters: - type: managedEntity regex: - ME - type: sampler regex: - JobStateTracker columns: - Job - previousState - state - time configExtracts: - name: configuredTrackerTemplate xpath: plugin/stateTracker/trackerGroup/trackers/tracker[name="SharedJobLog"]/name - name: configuredMonitorPath xpath: plugin/stateTracker/trackerGroup/trackers/tracker[name="SharedJobLog"]/filename - name: configuredTransitionStateNames wpath: plugin/stateTracker/trackerGroup/trackers/tracker[name="SharedJobLog"]/transitionStates/*/@name join: ", " ``` The XPath strings in these examples are illustrative. They must match the resolved sampler setup shape for your plugin version and estate. #### Extra FKM Columns For FKM, it is important to be aware of the configuration information associated with each file being monitored. This includes the table cells and headlines extracted from the FKM dataview. The **Details** sections for the FKM plugin will also provide four additional types of information in a dedicated `config` object: - **fkmActiveTime** — the active time associated with the file (if defined) - **ignoreKeys** — a list of ignore keys for this file - **failKeys** — a list of fail keys for this file - **warningKeys** — a list of warning keys for this file These keys (Critical, Warning, and Ignore) are automatically extracted if the plugin type is `fkm`. That built-in FKM `config` output is unchanged. You can also add `configExtracts` to an FKM details entry to extract additional setup values into `additionalInfo`. Use distinct field names so they don't collide with the built-in `config` keys. **Note** Built-in FKM `config` extraction requires Netprobe version 7.1.0 or later. ## Generating a Gateway report using a REST request Enabling a [REST Service](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/monitoring-and-alerts/geneos_commands_tr/index.html#Heading-rest-service) in the Gateway allows you to directly request a monitoring report. The [YAML report configuration](#adding-custom-details) can be passed as the POST body of the REST request. A report in JSON format will be returned from the request. To run the report on an insecure Gateway running on the same machine with no authentication, run the following: ``` curl -X POST -H 'Content-Type: application/json' http://127.0.0.1:7039/rest/reporting/monitoring ``` To connect to a Gateway with authentication, refer to [user authentication](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/monitoring-and-alerts/geneos_commands_tr/index.html#user-authentication) and [authorizing an SSO user](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/monitoring-and-alerts/geneos_commands_tr/index.html#authorise-sso-user). ## Generating Excel reports The JSON reports can be converted into an Excel workbook through a `python` script. This script will convert the JSON reports from multiple Gateways into a single workbook with multiple sheets. The script (`monitoring_report_to_excel.py`) is included in the Gateway package at `/templates/monitoring_report_to_excel.py`. It's provided as an example and works with reports generated using the default parameters. Values from `additionalInfo` (including extracted columns, headlines, and configExtract values) are written as additional columns on the details worksheets. **Note** Install the `xlsxwriter` Python package before you run the script. You can download a [sample JSON report and Excel workbook](/geneos/downloads/example_report.tar.gz) generated by `monitoring_report_to_excel.py`.