--- title: "Operating Environment" description: "" url: "https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/gateway_operating_environment/index.html" --- ## Overview The Gateway **Operating environment** top-level section contains settings that affect the Gateway as a whole, and do not belong to any other section. ## Operation Although there are no mandatory settings in this section, it is essential that the Gateway name is set either with [operatingEnvironment > gatewayName](#operatingenvironment--gatewayname) or using the `-gateway-name` or `-demo` command line options. This name is displayed to all users that connect to the Gateway, and is also used in name lookup and database logging. This name must be unique for each Gateway in order to use features such as Self-announcing Netprobes, Gateway Sharing and Gateway Hub connectivity. The Gateway listen ports can also be set in the operating environment ([operatingEnvironment > listenPorts](#operatingenvironment--listenports)). The listen ports are used by components connecting to gateway such as Active Console or Webslinger to request monitoring data for display to users. **Note** This does not include Netprobe connections, as configuration for these are contained within [Probes](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/gateway_probes/index.html). If not configured, the gateway listen port defaults to port `7039` for insecure channel and `7038` for secure channel. ### Data quality options Settings in Operating environment control the data quality algorithm that Gateways use to maintain a consistent level of service under excessive load. This algorithm runs throughout the lifetime of a Gateway (unless [operatingEnvironment > dataQueues > disableChecks](#operatingenvironment--dataqueues--disablechecks) is set) and operates as follows: - A Gateway monitors dataview updates to determine if the oldest pending update becomes stale (as controlled by the [maxDataAgeMs](#operatingenvironment--dataqueues--suspendonbreach--samplerprobe--maxdataagems) setting). If this occurs, the connection of a probe or a sampler is dropped to reduce the Gateway's load and restore timely data processing. - The Gateway determines which connection to drop based on CPU usage when processing the incoming data over the previous minute. The connection with the highest load is then suspended for a period (see [connectionSuspensionDuration](#operatingenvironment--dataqueues--suspendonbreach--samplerprobe--connectionsuspensionduration)) before the Gateway reconnects. - Once a connection has been suspended, no further suspensions will occur until a grace period (see [suspendGracePeriod](#operatingenvironment--dataqueues--suspendonbreach--samplerprobe--suspendgraceperiod)) has elapsed, allowing time to evaluate the effect of the suspension on the quality of the data. - Setup changes represent a special case where the data age metrics may spike in the Gateway. During setup application, no incoming data from Netprobes is processed, leading to a backlog of updates to be applied. To avoid unnecessary suspensions, the algorithm is disabled during setup changes and for [setupGracePeriod](#operatingenvironment--dataqueues--suspendonbreach--samplerprobe--setupgraceperiod) seconds afterwards. Additionally, commands can also be run to control [probe suspension](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/monitoring-and-alerts/geneos_commands_tr/index.html#netprobe-commands) and [sampler suspension](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/monitoring-and-alerts/geneos_commands_tr/index.html#sampler-commands). For more information regarding data quality, see [Data Quality User Guide](https://docs.itrsgroup.com/docs/geneos/7.11.0/administration/data_quality_guide/index.html). ### Memory protection Settings for memory protection are found in the [Data Quality tab](#data-quality-tab). When data quality is disabled, or in extreme situations when it cannot suspend sufficient probes to prevent the gateway becoming overloaded, the gateway throttles the reading of TCP data to prevent the backlogged data-queues from unbounded growth. This is necessary but less preferable to a managed data-quality suspension because if it continues without recovery, netprobe connections either flow-control or timeout and netprobes are dropped at random. There are two threshold levels: - Low-priority threshold ([operatingEnvironment > dataQueues > memoryProtection > lowPriorityThresholdMB](#operatingenvironment--dataqueues--memoryprotection--lowprioritythresholdmb)). - High-priority threshold ([operatingEnvironment > dataQueues > memoryProtection > highPriorityThresholdMB](#operatingenvironment--dataqueues--memoryprotection--highprioritythresholdmb)). When the low-priority threshold is breached, the gateway throttles reads from all importing (netprobe) connections but remain responsive to downstream components, such as Active Console. In the unlikely event that this fails to prevent memory growth and the high-priority threshold is breached, the gateway throttles reads from all connections and become unresponsive until it recovers. The default for the low-priority threshold is 250 MB. This is calculated to be enough to buffer 77 seconds (see [operatingEnvironment > dataQueues > maxDataAgeMs](#operatingenvironment--dataqueues--maxdataagems)) worth of data on a high bandwidth gateway (approximately 30,000 cell updates per second). This is in order to give the data-quality algorithm time to step in and save the situation before the threshold is reached. **Note** These thresholds govern memory usage by unprocessed EMF messages only. The gateway memory footprint as a whole is typically far more influenced by other factors and could potentially exceed these thresholds without issue. It is unusual for unprocessed EMF messages to account for more than a few megabytes in a normally operating gateway. ### Conflation Settings for conflation are found in the [Data Quality tab](#data-quality-tab). Conflation is an optional and less drastic method of coping with an overloaded gateway than a data quality suspension. When the data queues (containing incoming sampler updates from netprobes) become backlogged due to the gateway being unable to process them as fast as they arrive, conflation allows the gateway to discard out-of-date cell updates and only process and publish the latest cell values. As this could potentially result in the gateway discarding important updates, or missing short-lived events, conflation is disabled by default and should only be used with care. #### Conflation examples ##### Rapid cell updates When a Netprobe has published several updates to the same cell before the gateway has processed the first update: - Update cell from *1* to *2* - Update cell from *2* to *3* - Update cell from *3* to *4* - Update cell from *4* to *5* With conflation active, gateway only publishes the latest value: - Update cell from *1* to *5* ##### Updates to a recently created row When a netprobe updates values in a recently created row before the gateway has processed the create: - Create row *newRow* with three cells: *100,200,300*. - Update first cell in *newRow* from *100* to *111*. - Update second cell in *newRow* from *200* to *222*. With conflation active, gateway adds the row to the dataview with the latest values: - Create row *newRow* with three cells: *111,222,300*. ##### Updates to a row that is then removed When a netprobe updates values in a row and then removes the row before the gateway has processed the updates: - Update first cell in *row1* from *100* to *111*. - Update second cell in *row1* from *200* to *222*. - Remove row *row1*. With conflation active, gateway discards the updates and only process the row-removal: - Remove row *row1*. ##### Short lived rows When a netprobe creates a row and then removes it again before the gateway has processed the create: - Create row *newRow* with four cells: *100,200,300,400*. - Update first cell in *newRow* from *100* to *111*. - Remove row *newRow*. With conflation active, gateway conflates away the update as normal but does not conflate away the entire row: - Create row *newRow* with four cells: *111,200,300,400*. - Remove row *newRow*. #### Potential Issues Conflation can prevent a gateway from becoming overloaded, and ensure that published values are always up-to-date, but there are a number of potential issues which you should be aware of. ##### Lost Spikes A dataview cell that updates from *32%* to *34%* to *33%* is unlikely to cause issues by having the intermediate update conflated away, but one that updates from *32%* to *99%* to *33%* may miss an important spike. Similarly, a cell that goes from *OK* to *ERROR* to *OK* again, could cause an alert to be missed if conflation is enabled. This might also affect compute-engine rules that use statistical functions such as maximum or minimum. ##### Rate Function Rate function triggers off the time an update is processed, rather than the time it is generated, and therefore its general performance is likely be improved by conflation. **Note** Spikes in the rate-of-change in a cell may be conflated away. ##### Database Logging Cell updates that are discarded by conflation will not be logged. ##### sampleTime and logNetprobeSampleTimeForDataItems If logNetprobeSampleTimeForDataItems is configured, cell updates may be logged with sample-times later than they were produced with. This is because the sample-time is published by the netprobe along with the sample-data and will be conflated to the latest value. E.g. A series of updates produced at twenty second intervals by the netprobe: - Update *cell1* @ *09:25:02* - Update *cell2* @ *09:25:22* - Update *cell3* @ *09:25:42* Might be conflated into a single update with the latest sample-time: - Update *cell1*, *cell2*, and cell3 @ *09:25:42* The updates to *cell1* and *cell2* may be logged with this later sample-time. Similarly, rules that reference the sample-time may only see the later value. ## Configuration ### Basic tab These settings are found under the **Basic** tab. #### operatingEnvironment Gateway-wide options are configured here. #### operatingEnvironment > gatewayName A short name identifying the Gateway. A Gateway name is required for all Gateways. However, you may use either the **operatingEnvironment > gatewayName** field or the `-gateway-name` command line option when specifying a name. The command line option is often useful in orchestrated environments, such as Kubernetes. For more information see [Command line options in Gateway Installation Guide](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/gateway_installation_guide/index.html#command-line-options). When using database logging functionality, this name is also logged to the database, and is used to identify records for the Gateway. Mandatory: No If the `<> FATAL: Gateway Mandatory 'gatewayName' has not been specified. [/gateway/operatingEnvironment]` error message appears, this means that the Gateway name has not been set using the command line option, and that the setting of the configuration file with the highest priority has no value. To resolve this: 1. Look for any enabled **Gateway name** settings in the Gateway Setup Editor, and check the main and include files. 2. Click **Includes > Priority** field, and increase the priority of the configuration file (main or include files) that has a **Gateway name** setting with a value. 3. Click **Save current document** ![](https://docscdn.itrsgroup.com/geneos/images/active-console/userGuide-images/ac2_toolbar_save.png "icon") to apply the changes. #### operatingEnvironment > licensingGroup Group that the Gateway requests licences from on the Licence Daemon. Mandatory: No #### operatingEnvironment > listenPorts The gateway listen ports for incoming connections. - If [operatingEnvironment > listenPorts > secure](#operatingenvironment--listenports--secure) is set and [operatingEnvironment > listenPorts > insecure](#operatingenvironment--listenports--insecure) is not set, then the Gateway only listens on a secure port. - If [operatingEnvironment > listenPorts > secure](#operatingenvironment--listenports--secure) is not set then the gateway only listens on an insecure port. - If both [operatingEnvironment > listenPorts > secure](#operatingenvironment--listenports--secure) and [operatingEnvironment > listenPorts > insecure](#operatingenvironment--listenports--insecure) are set, the gateway listens on two ports. See [Secure Communications](https://docs.itrsgroup.com/docs/geneos/7.11.0/security/secure-communications/ssl_ug/index.html) for more details. The listen port can also be specified as a command-line argument to gateway. If this is done, then the command-line value is used for the lifetime of the gateway process - it cannot be overridden or altered by editing the gateway setup file. An example of using this command-line option is shown below: ``` gateway2 -port <12345> ``` Mandatory: No Default: The gateway listens insecurely on port 7039 #### operatingEnvironment > listenPorts > secure This specifies that the gateway should listen securely. In order to listen securely, a SSL certificate needs to be provided using either the *-ssl-certificate* or *-ssl-certificate-key* command line option. By default if configured to be secure, the gateway will listen on port 7038. This can be overridden by using the child setting [operatingEnvironment > listenPorts > secure > listenPort](#operatingenvironment--listenports--secure--listenport). Mandatory: No Default: The gateway will not listen securely on any port #### operatingEnvironment > listenPorts > secure > listenPort This value overrides the default secure listenPort. Specify an integer in the range 1-65535. Mandatory: No Default: 7038 #### operatingEnvironment > listenPorts > insecure This specifies that the Gateway should listen insecurely. By default if configured to allow insecure connections, the gateway will listen on port 7039. This can be overridden by using the child setting [operatingEnvironment > listenPorts > insecure > listenPort](#operatingenvironment--listenports--insecure--listenport). Mandatory: No Default: See [operatingEnvironment > listenPorts](#operatingenvironment--listenports) #### operatingEnvironment > listenPorts > insecure > listenPort This value overrides the default insecure listenPort. Specify an integer in the range 1-65535. Mandatory: No Default: 7039 #### operatingEnvironment > selfMonitoring Self-monitoring is active by default. Use this option to configure additional settings or to disable self-monitoring. ##### Gateway self-monitoring Gateway self-monitoring creates a Managed Entity named ` Info` with dataviews in the sampler groups listed below. Some of these dataviews are optional and are created only if the corresponding section is present in the Gateway setup: | Sampler group | Dataview | Plugin | |-------|-------|-------| | Database Logging (optional) | Cache | [Database logging](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#database-logging) | | Database Logging (optional) | Items | [Database logging](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#database-logging) | | Database Logging (optional) | Summary | [Database logging](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#database-logging) | | Gateway Hub (optional) | Nodes | [Gateway Hub data](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#gateway-hub-data) | | Gateway Hub (optional) | Summary | [Gateway Hub data](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#gateway-hub-data) | | Gateway Info | Client Connections | [Client connection data](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#client-connection-data) | | Gateway Info | Entities | [Managed entities data](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#managed-entities-data) | | Gateway Info | Gateway | [Gateway data](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#gateway-data) | | Gateway Info | Netprobes | [Probe data](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#probe-data) | | Gateway Info | Performance | [Gateway performance](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#performance) | | Gateway Info | Scheduled Commands (optional) | [Scheduled command data](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#scheduled-command-data) | | Gateway Info | Setup Includes | [Includes data](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#includes-data) | | Gateway Info | Severity Count | [Severity count](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#severity-count) | | Gateway Info | Snoozed Items | [Snooze data](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#snooze-data) | | Gateway Info | User Assignments | [User assignment data](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#user-assignment-data) | | Import Export (optional) | Exported Data | [Exported data](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#exported-data) | | Import Export (optional) | Imported Data | [Imported data](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#imported-data) | | ITRS Analytics (optional) | Detail | [ITRS Analytics Connection](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#itrs-analytics-connection) | | ITRS Analytics (optional) | Summary | [ITRS Analytics Connection](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#itrs-analytics-connection) | **Note** Optional sampler groups are only created if the corresponding sections are present in the Gateway setup. ##### Netprobe self-monitoring Netprobe self-monitoring creates a Managed Entity named ` Info` for each Netprobe that has dynamic mappings configured, or that has a sampler with the **Publish to ITRS Analytics** pilot feature enabled. The following dataviews are optional and are created only if the corresponding setup items are present.
Sampler group Dataview Plugin
Netprobe Info Entities Dynamic Entities Health
Netprobe Info InvalidMetrics Dynamic Entities Health
Netprobe Info Streams Dynamic Entities Health
Netprobe Info InvalidStreams Dynamic Entities Health
#### operatingEnvironment > selfMonitoring Self-monitoring is active by default. Use this option to configure additional settings or to disable self-monitoring. #### operatingEnvironment > selfMonitoring > attribute Add attributes to the ` Info` entity. This allows you to assign the self-monitoring entity to the correct location in your custom hierarchy. Mandatory: No #### operatingEnvironment > selfMonitoring > gatewayInfo This setting enables or disables Gateway self-monitoring. Mandatory: No Default: True #### operatingEnvironment > selfMonitoring > netprobeInfo This setting enables or disables Netprobe self-monitoring. Mandatory: No Default: True #### operatingEnvironment > selfMonitoring > gatewaySamplerOverride This setting overrides the samplers that are created in the self-monitoring Managed Entity on the Gateway. This setting also allows you to: - Disable a sampler that is created by default - Override the setup of a sampler that is created by default by referencing a sampler that you have defined. - Add an extra Gateway sampler that you have defined in the Samplers section of the setup. #### operatingEnvironment > selfMonitoring > gatewaySamplerOverride > what This setting allows you to choose the sampler you want to override from the dropdown list. To include an additional sampler, choose the **Custom** option. Mandatory: No #### operatingEnvironment > selfMonitoring > gatewaySamplerOverride > how This setting allows you to disable a sampler or to override its setup by referencing a sampler that you have defined. If you choose to include an additional sampler by configuring `gatewaySamplerOverride > what` as **Custom**, you must refer to any sampler that uses a Gateway plugin. However, if you choose to override a setup of a sampler that is created by default, you must reference a sampler that uses the same Gateway plugin as the sampler you are overriding. Mandatory: No #### operatingEnvironment > selfMonitoring > netprobeOverride This setting allows you to override the samplers that are created in the self-monitoring Managed Entity on the Netprobes that connect to the Gateway. This setting also allows you to: - Disable a sampler that is created by default. - Override the setup of a sampler that is created by default by referencing a sampler that you have defined. #### operatingEnvironment > selfMonitoring > netprobeOverride > what This setting allows you to choose the sampler you want to override from the dropdown list. Mandatory: No #### operatingEnvironment > selfMonitoring > netprobeOverride > how This setting allows you to disable a sampler or to override its setup by referencing a sampler that you have defined. If you choose to override the setup of a sampler that is created by default, you must reference a sampler that uses the same plugin as the sampler you are overriding. Mandatory: No #### operatingEnvironment > var List of user environment variable definitions. See [User Variables and Environments](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/gateway-administration/gateway_user_variables_and_environments/index.html) for details on how to configure environment variables. ### Advanced tab These settings are found under the **Advanced** tab. #### operatingEnvironment > description An optional description of the gateway. Mandatory: No #### operatingEnvironment > maxLogFileSizeMb Maximum size in Megabytes of the Gateway log file before it rolls that log file over. Valid values are 1-2047 inclusive for 32-bit Gateways. Mandatory: No Default: `10` #### operatingEnvironment > logArchiveScript The name of a batch file or shell script that should be executed when the log file is rolled over. Using **operatingEnvironment > logArchiveScript** overrides **LOG\_ARCHIVE\_SCRIPT** (if set). Mandatory: No #### operatingEnvironment > Log time format The time format used to record timestamps in the log file. Choose from: - (Default) Iso-8601: `2019-09-25 09:18:28.871-0400` - Iso-8601-utc: `2019-09-25 13:18:28.871Z` - Legacy: `` Mandatory: No **Note** The log time format can also be set using an environment variable. for more information, see [Specify the log file time formatting in Gateway Log File](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/gateway-administration/gateway_log_file/index.html#specify-the-log-file-time-formatting) . #### operatingEnvironment > timezone This sets the `TZ` environment variable which determines the time zone the Gateway runs in. This allows a Gateway in one country to monitor Netprobes in another country whilst keeping the time zones the same. The time zone can be specified in any format compatible with the unix `TZ` environment variable: - (Recommended) IANA time zone name, such as `America/New_York` or `Europe/London` - (Not recommended) POSIX time zone specification, such as `EST+5EDT,M3.2.0/2,M11.1.0/2` For more information, see the POSIX manual page for the `tzset` command. The Gateway attempts to validate your choice of time zone against the system time zones, and issues a warning if the time zone cannot be verified. Mandatory: No Default: Gateway uses the current value of the `TZ` environment variable. If `TZ` is not set, Gateway uses the local time of its host machine. #### operatingEnvironment > timezoneabbreviation A list of time zone abbreviations and their default time zone regions. This is used to override the time zone abbreviations when parsing dates in rules and when parsing information from dataviews for standard formatting. #### operatingEnvironment > internalQueueSizeLimit Controls the maximum length of the internal update queue. Updates to data-items (e.g. a severity change as the result of running a rule) are placed in the queue temporarily between data updates. The default maximum limit should be adequate for normal gateway operation. If a pair (or more) of rules are configured such that an update caused by rule A makes rule B fire and update, then this can cause the internal queue to fill faster than it is processed. If the queue is completely filled an error message is logged, and gateway performance is likely to be affected. The solution is to write rules A and B to be more selective, so that they do not fire each other. Certain compute engine rules (typically involving wildcarded paths) can also fill the processing queue during gateway startup. The queue limit can be increased to prevent warning messages if required, however this should only be done if it is known that this situation is a "one off". Mandatory: No Default: `4000` #### operatingEnvironment > numRuleEvaluationThreads Specifies the maximum number of rule evaluation threads the Gateway can run. These threads are used to execute rules on data changes, and can be enabled if rule execution is becoming a bottleneck on a busy Gateway. It is recommended that this is not set too high as doubling the number of threads does not double throughput. It should not be set higher than the number of CPU cores available. To set the number of rule evaluation threads used, the Gateway will determine the number of available processors using the `taskset` command on Linux systems or the `psrset` command on Solaris systems, and evaluate the `numRuleEvaluationThreads` variable. The lower value will be used to specify the number of threads used by the Gateway to execute rules. The number of rule evaluation threads used is recorded in the Gateway log. If the available number of processors is changed while the Gateway is running then the number of threads to use is re-evaluated at the next setup change. For more detailed information about the optimum value to use, see [Gateway Performance Tuning](https://docs.itrsgroup.com/docs/geneos/7.11.0/administration/performance_tuning_guide/index.html). A hard limit can also be placed on the number of rule threads by setting the environment variable `MAX_RULE_THREADS` to a positive integer. This will override the value specified in the Gateway Setup Editor. Mandatory: No Default: `2` #### operatingEnvironment > customDataviewMaxRows Sets the maximum number of rows for dynamically sized dataviews in toolkit plugins. A single Gateway-wide default limits the size of these dataviews, helping protect the Gateway from issues caused by large dataviews, particularly when managing multiple toolkit dataviews. This setting applies to the following plugins and overrides any row limits configured at the plugin level: - Netprobe plugins: - API - Dynamic Entities Health - IBMi Message - IBMi Queue - IBMi JDBC - IBMi Job - IBMi Subsystem - IX-MA - REST-API - Prometheus Query - Rest Extractor - SQL Toolkit - Toolkit - Top - Trapmon - Gateway plugin: - Gateway-sql Mandatory: No Default: `200` #### operatingEnvironment > historyFiles The maximum number of history files that the gateway is allowed to create when receiving set-up changes. Valid values are 0-9999 inclusive. To suppress history file creation altogether, set this to zero. Mandatory: No Default: `10` #### operatingEnvironment > dataDirectory Allows you to specify where temporary files which the gateway may produce while running should be stored. If not set, files are stored in the current working directory. If the directory specified already contains any of these temporary files, they are over-written. The data directory must have read, write, and execute permissions as it needs to be able to read, write and search within it. Mandatory: No Default: Current working directory #### operatingEnvironment > duplicateRowAlerts When duplicate rows in a single dataview are detected, gateway alerts the user of this fact as it indicates a configuration error. These alerts can be adjusted using this setting. Mandatory: No Default: `TICKER_AND_STATUS`
Value Effect
NONE No alerts regarding duplicate rows are produced.
STATUS The dataview samplingStatus headline is updated with an error message warning about duplicate rows.
TICKER_AND_STATUS The samplerStatus headline is updated as above, and additionally an event ticker event is created regarding the duplicate rows.
#### operatingEnvironment > excludedConfiguration A list of XPaths that specify which configuration options are excluded from the final setup used by the Gateway. If options are excluded, validation errors for those options are reported in the Gateway Setup Editor. Use this to prevent arbitrary script deployment, for example from the Toolkit sampler **Advanced** tab. To exclude all script options for all Toolkit samplers, add the XPath `//samplers//sampler/plugin/toolkit/script` to this list. Mandatory: No #### operatingEnvironment > externalPasswords Retrieve passwords from an external provider. This allows passwords to be encrypted and stored by another application. The Gateway supports the following external providers: - CyberArk Central CP - CyberArk Local CP - CyberArk Multiple Central CP - HashiCorp Vault #### operatingEnvironment > externalPasswords > provider > cyberArk The CyberArk Central Credential Provider (Central CP) allows the Gateway to fetch passwords from a central password vault using an API. | Field | Description | |-------|-------------| | Server name | URL of the server. This should take the form `https://:`. If no port is specified, the default port `443` will be used. Mandatory: Yes | | Application id | Name used to identify the CyberArk permissions for the Gateway. This determines which passwords can be requested and must be configured in the CyberArk web interface. Mandatory: Yes | | Client certificate | Path to the `.pem` file containing the certificates needed to connect to CyberArk securely. Mandatory: Yes | | Client key | Path to the file containing the client key. This is not required if the client key has been provided as part of the `.pem` file. Mandatory: No | | Client key passphrase | Passphrase used to decrypt an encrypted client key. Mandatory: No | #### operatingEnvironment > externalPasswords > provider > cyberArkLocal The CyberArk Local Credential Provider allows the Gateway and the Netprobe to maintain a secure, actively refreshed password cache locally. This uses a local agent to access your central password vault. | Field | Description | |-------|-------------| | Application id | Name used to identify the CyberArk permissions for the Gateway. This determines which passwords can be requested and must be configured in the CyberArk web interface. Default: `ITRS-Geneos` Mandatory: No | | SDK path | Path to the application password SDK installed on the Gateway host. The path must be fully specified and cannot embed any references to environment variables. If no path is specified, the default CyberArk installation directory `/opt/CARKaim/sdk/clipasswordsdk` will be used. Mandatory: No | #### operatingEnvironment > externalPasswords > provider > cyberArkMultiple This allows the Gateway to connect to multiple CyberArk servers. To specify which of the servers to use for each external password, you must add an extra key called `GENEOS_PASSWORD_SERVER`, wherein its value should be set to the name of the CyberArk server to use. The external password will not be sent if this key is missing. When using CyberArk (Central CP), this key is treated like any other key and sent to the CyberArk server. | Field | Description | |-------|-------------| | Name | User-defined identifier for the server configuration. This value is used to reference the server when requesting passwords. Mandatory: Yes | | Server name | URL of the server. This should take the form `https://:`. If no port is specified, the default port `443` will be used. Mandatory: Yes | | Application id | Name used to identify the CyberArk permissions for the Gateway. This determines which passwords can be requested and must be configured in the CyberArk web interface. Mandatory: Yes | | Client certificate | Path to the `.pem` file containing the certificates needed to connect to CyberArk securely. Mandatory: Yes | | Client key | Path to the file containing the client key. This is not required if the client key has been provided as part of the `.pem` file. Mandatory: No | | Client key passphrase | Passphrase used to decrypt an encrypted client key. Mandatory: No | #### operatingEnvironment > externalPasswords > provider > hashiCorpVault / hashiCorpVaultAgent The HashiCorp Vault integration allows the Gateway to fetch passwords from the Vault when it validates its setup. There are two HashiCorp providers, each varying in how they manage namespaces: - `hashCorpVault` — this setup assumes that the user has configured the namespace in the URL of the secret in the **extPwd** section. In this case, the namespace is only used for the authentication request. - `hashCorpVaultAgent` — this setup adds the namespaces to HTTP requests by including the X-Vault-Namespace header. To make authenticated requests for secrets held in the Vault, the Gateway needs to obtain a Vault token, which it can obtain in any of the following ways: - AppRole authentication — where the role ID and secret ID are configured in the Gateway setup. - TLS certificate authentication — where a certificate is configured as trusted by a Vault administrator and the certificate and private key are specified in the Gateway setup. - Custom authentication — where the Gateway runs an external script which obtains the Vault token. | Field | Description | |-------|-------------| | Server name | URL of the Vault server. This should take the form `https://:`. If no port is specified, the default port `443` will be used. Mandatory: Yes | | Server CA certificate | Path to a PEM file containing a Certificate Authority certificate (or certificate chain) which can be used to validate the certificate used by the Vault server. This is not needed if the Vault server certificate is already trusted by the Gateway host. Mandatory: Yes | | Verify server certificate | Enables or disables server certificate verification. This setting can be set to false if the Vault server certificate is not trusted by the Gateway host and the CA certificate is not available. It should not be used in a production environment. Default: `True` | | Vault namespace | Namespace path of the HashiCorp Vault. Mandatory: No | | Authentication mode | Selects the authentication method to use when obtaining a Vault token. Mandatory: No | | Authentication mode > AppRole > Role ID | This is provided by a Vault administrator, and it is used by the Gateway to authenticate to the Vault. Mandatory: Yes | | Authentication mode > AppRole > Secret ID | This is provided by a Vault administrator, and it is used by the Gateway to authenticate to the Vault. The secret ID should be stored using AES encryption, so that the plaintext value does not appear in the Gateway setup file. Mandatory: Yes | | Authentication mode > TLS certificate > Client certificate | Path to a certificate in PEM format which has been configured as a trusted certificate by the Vault administrator. Mandatory: Yes | | Authentication mode > TLS certificate > Client key | Path to the file containing the client key. This is not required if the client key has been included in the client certificate PEM file. Mandatory: No | | Authentication mode > TLS certificate > Client key passphrase | Passphrase used to decrypt an encrypted client key. Mandatory: No | | Authentication mode > TLS certificate > Role name | Name of the Vault role configured for TLS certificate authentication. This role must be configured in Vault by a Vault administrator and defines which certificates are allowed and what policies they receive. Mandatory: No | | Authentication mode > Custom > Executable | Path to the script or other executable which the Gateway will run to obtain a token. This script should simply write the JSON object returned by the Vault API to its standard output; the Gateway will parse the token value from the JSON object. Mandatory: Yes | | Authentication mode > Custom > Environment variables | These variables will be available to the script executed by the Gateway. Mandatory: No | | Authentication mode > Custom > Secure environment variables | Environment variables containing sensitive values that are encrypted using AES encryption. These encrypted variables will be available to the script executed by the Gateway. The script can decode these variables using the method described in [Decode encrypted environment variables](#decode-encrypted-environment-variables). Mandatory: No | #### operatingEnvironment > externalPasswords > validationTimeout Determines how long the Gateway should wait after setup validation for responses to outstanding password requests. Mandatory: No Default: `30` #### operatingEnvironment > insecurePasswordLevel In a number of places throughout the Gateway configuration, passwords have to be specified. Examples of this might be plugins that require logins to systems to retrieve the data they need, the gateway's connection to the database, or the configuration of users. In most of these places it is possible to enter the password in a number of different formats (depending on context), from a cleartext format, to more secure formats such as AES (for two way), and crypt (for one way). While it may be useful to use a cleartext format in a UAT or testing environment, you may prefer to ensure that a secure format is used when in a production environment. This setting helps locate these, by causing each insecure password to generate an issue at the specified level. This is shown when validating or saving the setup. **Note** We have deemed standard encoded passwords (std) to be insecure since they are encoded rather than encrypted, and these will be flagged in the same way as cleartext passwords. The setting has the following effects: - `None` — no checks are performed on the security of the passwords and no issues are reported. - `Critical` — the setup cannot be saved and the Gateway cannot be started with any insecure passwords present. With `Warning` or `Error` set, the ability to save the setup with insecure passwords present depends on if the `-max-severity` command line parameter is set. See the [Gateway Installation Guide](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/gateway_installation_guide/index.html). The [Gateway Data Plug-In](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html) reports the level of this setting. Mandatory: No Default: `Warning` #### operatingEnvironment > allowPasswordsInToolkitEnvironmentVariables **Warning** This feature is not secure and should be used with caution as it allows plaintext passwords to be available in the toolkit process environment. It is not recommended for general use but is available to allow legacy scripts to function without modification during migration. When enabled, the Gateway decodes AES and external passwords when passing them to toolkit scripts through environment variables. This allows toolkits to access passwords without having to decrypt them. This will not work with [CyberArk local passwords](https://docs.itrsgroup.com/docs/geneos/7.11.0/security/access-controls/gateway_secure_passwords/index.html#cyberark-local-credential-provider). In this case, the Gateway passes the key to the script, which the script can use to request the password itself. Mandatory: No Default: `false` #### operatingEnvironment > allowComputeEngine Specifies whether the Gateway compute engine feature is available to add additional data to existing dataviews. It is allowed by default, but administrators and users can use this setting to disallow compute engine features. See [Compute Engine](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/monitoring-and-alerts/geneos_rulesactionsalerts_tr/index.html#compute-engine). Mandatory: No Default: `true` #### operatingEnvironment > writeStatsToFile The "write stats to file" section contains settings controlling how load monitoring statistics are written out from the gateway. These statistics can then be read by the [Load Monitoring plugin](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#gateway-load). Also see the [Gateway Performance Tuning](https://docs.itrsgroup.com/docs/geneos/7.11.0/administration/performance_tuning_guide/index.html) for more information. Mandatory: No Default: No statistics written. #### operatingEnvironment > writeStatsToFile > filename The file to write statistics to. Mandatory: Yes #### operatingEnvironment > writeStatsToFile > enablePeriodicWrite Specifies whether to write data to file periodically. If `false`, statistics are only written when the "write statistics" command is executed. Mandatory: No Default: `true` #### operatingEnvironment > writeStatsToFile > writeInterval Interval in seconds at which statistics are written to file if periodic writes are enabled. Mandatory: Yes Default: `20` ### Connections tab These settings are found under the **Connections** tab. #### operatingEnvironment > heartbeatInterval Number of seconds before a Gateway sends a heartbeat message to a connected component if it does not receive any communication from the component. Gateway expects a reply within the number of seconds specified by the connectWait setting. If the reply is not received within this time, the connection is terminated and re-established. The valid range for the heartbeat interval is 20-300 seconds inclusive. Mandatory: No Default: `75` (seconds) #### operatingEnvironment > connectWait Time in seconds to wait for a connection to Netprobe to be established. That is, the maximum duration the gateway waits after sending the initial TCP SYN segment for a SYN/ACK reply from the Netprobe. The valid range is 1-300 seconds inclusive. Mandatory: No Default: `30` (seconds) #### operatingEnvironment > dnsCacheExpiryTime Time in minutes that the Gateway caches the result of resolving a hostname to an IP address. Valid values are `0-2880` inclusive. If set to `0`, hostnames are cached indefinitely. Mandatory: No Default: `720` (12 hours) #### operatingEnvironment > clientConnectionRequirements > minimumComponentVersion > minimumForAllComponents This instructs the Gateway to reject connections from every component with versions older than the specified version. You can specify the minimum version using the: - Version number. For example, `GA4.7.0`, or `GA2011.2.1`. - Version number with the build date. For example, `GA4.7.0-180529`. Mandatory: No #### operatingEnvironment > clientConnectionRequirements > minimumComponentVersion > components > component > name Name of a Geneos component type. The drop-down list has the following options: - Active Console - Gateway - Licence Daemon - Netprobe - Web Dashboard - Webslinger Mandatory: No #### operatingEnvironment > clientConnectionRequirements > minimumComponentVersion > components > component > version The minimum version of the component selected in [operatingEnvironment > clientConnectionRequirements > minimumComponentVersion > components > component > name](#operatingenvironment--clientconnectionrequirements--minimumcomponentversion--components--component--name) that the Gateway accepts connections from. You can specify the minimum version using the: - Version number. For example, `GA4.7.0`, or `GA2011.2.1`. - Version number with the build date. For example, `GA4.7.0-180529`. Mandatory: No #### operatingEnvironment > clientConnectionRequirements > requireCertificates This allows the gateway to require certificates when connections are made to the gateway for certain connection types. This can be enabled/disabled for each supported connection type. The following connection types are supported: - **Netprobe**: Incoming connections from netprobes (this will include Floating Netprobes and Self-announcing Netprobes). - **Importing Gateways**: Incoming Importing Gateway connections. - **Importing Gateways**: Incoming Gateway connections from Importing Gateways (to which this gateway exports data). - **Secondary Gateways**: Incoming Gateway connections from the secondary Gateway in a Hot Standby configuration. Mandatory: No Default: Certificates are not required for any incoming connections. #### operatingEnvironment > httpConnectionRequirements This group of settings allow HTTP requests made to the Gateway (e.g. from a web browser) to be restricted. #### operatingEnvironment > httpConnectionRequirements > internalData The internal data web pages provide low level information about various parts of the system. They may be requested by ITRS support when debugging issues. They do not form part of the normal operation of the Gateway so can safely be restricted to Geneos administrators. The internal data pages available, and the information available on each page, can vary by version. However, the connection requirements cover all internal data pages, so will secure any pages added in future versions. #### operatingEnvironment > httpConnectionRequirements > internalData > acceptHosts This allows the internal data web pages, used for debugging issues, to be viewed only from particular locations. The available settings are: - `All` — allow access from any host. - `Local` — allow access only from the local loopback interface where the Gateway is running (127.0.0.1). - `None` — prevent access completely. - `Specific` — a list of locations may be entered. Each item in the list can be specified as a hostname (if a reverse DNS entry is available for the remote host) or as an IP address. The source of any HTTP requests must match at least one item in the list otherwise they are rejected. If no items are specified, access is prevented completely. The remote hostname and IP address are written to the Gateway log file, along with the URL requested, for any attempts that are blocked. This can be useful to see if the Gateway host is able to access a reverse DNS entry for the remote host and therefore what would need to be added to the 'specific' list for the request to be accepted. If a hostname is not available then the IP address is seen instead of the name in the log file, so will appear twice. Mandatory: No Default: `local` #### operatingEnvironment > DNS > maxAcceptableDNSLookupTime The maximum time in seconds that the Gateway is allowed to perform a reverse DNS lookup. If this time is exceeded, reverse DNS lookups are disabled for the IP address for the number of units of time specified in [operatingEnvironment > DNS > DNSReverseLookupDisableTime > value](#operatingenvironment--dns--dnsreverselookupdisabletime--value). For non-Gateway components, this setting defaults to the time specified in the environment variable `$HR_TIMEOUT`. Mandatory: No Default: `1` #### operatingEnvironment > DNS > DNSReverseLookupDisableTime > value Number of units of time that reverse DNS lookups are disabled for after exceeding [operatingEnvironment > DNS > maxAcceptableDNSLookupTime](#operatingenvironment--dns--maxacceptablednslookuptime). The unit of time is specified using [operatingEnvironment > DNS > DNSReverseLookupDisableTime > units](#operatingenvironment--dns--dnsreverselookupdisabletime--value). Once the time has elapsed, reverse DNS lookups are re-enabled for the IP address. For non-Gateway components, DNSReverseLookupDisableTime can be specified using the environment variable `$HR_REVERSE_LOOKUP_DISABLE_TIME`. Mandatory: No Default: `5` #### operatingEnvironment > DNS > DNSReverseLookupDisableTime > units The unit of time used to determine how long DNS lookups are disabled for after exceeding [operatingEnvironment > DNS > maxAcceptableDNSLookupTime](#operatingenvironment--dns--maxacceptablednslookuptime). There are two options: - `minutes` - `seconds` Mandatory: No Default: `minutes` ### Debug tab These settings are found under the **Debug** tab. #### operatingEnvironment > debug A list of gateway debug settings. These settings are only intended for debugging error conditions and should be enabled with care. **Caution**  Use of these setting is likely to adversely impact the performance of the Gateway and should only be enabled when debugging a particular configuration and in coordination with ITRS support staff. ### Data Quality tab These settings are found under the **Data Quality** tab. **Note** Starting with Geneos version 7.7.0, the following data quality settings have been deprecated and replaced with probe- and sampler-specific suspension settings: - Max data age (probe only) - Connection suspension duration (probe only) - Suspend grace period (probe only) - Setup grace period (probe only) #### operatingEnvironment > dataQueues > disableChecks Enable this setting to disable the data quality checking algorithm. Mandatory: No Default: `false` (algorithm is run) #### operatingEnvironment > dataQueues > suspendOnBreach Configure whether to suspend the probe or the sampler when system thresholds are breached. This is a load shedding mechanism that suspends noisy probes or samplers to maintain Gateway performance and responsiveness. When sampler suspension is enabled, the corresponding sampler dataviews display the sampling status as `Sampler has been suspended.` **Note** Sampler suspension only works for Netprobe and Gateway samplers. Imported or dynamic samplers cannot be suspended. Mandatory: No Default: `false` (algorithm is run) #### operatingEnvironment > dataQueues > suspendOnBreach > sampler/probe > maxDataAgeMs The maximum acceptable age, in milliseconds, for a pending update to a dataview. The limit is inclusive, so an update that is equal to or older than the set value causes a connection to be suspended. The unit may change from milliseconds to seconds depending on the configured value. For more details on how this setting is used, see [Data quality options](#data-quality-options). **Note** The default value is set to approximate the behaviour of Gateway versions prior to the introduction of the data quality feature. Mandatory: No Default: `77000` (77 seconds) #### operatingEnvironment > dataQueues > suspendOnBreach > sampler/probe > connectionSuspensionDuration Time in seconds that a connection (to a Netprobe) is suspended before the Gateway reconnects. For more details on how this setting is used, see [Data quality options](#data-quality-options). Mandatory: No Default: `300` (5 minutes) #### operatingEnvironment > dataQueues > suspendOnBreach > sampler/probe > suspendGracePeriod Time in seconds specifying how long the Gateway waits after suspending a connection before allowing further connections to be suspended. For more details on how this setting is used, see [Data quality options](#data-quality-options). Mandatory: No Default: `60` (1 minute) #### operatingEnvironment > dataQueues > suspendOnBreach > sampler/probe > setupGracePeriod Time in seconds specifying how long the Gateway suspends the data quality algorithm for after a setup change. For more details on how this setting is used, see [Data quality options](#data-quality-options). Mandatory: No Default: `60` (1 minute) #### operatingEnvironment > dataQueues > memoryProtection Allows overriding the default data-queue memory protection settings. For more details on how this setting is used, see [Memory protection](#memory-protection). Mandatory: No #### operatingEnvironment > dataQueues > memoryProtection > lowPriorityThresholdMB Threshold size in MB for backlogged EMF messages at which the Gateway throttles read-data from low-priority connections. Low priority connections are importing EMF connections (Netprobe connections only). All other Gateway connections continue to operate normally. For more details on how this setting is used, see [Memory protection](#memory-protection). Mandatory: Yes Default: `500` #### operatingEnvironment > dataQueues > memoryProtection > highPriorityThresholdMB Threshold size in MB for backlogged EMF messages at which the Gateway throttles read-data from all connections. In practice it is very unlikely even for a heavily overloaded Gateway to hit this threshold, as the low-priority threshold is hit first. For more details on how this setting is used, see [Memory protection](#memory-protection). Mandatory: Yes Default: `750` #### operatingEnvironment > conflation Settings to control conflation of incoming monitoring data. For more details on how this setting is used, see [Conflation](#conflation) . Mandatory: No #### operatingEnvironment > conflation > enabled Whether conflation is enabled. Conflation can significantly aid an overloaded Gateway and ensure that all published data is as up-to-date as possible. However, it does this by discarding out-of-date cell updates and should not be enabled if this is unacceptable. For more details on how this setting is used, see [Conflation](#conflation) . Mandatory: Yes #### operatingEnvironment > conflation > strategy Specify the strategy for controlling Gateway conflation, so that conflation is only enabled when required and no updates are unnecessarily discarded. Mandatory: Yes #### operatingEnvironment > conflation > strategy > maxDataAgeThreshold Under this strategy the Gateway does not enable conflation unless the maximum age of backlogged updates (as displayed by the [Probe Data plugin](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#probe-data) ) exceeds a certain threshold. Conflation works best when it is preventing stale data from building up rather than clearing large backlogs (not only does it have fewer backlogged messages to process, but it minimises the amount of updates conflated away) and the threshold should not be set too high. An ideal value for the threshold is the minimum [samplers > sampler > sampleInterval](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/gateway_samplers/index.html#samplers-sampler-sampleinterval) used in the setup. For this reason it defaults to the default sampleInterval of twenty seconds. Mandatory: No #### operatingEnvironment > conflation > strategy > maxDataAgeThreshold > threshold Time in milliseconds for the threshold maxDataAge above which conflation is enabled. An ideal value for this setting is the minimum [samplers > sampler > sampleInterval](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/gateway_samplers/index.html#samplers-sampler-sampleinterval) used in the setup. Mandatory: No Default: `20000` #### operatingEnvironment > conflation > strategy > alwaysOn Under this strategy the gateway permanently enables conflation. This strategy always results in the most up-to-date data being published, but may cause updates to be discarded when it is not strictly necessary. Mandatory: No