Operating Environment

Overview Copied

The Gateway Operating environment top-level section contains settings that affect the Gateway as a whole, and do not belong to any other section.

Operation Copied

Although there are no mandatory settings in this section, it is essential that the Gateway name is set either with operatingEnvironment > gatewayName or using the -gateway-name or -demo command line options. This name is displayed to all users that connect to the Gateway, and is also used in name lookup and database logging. This name must be unique for each Gateway in order to use features such as Self-announcing Netprobes, Gateway Sharing and Gateway Hub connectivity.

The Gateway listen ports can also be set in the operating environment (operatingEnvironment > listenPorts). The listen ports are used by components connecting to gateway such as Active Console or Webslinger to request monitoring data for display to users.

Note

This does not include Netprobe connections, as configuration for these are contained within Probes. If not configured, the gateway listen port defaults to port 7039 for insecure channel and 7038 for secure channel.

Data quality options Copied

Settings in Operating environment control the data quality algorithm that Gateways use to maintain a consistent level of service under excessive load. This algorithm runs throughout the lifetime of a gateway (unless operatingEnvironment > dataQueues > disableChecks is set) and operates as follows:

Probes suspension may additionally be controlled by the suspend probe and unsuspend probe commands. See the commands appendix for details.

For more information regarding Data quality, see Data Quality User Guide.

Memory protection Copied

Settings for memory protection are found in the Data Quality tab.

When data quality is disabled, or in extreme situations when it cannot suspend sufficient probes to prevent the gateway becoming overloaded, the gateway throttles the reading of TCP data to prevent the backlogged data-queues from unbounded growth. This is necessary but less preferable to a managed data-quality suspension because if it continues without recovery, netprobe connections either flow-control or timeout and netprobes are dropped at random.

There are two threshold levels:

When the low-priority threshold is breached, the gateway throttles reads from all importing (netprobe) connections but remain responsive to downstream components, such as Active Console. In the unlikely event that this fails to prevent memory growth and the high-priority threshold is breached, the gateway throttles reads from all connections and become unresponsive until it recovers.

The default for the low-priority threshold is 250 MB. This is calculated to be enough to buffer 77 seconds (see operatingEnvironment > dataQueues > maxDataAgeMs) worth of data on a high bandwidth gateway (approximately 30,000 cell updates per second). This is in order to give the data-quality algorithm time to step in and save the situation before the threshold is reached.

Note

These thresholds govern memory usage by unprocessed EMF messages only. The gateway memory footprint as a whole is typically far more influenced by other factors and could potentially exceed these thresholds without issue. It is unusual for unprocessed EMF messages to account for more than a few megabytes in a normally operating gateway.

Conflation Copied

Settings for conflation are found in the Data Quality tab.

Conflation is an optional and less drastic method of coping with an overloaded gateway than a data quality suspension. When the data queues (containing incoming sampler updates from netprobes) become backlogged due to the gateway being unable to process them as fast as they arrive, conflation allows the gateway to discard out-of-date cell updates and only process and publish the latest cell values. As this could potentially result in the gateway discarding important updates, or missing short-lived events, conflation is disabled by default and should only be used with care.

Conflation examples Copied

Rapid cell updates Copied

When a Netprobe has published several updates to the same cell before the gateway has processed the first update:

With conflation active, gateway only publishes the latest value:

Updates to a recently created row Copied

When a netprobe updates values in a recently created row before the gateway has processed the create:

With conflation active, gateway adds the row to the dataview with the latest values:

Updates to a row that is then removed Copied

When a netprobe updates values in a row and then removes the row before the gateway has processed the updates:

With conflation active, gateway discards the updates and only process the row-removal:

Short lived rows Copied

When a netprobe creates a row and then removes it again before the gateway has processed the create:

With conflation active, gateway conflates away the update as normal but does not conflate away the entire row:

Potential Issues Copied

Conflation can prevent a gateway from becoming overloaded, and ensure that published values are always up-to-date, but there are a number of potential issues which you should be aware of.

Lost Spikes Copied

A dataview cell that updates from 32% to 34% to 33% is unlikely to cause issues by having the intermediate update conflated away, but one that updates from 32% to 99% to 33% may miss an important spike.

Similarly, a cell that goes from OK to ERROR to OK again, could cause an alert to be missed if conflation is enabled.

This might also affect compute-engine rules that use statistical functions such as maximum or minimum.

Rate Function Copied

Rate function triggers off the time an update is processed, rather than the time it is generated, and therefore its general performance is likely be improved by conflation.

Note

Spikes in the rate-of-change in a cell may be conflated away.
Database Logging Copied

Cell updates that are discarded by conflation will not be logged.

sampleTime and logNetprobeSampleTimeForDataItems Copied

If logNetprobeSampleTimeForDataItems is configured, cell updates may be logged with sample-times later than they were produced with. This is because the sample-time is published by the netprobe along with the sample-data and will be conflated to the latest value.

E.g. A series of updates produced at twenty second intervals by the netprobe:

Might be conflated into a single update with the latest sample-time:

The updates to cell1 and cell2 may be logged with this later sample-time. Similarly, rules that reference the sample-time may only see the later value.

Configuration Copied

Basic tab Copied

These settings are found under the Basic tab.

operatingEnvironment Copied

Gateway-wide options are configured here.

operatingEnvironment > gatewayName Copied

A short name identifying the Gateway.

A Gateway name is required for all Gateways. However, you may use either the operatingEnvironment > gatewayName field or the -gateway-name command line option when specifying a name. The command line option is often useful in orchestrated environments, such as Kubernetes. For more information see Command line options in Gateway Installation Guide.

When using database logging functionality, this name is also logged to the database, and is used to identify records for the Gateway.

Mandatory: No

If the <<timestamp>> FATAL: Gateway Mandatory 'gatewayName' has not been specified. [/gateway/operatingEnvironment] error message appears, this means that the Gateway name has not been set using the command line option, and that the setting of the configuration file with the highest priority has no value. To resolve this:

  1. Look for any enabled Gateway name settings in the Gateway Setup Editor, and check the main and include files.
  2. Click Includes > Priority field, and increase the priority of the configuration file (main or include files) that has a Gateway name setting with a value.
  3. Click Save current document to apply the changes.

operatingEnvironment > licensingGroup Copied

Group that the Gateway requests licences from on the Licence Daemon.

Mandatory: No

operatingEnvironment > listenPorts Copied

The gateway listen ports for incoming connections.

See Secure Communications for more details.

The listen port can also be specified as a command-line argument to gateway. If this is done, then the command-line value is used for the lifetime of the gateway process - it cannot be overridden or altered by editing the gateway setup file. An example of using this command-line option is shown below:

gateway2 -port <12345>

Mandatory: No Default: The gateway listens insecurely on port 7039

operatingEnvironment > listenPorts > secure Copied

This specifies that the gateway should listen securely. In order to listen securely, a SSL certificate needs to be provided using either the -ssl-certificate or -ssl-certificate-key command line option. By default if configured to be secure, the gateway will listen on port 7038. This can be overriden by using the child setting operatingEnvironment > listenPorts > secure > listenPort.

Mandatory: No

Default: The gateway will not listen securely on any port

operatingEnvironment > listenPorts > secure > listenPort Copied

This value overrides the default secure listenPort. Specify an integer in the range 1-65535.

Mandatory: No

Default: 7038

operatingEnvironment > listenPorts > insecure Copied

This specifies that the Gateway should insecurely. By default if configured to allow insecure connections, the gateway will listen on port 7039. This can be overriden by using the child setting operatingEnvironment > listenPorts > insecure > listenPort.

Mandatory: No

Default: See operatingEnvironment > listenPorts

operatingEnvironment > listenPorts > insecure > listenPort Copied

This value overrides the default insecure listenPort. Specify an integer in the range 1-65535.

Mandatory: No

Default: 7039

operatingEnvironment > selfMonitoring Copied

Self-monitoring is active by default. Use this option to configure additional settings or to disable self-monitoring.

Gateway self monitoring creates an Entity named <gateway_name> Info with the following sampler groups:

Sampler group Dataview Plugin
Database Logging (optional) Cache Database logging
Database Logging (optional) Items Database logging
Database Logging (optional) Summary Database logging
Gateway Hub (optional) Nodes Gateway Hub data
Gateway Hub (optional) Summary Gateway Hub data
Gateway Info Client Connections Client connection data
Gateway Info Entities Managed entities data
Gateway Info Gateway Gateway data
Gateway Info Netprobes Probe data
Gateway Info Setup Includes Includes data
Gateway Info Severity Count Severity count
Gateway Info Snoozed Items Snooze data
Gateway Info User Assignments User assignment data
Import Export Exported Data Exported data
Import Export Imported Data Imported data

Note

Optional sampler groups are only created if the corresponding sections are present in the Gateway setup.

operatingEnvironment > selfMonitoring > disabled Copied

Disable the default Gateway self monitoring.

If self monitoring is disabled, then names of the form <gateway_name> Info are not reserved in Gateway.

Mandatory: No

operatingEnvironment > selfMonitoring > dynamicEntityHealth Copied

If this option is enabled, then a <probe_name> Netprobe Info Managed Entities are created for all Netprobes with configured dynamic mappings. These Managed Entities will each have a Dynamic Entities Health plugin,which monitors the metrics and streams received by the Netprobe from the Collection Agent.

For more information on Dynamic Entities Health, see Dynamic Entities Health.

Mandatory: No

Default: True

operatingEnvironment > selfMonitoring > attribute Copied

Add attributes to the <gateway_name> Info entity.

This allows you to assign the self monitoring entity to the correct location in your custom hierarchy.

Mandatory: No

operatingEnvironment > var Copied

List of user environment variable definitions. See User Variables and Environments for details on how to configure environment variables.

Advanced tab Copied

These settings are found under the Advanced tab.

operatingEnvironment > description Copied

An optional description of the gateway.

Mandatory: No

operatingEnvironment > maxLogFileSizeMb Copied

Maximum size in Megabytes of the Gateway log file before it rolls that log file over.

Valid values are 1-2047 inclusive for 32-bit Gateways.

Mandatory: No

Default: 10

operatingEnvironment > logArchiveScript Copied

The name of a batch file or shell script that should be executed when the log file is rolled over.

Using operatingEnvironment > logArchiveScript overrides LOG_ARCHIVE_SCRIPT (if set). Mandatory: No

operatingEnvironment > Log time format Copied

The time format used to record timestamps in the log file.

Choose from:

Mandatory: No

Note

The log time format can also be set using an environment variable. for more information, see Specify the log file time formatting in Gateway Log File .

operatingEnvironment > timezone Copied

This sets the TZ environment variable which determines the time zone the Gateway runs in. This allows a Gateway in one country to monitor Netprobes in another country whilst keeping the time zones the same.

The time zone can be specified in any format compatible with the unix TZ environment variable:

For more information, see the POSIX manual page for the tzset command.

The Gateway attempts to validate your choice of time zone against the system time zones, and issues a warning if the time zone cannot be verified.

Mandatory: No

Default: Gateway uses the current value of the TZ environment variable. If TZ is not set, Gateway uses the local time of its host machine.

operatingEnvironment > timezoneabbreviation Copied

A list of time zone abbreviations and their default time zone regions. This is used to override the time zone abbreviations when parsing dates in rules and when parsing information from dataviews for standard formatting.

operatingEnvironment > internalQueueSizeLimit Copied

Controls the maximum length of the internal update queue. Updates to data-items (e.g. a severity change as the result of running a rule) are placed in the queue temporarily between data updates.

The default maximum limit should be adequate for normal gateway operation. If a pair (or more) of rules are configured such that an update caused by rule A makes rule B fire and update, then this can cause the internal queue to fill faster than it is processed. If the queue is completely filled an error message is logged, and gateway performance is likely to be affected. The solution is to write rules A and B to be more selective, so that they do not fire each other.

Certain compute engine rules (typically involving wildcarded paths) can also fill the processing queue during gateway startup. The queue limit can be increased to prevent warning messages if required, however this should only be done if it is known that this situation is a “one off”.

Mandatory: No

Default: 4000

operatingEnvironment > numRuleEvaluationThreads Copied

Specifies the maximum number of rule evaluation threads the Gateway can run. These threads are used to execute rules on data changes, and can be enabled if rule execution is becoming a bottleneck on a busy gateway.

It is recommended that this is not set too high as doubling the number of threads does not double throughput. It should not be set higher than the number of CPU cores available.

To set the number of rule evaluation threads used the Gateway will determine the number of available processors, using the taskset command on Linux systems or the psrset command on Solaris systems, and evaluate the numRuleEvaluationThreads variable. The lower value will be used to specify the number of threads used by the Gateway to execute rules.

The number of rule evaluation threads used is recorded in the gateway log. If the available number of processors is changed while the gateway is running then the number of threads to use is re-evaluated at the next setup change.

For more detailed information about the optimum value to use, see Gateway Performance Tuning.

A hard limit can also be placed on the number of rule threads by setting the environment variable MAX_RULE_THREADS to a positive integer. This will override the value specified in the Gateway Setup Editor.

Mandatory: No

Default: 0 (no threads used)

operatingEnvironment > historyFiles Copied

The maximum number of history files that the gateway is allowed to create when receiving set-up changes.

Valid values are 0-9999 inclusive.

To suppress history file creation altogether, set this to zero.

Mandatory: No

Default: 10

operatingEnvironment > dataDirectory Copied

Allows you to specify where temporary files which the gateway may produce while running should be stored.

If not set, files are stored in the current working directory. If the directory specified already contains any of these temporary files, they are over-written.

The data directory must have read, write, and execute permissions as it needs to be able to read, write and search within it.

Mandatory: No

Default: Current working directory

operatingEnvironment > duplicateRowAlerts Copied

When duplicate rows in a single dataview are detected, gateway alerts the user of this fact as it indicates a configuration error. These alerts can be adjusted using this setting.

Mandatory: No

Default: TICKER_AND_STATUS

Value Effect
NONE No alerts regarding duplicate rows are produced.
STATUS The dataview samplingStatus headline is updated with an error message warning about duplicate rows.
TICKER_AND_STATUS The samplerStatus headline is updated as above, and additionally an event ticker event is created regarding the duplicate rows.

operatingEnvironment > insecurePasswordLevel Copied

In a number of places throughout the Gateway configuration, passwords have to be specified.

Examples of this might be plugins that require logins to systems to retrieve the data they need, the gateway’s connection to the database, or the configuration of users. In most of these places it is possible to enter the password in a number of different formats (depending on context), from a cleartext format, to more secure formats such as AES (for two way), and crypt (for one way).

While it may be useful to use a cleartext format in a UAT or testing environment, you may prefer to ensure that a secure format is used when in a production environment. This setting helps locate these, by causing each insecure password to generate an issue at the specified level. This is shown when validating or saving the setup.

Note

We have deemed standard encoded passwords (std) to be insecure since they are encoded rather than encrypted, and these will be flagged in the same way as cleartext passwords.

The setting has the following effects:

With Warning or Error set, the ability to save the setup with insecure passwords present depends on if the -max-severity command line parameter is set. See the Gateway Installation Guide.

The Gateway Data Plug-In reports the level of this setting.

Mandatory: No

Default: None

operatingEnvironment > allowComputeEngine Copied

Specifies whether the Gateway compute engine feature is available to add additional data to existing dataviews.

It is allowed by default, but administrators and users can use this setting to disallow compute engine features. See Compute Engine.

Mandatory: No

Default: true

operatingEnvironment > writeStatsToFile Copied

The “write stats to file” section contains settings controlling how load monitoring statistics are written out from the gateway.

These statistics can then be read by the Load Monitoring plugin. Also see the Gateway Performance Tuning for more information.

Mandatory: No

Default: No statistics written.

operatingEnvironment > writeStatsToFile > filename Copied

The file to write statistics to.

Mandatory: Yes

operatingEnvironment > writeStatsToFile > enablePeriodicWrite Copied

Specifies whether to write data to file periodically.

If false, statistics are only written when the “write statistics” command is executed.

Mandatory: No

Default: true

operatingEnvironment > writeStatsToFile > writeInterval Copied

Interval in seconds at which statistics are written to file if periodic writes are enabled.

Mandatory: Yes

Default: 20

Connections tab Copied

These settings are found under the Connections tab.

operatingEnvironment > heartbeatInterval Copied

Number of seconds before a Gateway sends a heartbeat message to a connected component if it does not receive any communication from the component.

Gateway expects a reply within the number of seconds specified by the connectWait setting. If the reply is not received within this time, the connection is terminated and re-established.

The valid range for the heartbeat interval is 20-300 seconds inclusive.

Mandatory: No

Default: 75 (seconds)

operatingEnvironment > connectWait Copied

Time in seconds to wait for a connection to Netprobe to be established. That is, the maximum duration the gateway waits after sending the initial TCP SYN segment for a SYN/ACK reply from the Netprobe.

The valid range is 1-300 seconds inclusive.

Mandatory: No

Default: 30 (seconds)

operatingEnvironment > dnsCacheExpiryTime Copied

Time in minutes that the Gateway caches the result of resolving a hostname to an IP address.

Valid values are 0-2880 inclusive.

If set to 0, hostnames are cached indefinitely.

Mandatory: No

Default: 720 (12 hours)

operatingEnvironment > clientConnectionRequirements > minimumComponentVersion > minimumForAllComponents Copied

This instructs the Gateway to reject connections from every component with versions older than the specified version.

You can specify the minimum version using the:

Mandatory: No

operatingEnvironment > clientConnectionRequirements > minimumComponentVersion > components > component > name Copied

Name of a Geneos component type. The drop-down list has the following options:

Mandatory: No

operatingEnvironment > clientConnectionRequirements > minimumComponentVersion > components > component > version Copied

The minimum version of the component selected in operatingEnvironment > clientConnectionRequirements > minimumComponentVersion > components > component > name that the Gateway accepts connections from.

You can specify the minimum version using the:

Mandatory: No

operatingEnvironment > clientConnectionRequirements > requireCertificates Copied

This allows the gateway to require certificates when connections are made to the gateway for certain connection types. This can be enabled/disabled for each supported connection type. The following connection types are supported:

Mandatory: No

Default: Certificates are not required for any incoming connections.

operatingEnvironment > httpConnectionRequirements Copied

This group of settings allow HTTP requests made to the Gateway (e.g. from a web browser) to be restricted.

operatingEnvironment > httpConnectionRequirements > internalData Copied

The internal data web pages provide low level information about various parts of the system.

They may be requested by ITRS support when debugging issues. They do not form part of the normal operation of the Gateway so can safely be restricted to Geneos administrators.

The internal data pages available, and the information available on each page, can vary by version. However, the connection requirements cover all internal data pages, so will secure any pages added in future versions.

operatingEnvironment > httpConnectionRequirements > internalData > acceptHosts Copied

This allows the internal data web pages, used for debugging issues, to be viewed only from particular locations. The available settings are:

The remote hostname and IP address are written to the Gateway log file, along with the URL requested, for any attempts that are blocked. This can be useful to see if the Gateway host is able to access a reverse DNS entry for the remote host and therefore what would need to be added to the ‘specific’ list for the request to be accepted. If a hostname is not available then the IP address is seen instead of the name in the log file, so will appear twice.

Mandatory: No

Default: local

operatingEnvironment > DNS > maxAcceptableDNSLookupTime Copied

The maximum time in seconds that the Gateway is allowed to perform a reverse DNS lookup.

If this time is exceeded, reverse DNS lookups are disabled for the IP address for the number of units of time specified in operatingEnvironment > DNS > DNSReverseLookupDisableTime > value.

For non-Gateway components, this setting defaults to the time specified in the environment variable $HR_TIMEOUT.

Mandatory: No Default: 1

operatingEnvironment > DNS > DNSReverseLookupDisableTime > value Copied

Number of units of time that reverse DNS lookups are disabled for after exceeding operatingEnvironment > DNS > maxAcceptableDNSLookupTime.

The unit of time is specified using operatingEnvironment > DNS > DNSReverseLookupDisableTime > units.

Once the time has elapsed, reverse DNS lookups are re-enabled for the IP address.

For non-Gateway components, DNSReverseLookupDisableTime can be specified using the environment variable $HR_REVERSE_LOOKUP_DISABLE_TIME.

Mandatory: No

Default: 5

operatingEnvironment > DNS > DNSReverseLookupDisableTime > units Copied

The unit of time used to determine how long DNS lookups are disabled for after exceeding operatingEnvironment > DNS > maxAcceptableDNSLookupTime.

There are two options:

Mandatory: No

Default: minutes

Debug tab Copied

These settings are found under the Debug tab.

operatingEnvironment > debug Copied

A list of gateway debug settings. These settings are only intended for debugging error conditions and should be enabled with care.

Caution

Use of these setting is likely to adversely impact the performance of the Gateway and should only be enabled when debugging a particular configuration and in coordination with ITRS support staff.

Data Quality tab Copied

These settings are found under the Data Quality tab.

operatingEnvironment > dataQueues > disableChecks Copied

Enable this setting to disable the data quality checking algorithm.

Mandatory: No

Default: false (algorithm is run)

operatingEnvironment > dataQueues > maxDataAgeMs Copied

Time in milliseconds of the maximum acceptable age for a pending update to a dataview. The limit is inclusive, so an update must be older than the set value to cause a connection to be suspended.

For more details on how this setting is used, see Data quality options.

Note

The default value is set to approximate the behaviour of Gateway versions prior to the introduction of the data quality feature.

Mandatory: No

Default: 77000 (77 seconds)

operatingEnvironment > dataQueues > connectionSuspensionDuration Copied

Time in seconds that a connection (to a Netprobe) is suspended before the gateway reconnects.

For more details on how this setting is used, see Data quality options.

Mandatory: No

Default: 300 (5 minutes)

operatingEnvironment > dataQueues > suspendGracePeriod Copied

Time in seconds specifying how long the gateway waits after suspending a connection before allowing further connections to be suspended.

For more details on how this setting is used, see Data quality options.

Mandatory: No

Default: 60 (1 minute)

operatingEnvironment > dataQueues > setupGracePeriod Copied

Time in seconds specifying how long the gateway suspends the data quality algorithm for after a setup change.

For more details on how this setting is used, see Data quality options.

Mandatory: No

Default: 60 (1 minute)

operatingEnvironment > dataQueues > memoryProtection Copied

Allows overriding the default data-queue memory protection settings.

For more details on how this setting is used, see Memory protection. Mandatory: No

operatingEnvironment > dataQueues > memoryProtection > lowPriorityThresholdMB Copied

Threshold size in MB for backlogged EMF messages at which the gateway throttles read-data from low-priority connections.

Low priority connections are importing EMF connections (Netprobe connections only). All other gateway connections continue to operate normally.

For more details on how this setting is used, see Memory protection.

Mandatory: Yes

Default: 500

operatingEnvironment > dataQueues > memoryProtection > highPriorityThresholdMB Copied

Threshold size in MB for backlogged EMF messages at which the gateway throttles read-data from all connections.

In practice it is very unlikely even for a heavily overloaded gateway to hit this threshold, as the low-priority threshold is hit first.

For more details on how this setting is used, see Memory protection.

Mandatory: Yes

Default: 750

operatingEnvironment > conflation Copied

Settings to control conflation of incoming monitoring data.

For more details on how this setting is used, see Conflation .

Mandatory: No

operatingEnvironment > conflation > enabled Copied

Whether conflation is enabled.

Conflation can significantly aid an overloaded gateway and ensure that all published data is as up-to-date as possible. However, it does this by discarding out-of-date cell updates and should not be enabled if this is unacceptable.

For more details on how this setting is used, see Conflation .

Mandatory: Yes

operatingEnvironment > conflation > strategy Copied

Specify the strategy for controlling gateway conflation, so that conflation is only enabled when required and no updates are unnecessarily discarded.

Mandatory: Yes

operatingEnvironment > conflation > strategy > maxDataAgeThreshold Copied

Under this strategy the gateway does not enable conflation unless the maximum age of backlogged updates (as displayed by the Probe Data plugin ) exceeds a certain threshold.

Conflation works best when it is preventing stale data from building up rather than clearing large backlogs (not only does it have fewer backlogged messages to process, but it minimises the amount of updates conflated away) and the threshold should not be set too high.

An ideal value for the threshold is the minimum samplers > sampler > sampleInterval used in the setup. For this reason it defaults to the default sampleInterval of twenty seconds.

Mandatory: No

operatingEnvironment > conflation > strategy > maxDataAgeThreshold > threshold Copied

Time in milliseconds for the threshold maxDataAge above which conflation is enabled. An ideal value for this setting is the minimum samplers > sampler > sampleInterval used in the setup.

Mandatory: No

Default: 20000

operatingEnvironment > conflation > strategy > alwaysOn Copied

Under this strategy the gateway permanently enables conflation.

This strategy always results in the most up-to-date data being published, but may cause updates to be discarded when it is not strictly necessary.

Mandatory: No

["Geneos"] ["Geneos > Gateway"] ["Technical Reference"]

Was this topic helpful?