Gateway Hub Release Notes 1.3.0

Released: 14 June 2019

ITRS is pleased to announce the release of Gateway Hub 1.3.0.

Highlights

Gateway Hub is a brand new component in the Geneos ecosystem, providing the foundations for smarter monitoring, improved scalability, and a refreshed modern UI. The main features in this release are:

  • An open query API for retrieving metric data.
  • A new directory for storing Entity information.
  • Centralised Gateway configuration.

Metric Query API

Metric data can now be retrieved using an open REST API call:

POST /metrics/query

This endpoint can be used to retrieve metric data, grouped by time bucket with a number of built in aggregations such as median, mean, and percentiles, supporting simpler display of large amounts of monitored data. As of version 1.3 this endpoint is tuned for reporting and results can take several minutes, which is a substantial improvement over querying a relational database. Performance will be improved further in a future release.

A list of the aggregation functions is available using the following REST API call:

GET /metrics/aggregations

An OpenAPI definition is now available for these endpoints. Further documentation on using the metric query endpoints can be found here.

Entity Directory

This release marks the first version of the Entity directory, providing the basis for the next generation of the state tree. The entity directory brings a number of significant advantages over the existing Geneos state tree, including:

  • Support for significantly larger monitored estates
  • Better analysis and retrieval of data across groups of similar servers

The entity store is an important part of our long term goal to simplify the data model, in particular removing Gateways and Netprobes from the XPath hierarchy and moving towards dealing with Entities and Metrics only. This will have a number of benefits, including easier configuration of Self-announcing Netprobes that rebalance across different Gateways frequently, as well as ultimately simplifying the way monitoring is configured in Geneos. Although the existing state tree will remain available for existing setups, over time we anticipate you will choose to switch to the simpler model.

In 1.4 the Entity Directory will also serve as the backbone for the new web-based monitoring UI.

For more information about the improved data model, see Introduction to Entities and Metrics.

Centralised Gateway Configuration

Gateway Hub can now store and serve Gateway configuration centrally. This version allows include files to be uploaded to Gateway Hub and used by connected Gateways. This is the first step in the journey to significantly simplify administration of Gateways at scale. More features will be added in the coming release, including centralised validation and locking, as well as administration UI screens in the Web Console.

For more information about this feature, see Centralised Gateways User Guide.

New features and enhancements

Issue key Release description

STARK-8477

JMX instrumentation by various components has now been secured.

STARK-8315

Processing of schema messages from the Gateway has been substantially improved.

STARK-8248

The style for snackbar error and information messages in the Web Console has been updated.

STARK-8020

Web Console has improved handling of concurrency when changing configuration.

STARK-7913

Added a confirmation modal when leaving a Web Console page with errors.

STARK-7832

Improved design for system and ingestion error pages on mobile devices for the Web Console.

STARK-7713

In the Web Console, save buttons are now disabled in the presence of validation errors.

STARK-7628

Improved memory management for various Gateway Hub components.

STARK-7625

The Archiving jobs have been combined into a single Spark job for increased performance.

STARK-7623

A ITRS-Doc-Version HTTP header is required on all PUT endpoints, preventing concurrency issues. GET endpoints now return the current document version.

STARK-7302

Password field types in the Web Console have been improved to allow better password input.

STARK-7250

The documentation link in the About page is now correctly linked to the current Web Console version.

STARK-7228

The Web Console will now preserve the collapsed/expanded status of the sidebar between sessions.

STARK-7182

The status of Kafka publishing is now displayed in the publishing screen in the Web Console.

STARK-7132

The Web Console now handles an unlicensed system better.

STARK-7074

Ports are now configurable for Kafka, REST API, Web Console, SSO API, and MCS monitoring UI.

STARK-6877

DD MMM YYYY has been added to the list of supported date formats in the Web Console.

STARK-6688

MapR Enterprise Components (MEP) have been upgraded to 6.1.

STARK-6686

The MapR instance underpinning Gateway Hub has been upgraded to 6.1.

STARK-6526

System errors are no longer displayed alongside ingestion errors and have their own endpoint and Web Console screen instead.

STARK-6525

The Kafka offsets have been removed from ingestion errors and replaced with the raw message, so that older errors did not become outdated by Kafka rebalances and retention settings.

STARK-6520

The ingestion errors pages in the Web Console have been reworked for significantly better performance and robustness. Outdated links to old errors that have been resolved are now handled correctly.

Issues fixed

Issue key Release description
STARK-8991 Dataview columns and headline cells with a full stop/period in the name no longer cause an error when queried for metrics.
STARK-8931 Dataview columns with backticks (`) in the name no longer cause the archiver to crash.
STARK-8668 Updating a metric's type no longer causes the metric to be marked as deleted.

STARK-7617

Various improvements to the way Zookeeper is configured to improve stability.

STARK-7585

The /dataview endpoint has been updated to include a path parameter, allowing filtering by partial dataview path.

STARK-7227

The Web Console sidebar will no longer get stuck when collapsed under certain circumstances.

STARK-7174

Special characters (for example, ñ) are now handled correctly in Licence page of the Web Console.

STARK-7116

When publishing is disabled, schema and snapshot requests are no longer attempted.

STARK-6726

Column name escaping has been improved, preventing crashes when querying data from certain plugins.

Known issues

Issue key Known issue description
STARK-9095

Occasionally a container synchronisation issue occurs during an upgrade, resulting in the following error message:

fatal: [ec2-34-247-183-246.eu-west-1.compute.amazonaws.com]: FAILED! => changed=true 
attempts: 30 
cmd: /opt/mapr/server/mrconfig info containers resync local 

If this occurs, resync the containers manually using the following command:

/opt/mapr/server/mrconfig info containers resync local

STARK-7691

Due to a bug with Spark SQL, metric history cannot be retrieved from dataview columns with a semicolon in the name e.g. "test;". To resolve this issue update the dataview column name removing the semicolon.

STARK-7596

The SSL certificate install incorrectly uses the default 'hub' user when applying an SSL certificate. To specify a different install user please add the following to the hubctl installation file:

"ansible" : { 
  	"variables" : { 
		"hub_user":
 			"gwh", 
  		"hub_group":
 			"gwh", 
		} 
}

STARK-6505

Gateway Hub will not correctly handle a dataview defined in a gateway with columns named "sampleTime", "netprobeTime" or "dataViewId".

STARK-4463

Dataview data from additional rows computed in the Gateway is not stored for regular/non-computed columns.

STARK-4350

Data that is somehow received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine.

Disclaimer: The information contained in this document is for general information and guidance on our products, services, and other matters. It is only for information purposes and is not intended as advice which should be relied upon. We try to ensure that the content of this document is accurate and up-to-date, but this cannot be guaranteed. Changes may be made to our products, services, and other matters which are not noted or recorded herein. All liability for loss and damage arising from reliance on this document is excluded (except where death or personal injury arises from our negligence or loss or damage arises from any fraud on our part).

Other releases

Gateway Hub Release Notes 1.4 Released: 31 July 2019
Gateway Hub Release Notes 1.3.0 Released: 14 June 2019
Gateway Hub Release Notes 1.2.0 Released: 11 March, 2019
Gateway Hub Release Notes 1.1.0 Released: 21 January, 2019