Gateway Hub 2.x Release Notes

 

Gateway Hub 2.1.2

Released: June 2020

This release delivers the updated binaries for Gateway Hub in the ITRS Downloads page.

New features and enhancements

These are the new features and enhancements of this release:

Issue key Release description
STARK-13762 Manually triggered data retention jobs now finish considerably quicker than before.
STARK-13761 Snapshot and API daemon logging has been reduced, resulting in smaller disk usage.
   

Known issues

These are the known issues affecting this release:

Issue key Known issue description

STARK-13168

If noexec is set on /tmp, then the following setting must be applied to /opt/mapr/hbase/hbase-1.1.8/conf/hbase-env.sh in order to use HBase shell:

export HBASE_OPTS="-Djava.io.tmpdir=/opt/hub/tmp" in ()

STARK-13049

In order to use HTTPS with Webhooks, the SSL certificate used in the Webhooks target must be added to the Gateway Hub certificate store.

STARK-6505

Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named sampleTime, netprobeTime, or dataViewId.

STARK-4463

Dataview data from additional rows computed in the Gateway is not stored for regular or non-computed columns.

STARK-4350

Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine.
   

Gateway Hub 2.1.1

Released: June 2020

This release delivers the updated binaries for Gateway Hub in the ITRS Downloads page.

Issues fixed

These are the issues we have fixed in this release:

Issue key Release description
STARK-13742 The java.io.tmpdir directory is no longer hardcoded in the warden.conf.j2 template. This prevents a java.lang.UnsatisfiedLinkError occurring when deploying Gateway Hub with custom folders.
   

Known issues

These are the known issues affecting this release:

Issue key Known issue description

STARK-13168

If noexec is set on /tmp, then the following setting must be applied to /opt/mapr/hbase/hbase-1.1.8/conf/hbase-env.sh in order to use HBase shell:

export HBASE_OPTS="-Djava.io.tmpdir=/opt/hub/tmp" in ()

STARK-13049

In order to use HTTPS with Webhooks, the SSL certificate used in the Webhooks target must be added to the Gateway Hub certificate store.

STARK-6505

Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named sampleTime, netprobeTime, or dataViewId.

STARK-4463

Dataview data from additional rows computed in the Gateway is not stored for regular or non-computed columns.

STARK-4350

Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine.
   

Gateway Hub 2.1.0

Released: May 2020

This release delivers the updated binaries for Gateway Hub in the ITRS Downloads page.

Highlights

These are the highlights of this release:

  • Performance improvements
  • Anomaly detection

Performance improvements

Gateway Hub now handles entities with up to 10,000 metrics correctly. Additionally, resource allocation has been improved.

Anomaly detection

Gateway Hub now includes an Anomaly Detection UI component. You can now overlay metric charts with the expected normal bounds of that metric based on historical data.

New features and enhancements

These are the new features and enhancements of this release:

Issue key Release description
STARK-13386 The Events browser chart now provides more granular data.
STARK-13340 The entity filter parameter in the Grafana datasource plugin no longer requires quotes.
STARK-13302 Percentiles are now displayed more compactly, for example p95.
STARK-13245 The Grafana datasource plugin now supports Entity queries.
STARK-13244 The Grafana datasource plugin now supports Event queries.
STARK-13237 GPG keys are now correctly removed when uninstalling.
STARK-13205 Gateway Hub diagnostic files now include syslog.
STARK-13199 Metric queries now have improved performance.
STARK-13184 Entity metrics can now be filtered by typing in the column headers in the Web Console.
STARK-13078 The Dashboard home list now shows Dashboard descriptions.
STARK-13053 Dashboard panels can now display metrics filtered by row name.
STARK-13047 Users with Operator privileges can no longer modify other users' dashboards when SSO is enabled.
STARK-13046 Dashboard panels now support the following aggregations: first, last, p1, and p99.
STARK-13042

You can now set the output of an entity query as a Global parameter in a Dashboard.

Panels can be configured to use the Global parameter. This allows you to quickly switch the targets of panels from one entity to another.

STARK-12974 Area charts are now available in Dashboard panels.
STARK-12907 Dashboard metric selector has been made more compact.
STARK-12838 Gateway Hub can now handle entities with large numbers of metrics, for example 10,000+.
STARK-12732 Infinity and negative infinity values are now handled correctly.
STARK-12712 Hub checker no longer checks Spark configuration.
STARK-12642 The Gateway config UI now displays the Gateway validation messages when include files are assigned.
STARK-12058 Gateway Hub no longer supports using a KeyStore and TrustStore to configure TLS/SSL. You must provide PEM files instead.
STARK-12045 All hosts must now be specified using fully qualified domain names (FQDNs). IP addresses are no longer accepted as these cause SSL issues.
STARK-9804 The Dashboard list is now searchable.
STARK-9182 The entity overview metric table now displays a floating header.
STARK-5997

You can now display the expected normal bounds when viewing metric history charts.

This allows you to compare current values with the expected values based on historical data.

   

Issues fixed

These are the issues we have fixed in this release:

Issue key Release description
STARK-13496 Gateway Hub's data retention processes no longer causes NoClassDefFoundError errors.
STARK-13373 Restarting Gateway Hub is now less likely to result in apid errors.
STARK-13351 TimescaleDB packages included in the Gateway Hub installer are now signed correctly.
STARK-13347 The Web Console no longer incorrectly shows an Enable storage to show historical data prompt when viewing an entity without historical data.
STARK-13322 Hub checker now correctly fails if a non-existing child device is supplied as a MapR disk.
STARK-13312 Dashboard text panels now correctly show their text contents.
STARK-13304 Hub checker now correctly accepts CentOS 7.7.1908 as a supported version.
STARK-13301 Dashboard metric charts no longer interpolate empty buckets.
STARK-13289 Hub checker now correctly fails when mapr_disks is not set.
STARK-13278 Hub checker no longer reports a false positive for a missing Java runtime when installing as root.
STARK-13277 Hub checker now correctly handles blank lines in the hosts file.
STARK-13268 Hub checker no longer requires disks to be located in the /dev directory.
STARK-13249 Hub checker now tests for syslinux as a prerequisite.
STARK-13222 Removing a Managed Entity no longer causes a crash in the normaliser.
STARK-13220 Hub checker now correctly handles non-existing disks.
STARK-13219 Hub checker will no longer allow partitioned disks for MapR.
STARK-13193 Adding an include file through the Gateway config UI no longer adds an empty include group.
STARK-13119 Open API definitions now correctly refer to Roles rather than Groups.
STARK-13100 Dashboard errors now clear after some time has elapsed.
STARK-13099 The Dashboard panel KPI preview now correctly reflects the user's timezone.
STARK-13089 The Open API documentation now correctly shows the ITRS-Doc-Version.
STARK-13085 Displaying a tooltip is now easier for metric charts with a single point.
STARK-13068 The Dashboard header now follows the page when scrolling.
STARK-13035 KPI icons are now optional.
STARK-12609 The Gateway config UI now identifies locked Gateways and resources.
STARK-12597 Dashboard panels now account for the user timezone correctly.
STARK-12521 Changing a Dashboard panel's visualisation type will no longer reset the title and description.
   

Known issues

These are the known issues affecting this release:

Issue key Known issue description

STARK-13168

If noexec is set on /tmp, then the following setting must be applied to /opt/mapr/hbase/hbase-1.1.8/conf/hbase-env.sh in order to use HBase shell:

export HBASE_OPTS="-Djava.io.tmpdir=/opt/hub/tmp" in ()

STARK-13049

In order to use HTTPS with Webhooks, the SSL certificate used in the Webhooks target must be added to the Gateway Hub certificate store.

STARK-6505

Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named sampleTime, netprobeTime, or dataViewId.

STARK-4463

Dataview data from additional rows computed in the Gateway is not stored for regular or non-computed columns.

STARK-4350

Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine.
   

Gateway Hub 2.0.0

Released: 8 April 2020

This release delivers the updated binaries for Gateway Hub in the ITRS Downloads page.

Note: Upgrading from Gateway Hub 1.x.x to 2.0.0 is not currently supported.

Highlights

These are the highlights of this release:

  • Dashboards
  • Metric query improvements
  • Reduced hardware requirements
  • Centralised configuration UI

Dashboards

The Web Console now includes Dashboard functionality for easy visualisation of metrics and their related events. This release includes line charts, bar charts, KPIs, and table visualisations.

Metric query improvements

Metric storage and query performance has been improved. Queries that previously took up to a minute now take a few seconds or less. This is due to a change in the way that metric data is being stored, as well as various improvements to how metric data is being retrieved.

Reduced hardware requirements

Another benefit of improved query performance is reduced hardware required to run the Gateway Hub. The minimum memory requirements have decreased from 32 GB to 16 GB, and the disk space required for storage has also decreased.

The Gateway Hub sizing tool has been updated to account for these new minimums, although it is important to note that just like with all software, better hardware will result in better performance.

Centralised configuration UI

A new administration UI is provided for centralised configuration. Although the Gateway Setup Editor is still the best place to work with include files directly, the new Web Console pages provide some additional tools such as the ability to see which Gateways are using which includes and vice versa. It also provides the ability to see the history of include files, including the diffs between versions, as well as a UI tool to upload Gateway binaries to Gateway Hub, which is required for centralised setup validation.

New features and enhancements

These are the new features and enhancements of this release:

Issue key Release description
STARK-13034 Added a View in Dashboard control to the Events viewer, allowing the current entity date filters to be passed to a dashboard.
STARK-12685 Gateway Hub no longer requires JDK for installation. JRE is now sufficient to install.
STARK-12454 A tooltip has been added to the Events overlay in metric history charts.
STARK-12419

Additional pre-install checks were added:

  • Operating system is Red Hat 7 or CentOS 7.
  • Architecture is x86_64.
  • If metric data and write ahead log locations are already created, then these are empty.
STARK-12295 Entities that have been grouped by severity are now displayed in decreasing order of severity instead of by count.
STARK-12117 A legend has been added to the Events chart in the Web Console.
STARK-11134 Creation time was added to the Events result table in the Web Console.
STARK-11098 First and Last aggregations have been added.
STARK-10296 Web Console no longer displays generic 401 error messages when a more specific error is available from the backend.

Issues fixed

These are the issues we have fixed in this release:

Issue key Release description
STARK-13142 Significantly decreased storage requirements for events.
STARK-13117 The documentation link in the About page of the snapshot versions of the Web Console (used in the snapshot versions of MiniGeneos) now correctly points to the latest published version of the documentation.
STARK-13088 Fixed a null pointer exception when Webhooks is publishing events from dataview headlines.
STARK-13023 Added ITRS-Doc-Version that was missing from the API definition for GET /documents/{id}.
STARK-12936 The Events chart tooltip now works correctly.
STARK-12782 Gateway Hub install checker now handles multiple Kafka directories correctly.
STARK-12750 Empty entity filters can no longer be set in the Web Console Monitoring home page.
STARK-12652 Events screen chart X-axis no longer overflows the panel.
   

Known issues

These are the known issues affecting this release:

Issue key Known issue description

STARK-13180

For large scale deployments, the Gateway Hub 2.0.0 installer can set PostgreSQL min_wal_size and max_wal_size values smaller than they should be.

These values are found in the {{ hub.install.postgres_data }}/postgresql.conf file, where hub.install.postgres_data is the directory you specified in your hubctl configuration file.

When these values are too small the Hub will perform poorly for both ingesting and querying metric data. You will also see entries like the following in the PostgreSQL logs at {{ hub.install.postgres_data }}/log:

2020-04-07 09:08:35.566 UTC [26000] LOG: checkpoints are occurring too frequently (1 second apart).

2020-04-07 09:08:35.566 UTC [26000] HINT: consider increasing the configuration parameter max_wal_size.

To resolve the situation, do the following on each Hub node after installation:

  1. Check the size of the disk that will contain your PostgreSQL write ahead log:
  2. df {{ hub.install.postgres_wal }} -BG
  3. Modify the PostgreSQL configuration by running the following command as the postgres user or via sudo:
  4. timescaledb-tune -pg-config /usr/pgsql-11/bin/pg_config -conf-path {{ hub.install.postgres_data }}/postgresql.conf -wal-disk-size {{ postgres_wal_disk_gb }}GB
  5. Restart PostgreSQL:
  6. systemctl restart postgresql-11.service

STARK-13168

If noexec is set on /tmp, then the following setting must be applied to /opt/mapr/hbase/hbase-1.1.8/conf/hbase-env.sh in order to use HBase shell:

export HBASE_OPTS="-Djava.io.tmpdir=/opt/hub/tmp" in ()

STARK-13049

In order to use HTTPS with Webhooks, the SSL certificate used in the Webhooks target must be added to the Gateway Hub certificate store.

STARK-6505

Gateway Hub will not correctly handle a dataview defined in a Gateway with columns named sampleTime, netprobeTime, or dataViewId.

STARK-4463

Dataview data from additional rows computed in the Gateway is not stored for regular or non-computed columns.

STARK-4350

Data that is received without a timestamp has a default timestamp incorrectly applied with the timezone of the local machine.
   

Other Gateway Hub releases

Gateway Hub 2.x Release Notes Released: 8 April 2020
Gateway Hub 1.x Release Notes Released: 21 January 2019

Disclaimer: The information contained in this document is for general information and guidance on our products, services, and other matters. It is only for information purposes and is not intended as advice which should be relied upon. We try to ensure that the content of this document is accurate and up-to-date, but this cannot be guaranteed. Changes may be made to our products, services, and other matters which are not noted or recorded herein. All liability for loss and damage arising from reliance on this document is excluded (except where death or personal injury arises from our negligence or loss or damage arises from any fraud on our part).