Obcerv Platform 1.x Release Notes

Overview Copied

Obcerv Platform release notes contain the list of new features and known issues in the Obcerv Platform.

For the latest highlights, see What’s New in Obcerv.

Before you install Obcerv, make sure to review the Obcerv Compatibility Matrix. For details on installation, administration, and usage, you may refer to Obcerv Documentation.

To learn more about Obcerv, visit the Obcerv product page.

Obcerv Platform 1.4.1 Copied

Released: 19 July 2023

Note

Upgrading to Obcerv Platform version 1.4.1 is supported from version 1.3.2 and later.

New features and enhancements Copied

These are the new features and enhancements of this release:

Issue key Release description
HP-1876 The chunk tuner now automatically adjusts Timescale chunk sizes to optimize Postgres performance and mitigate resource exhaustion due to large chunks. A signal is created for large chunks that need manual attention. For more details, see Troubleshooting.
HP-1937 Certain table triggers were removed to allow late-arriving data to be inserted into Timescale chunks that were already compressed.
HP-1981 Relaxed task validation to allow multiple tasks that match the same entity.

Issues fixed Copied

These are the issues we have fixed in this release:

Issue key Release description
HP-1933 Fixed an issue in sinkd where a data point may not be inserted due to stale configuration.
HP-1950 Entities and signals that were evicted but reappeared while still cached no longer cause sinkd to crash.
HP-1951 The timeseriesDiskSize can now be changed after the initial installation, since the parameter is now mutable.
HP-1952 Upgraded Snappy and Guava dependencies due to security issues.

Obcerv Platform 1.4.0 Copied

Released: 31 May 2023

Note

Obcerv Platform version 1.4.0 requires you to upgrade to App Query Service 1.8.0, since its earlier versions are not compatible with Obcerv Platform 1.4.0.

Upgrade notes Copied

Make sure to read and follow the steps in this section before upgrading the Obcerv Platform.

Obcerv configuration Copied

Before upgrading, you must compare your current configuration with the latest reference configurations to identify any changes that should be applied for compatibility with the new version.

While comparing, use these guidelines to determine which version of the configuration settings to use:

Data migration Copied

During the upgrade, a migration job will restructure large amounts of data in Timescale. This job will continue to run in the background after the upgrade is complete and may take several hours.

Caution

Running out of disk space is possible during the migration. Before initiating the upgrade, make sure that there is at least 60% free space on the Timescale data volumes.

If the migration fails due to insufficient capacity, you should expand those PVCs, and the migration will automatically resume. There may be gaps in metric queries until the migration is complete.

To check the free space:

$ kubectl exec -n itrs -it timescale-0 -c timescale -- df -h |grep postgres

/dev/sde         98G  1.2G   92G   2% /var/lib/postgresql
/dev/sdf         98G  4.4G   89G   5% /var/lib/postgresql/wal
/dev/sdg        503G  3.4G  475G   1% /var/lib/postgresql/tablespaces/timeseries_tablespace_2
/dev/sdd        503G  3.7G  474G   1% /var/lib/postgresql/tablespaces/timeseries_tablespace_1

If you see only the first two filesystems, ensure the usage percentage for /var/lib/postgresql is at most 40% (which means 60% free). The corresponding PVCs are named timescale-ha-data-timescale-X. If you need to grow the PVCs, grow all that match that name pattern (there should be 3).

If you see any filesystems named *tablespace*, then ensure each of those is at most 40% used (which means 60% free). The corresponding PVCs are named timescale-ha-tablespace-data-X-timescale-Y. If you need to grow the PVCs, grow all that match that name pattern.

New features and enhancements Copied

These are the new features and enhancements of this release:

Issue key Release description
HP-1197 Improved support for multiple availability zones.
HP-1519 Multi-table metrics schema used for improved scalability.
HP-1572 Signal service subscriptions now include the last signal event for any signal that is active at the start of a signal subscription window.
HP-1573 The GetLatestSignals queries can now be filtered by severity and snooze status.
HP-1574 Attribute subscription starting from the past now returns the last known value before the beginning of the look-back period.
HP-1592 Signals data is now persisted in Postgres, while the GetLatestSignals API is now paginated.
HP-1596 Signal event timestamps are now preserved for signal events occurring while a signal is snoozed.
HP-1662 Entity and entity attribute subscriptions now return an indication that the end of the snapshot has been reached.
HP-1703 The Gateway name is now stored as a searchable label in Loki.
HP-1710 Signal subscription now supports retroactive subscription from a provided start time.
HP-1713 Ensured that DP headers contain valid events and observed timestamps during datapoint validation.
HP-1723 Enhanced hypertable retention configuration (phase one).
HP-1731 Scheduled job to defragment etcd.
HP-1734 Changed the default metric data retention in Timescale.
HP-1767 Datapoint validation ensures that dimensions are not empty.
HP-1768 Size metrics for non-hypertable Postgres tables are now collected.
HP-1776 Included namespace in the CA DaemonSet persistence path.
HP-1779 Datapoint validation is now performed at the edge CAs.
HP-1780 The schema version now matches the Obcerv Platform version.
HP-1806 Enabled pg_rewind for recovery from missing timeline during replication.
HP-1808 Patroni config changes now correctly applied on upgrades.
HP-1814 Improved signal retention, TTL, and eviction.
HP-1838 Implemented a background job for the migration of pre-1.4 metrics data.
HP-1842 Supported configurable JVM override options for the DS stream.
HP-1862 Created a utility docker image for Obcerv.
HP-1867 Minimum Timescale chunk sizes and retention configuration are now enforced.
HP-1905 Signal data is migrated from Kafka to Postgres during upgrade.
HP-1906 Added support for Geneos signals that contain geneos_signal/ or geneos_snooze/ prefixes to the signal name based on the type of event. These are now normalized to a geneos/ prefix.
HP-1907 IES events and entity severity updates are migrated during upgrade.
HP-1912 Changed IAM configuration to support offline_access tokens, which are used by the web platform to keep sessions open.

Issues fixed Copied

These are the issues we have fixed in this release:

Issue key Release description
HP-1763 Datapoints with future timestamps are now rejected.
HP-1761 Added a missing Keycloak configuration for valid post-logout redirect URIs.
HP-1765 Fixed an issue where multiplexed DPD pipelines did not consider entity attribute mutations.
HP-1772 Fixed a platform-metrics error when parsing Kubernetes stats with a number that exceeded the maximum long value.
HP-1789 Having an externalHostname with more than 64 characters no longer results in an install error.
HP-1798 Kafka consumer now properly transitions to an error state upon leaving a consumer group.
HP-1805 KVS no longer throws an exception when accessing an entry with a name that is the prefix of another entry.
HP-1855 Fixed an issue where upon changing classification rules or re-installing the System Overview app, some entities could end up with an inconsistent or missing inactive-since or severity attribute.
HP-1857 Fixed an issue where DS Stream throws NPE on a null TimeBucket.
HP-1861 Resolved an issue where the DS stream fails if the status metric is too large.
HP-1885 Missing entity containment rules no longer cause entities without a hierarchy to be skipped by eviction.
HP-1900 Fixed an error where the Kubernetes CRI log collector parse fails if the message contains stderr or stdout.
HP-1908 Timeseries records are no longer duplicated.

Obcerv Platform 1.3.2 Copied

Released: 5 April 2023

Important

Upgrading to version 1.3.2 is mandatory for those using previous versions of the Obcerv Platform. Depending on your configuration, the upgrade process may take some time to complete.

New features and enhancements Copied

These are the new features and enhancements of this release:

Issue key Release description
HP-1760 Pending or unscheduled pods for the CA DaemonSet are prevented from blocking deployment.

Issues fixed Copied

These are the issues we have fixed in this release:

Issue key Release description
HP-1816 Corrected how TimestampNanos is used on entity-attributes-updates records.
HP-1832 Fixed an issue where the CA DaemonSet nodeSelector parameter failed to take effect.

Obcerv Platform 1.3.1 Copied

Released: 20 February 2023

New features and enhancements Copied

These are the new features and enhancements of this release:

Issue key Release description
HP-1715 The ETCD_SNAPSHOT_COUNT value has been reduced to 10000.
HP-1737 The etcd quota size can now be configured to a maximum of 8GB.

Issues fixed Copied

These are the issues we have fixed in this release:

Issue key Release description
HP-1637 Fixed an issue where out-of-order metrics caused DS metrics stream performance degradation.
HP-1730 Fixed an issue where NullPointerException collects Kubernetes fs usage percentage metrics.
HP-1736 maxPageSizenow works properly in GetMetricMetadata.
HP-1743 Resolved a false positive connection leak warning in sinkd.
HP-1756 The FES setting attribute now correctly updates the timestamp in nanoseconds instead of in milliseconds.
HP-1762 The FES setting attribute now correctly updates the timestamp in nanoseconds instead of in milliseconds.

Obcerv Platform 1.3.0 Copied

Released: 30 January 2023

New features and enhancements Copied

These are the new features and enhancements of this release:

Note

Upgrading to this version requires App Query Service version 1.6.0.
Issue key Release description
HP-1623 The usage of root user in Docker images has been audited.
HP-1632 Timescale has been upgraded to version 2.8.1, while PostgreSQL has been upgraded to version 14.6.
HP-1661 A new parameter, jvmOpts, has been added so the heap % can be overridden. While the default should be blank, these parameters should only be used in one-off scenarios when there is a necessity for changing the default % heap usage.

Issues fixed Copied

These are the issues we have fixed in this release:

Issue key Release description
HP-1589 Data points affecting entity attributes and status metrics that are received out of order due to an upstream load-balancer are no longer getting applied.
HP-1594 Compiled entity expressions now support status metrics. This means that searches can now be performed on status metrics.

Obcerv Platform 1.2.0 Copied

Released: 24 October 2022

Highlights Copied

These are the highlights of this release:

New features and enhancements Copied

These are the new features and enhancements of this release:

Issue key Release description
HP-1405 Signals now have the ability to filter subscriptions.
HP-1450 Timescale has been upgraded to version 2.7.
HP-1452 Kafka has been upgraded to version 3.2.0. This includes the upgraded RocksDB version 6.29.4.1, which adds support for ARM-based Macs.
HP-1470 Protobuf has been upgraded to version 3.21, while gRPC has been upgraded to version 1.47.
HP-1511 Open Telemetry is now supported with a new ingestion service in the Obcerv Platform.
HP-1582 History retention for signals has been increased to 7 days.
HP-1583 Kafka has been upgraded to version 3.2.3 to address the security vulnerability: CVE-2022-34917.
HP-1584 SnakeYAML has been upgraded to version 1.32.
HP-1586 Keycloak has been upgraded to version 19.02.
HP-1600 Kubernetes node, pod, and container FS usage percent metric are now collected.

Issues fixed Copied

These are the issues we have fixed in this release:

Issue key Release description
HP-1532 An internal server error for the Metric-forecastd pod is no longer encountered in the web console.
HP-1533 Fixed an issue where the Best Fit bucket size attempts to select an invalid bucket size (no bucket, such as with raw data) for a short-time window when forecasting is enabled.

Obcerv Platform 1.1.0 Copied

Released: 1 August 2022

Highlights Copied

These are the highlights of this release:

New features and enhancements Copied

These are the new features and enhancements of this release:

Issue key Release description
HP-791 Support for automatic eviction of stale entities (those which have not received data for an extended period of time). Eviction rules can be configured using the Obcerv operator.
HP-1401 When the count function is used, unit values are omitted from GetMetrics* responses.
HP-1423 Support for tablespaces within Postgres/Timescale. This is helpful for very large installations when there are constraints on the maximum volume sizes set by a cloud service provider.

Issues fixed Copied

These are the issues we have fixed in this release:

Issue key Release description
HP-1471 Manually resizing WAL disks now also updates Postgres tuning settings.
HP-1475 Missing Kubernetes labels have been added to workload pods.
HP-1507 The PL/Java libjvm path setting is correctly replicated in Timescale secondaries.

Obcerv Platform 1.0.2 Copied

Released: 14 July 2022

Issues fixed Copied

These are the issues we have fixed in this release:

Issue key Release description
HP-1455 When an existing entity classification gets deleted, some entities can be left enriched with attributes defined in the classification. The issue is caused by how RocksDB handles its cache updates when used in conjunction with Kafka streams.
HP-1468 An error in deploying Obcerv when the cluster can’t resolve the node hostname has been fixed.

Obcerv 1.0.0 Copied

Released: 1 June 2022

Highlights Copied

These are the highlights of this release:

Known issues Copied

These are the known issues affecting this release:

Issue key Known issue description
CPPUB-114 Connection attempts fail when the Capacity Planner app uses the Apache HTTP Proxy. This is due to LinkerD preventing the Capacity Planner app from publishing data to Capacity Planner.
HP-1387 Before successfully completing the setup, an error using the PL/Java extension can occur on the first set of queries for down-sampled metrics. This happens when the bucket_function is for percentiles and the time bucket is 5 minutes or more (down-sampled metrics). The PL/Java extension initialises a Java virtual machine (JVM) per connection, while the startup takes a few seconds to handle the first query in the connection. The query will fail due to the error. However, subsequent queries are likely to succeed since enough time may have passed for the connection’s PL/Java JVM to initialise.
HP-1433 When you manually set a Timescale multi-server cluster to only one server using kubectl, Timescale read operations will fail.
HP-1434 If no secondary Timescale is available in a multi-server cluster, Timescale read operations will fail.

Disclaimer

The information contained in this document is for general information and guidance on our products, services, and other matters. It is only for information purposes and is not intended as advice which should be relied upon. We try to ensure that the content of this document is accurate and up-to-date, but this cannot be guaranteed. Changes may be made to our products, services, and other matters which are not noted or recorded herein. All liability for loss and damage arising from reliance on this document is excluded (except where death or personal injury arises from our negligence or loss or damage arises from any fraud on our part).
["Obcerv"] ["Release Notes"]

Was this topic helpful?