ITRS Log Analytics 7.x Release Notes

Overview

Log Analytics release notes contain the list of all new or enhanced features and a list of all issues fixed in the current release.

To view the Log Analytics 6.x.x release notes, see ITRS Log Analytics 6.x Release Notes.

For more information, see Log Analytics documentation .

Important

ITRS has identified the following products and components impacted by Apache Log4j vulnerabilities CVE-2021-44228 and CVE-2021-45046.

To know more about the impact of this issue on ITRS Log Analytics 6.x and 7.x, see Vulnerability in Apache Log4j (CVE-2021-44228, CVE-2021-45046). A workaround is provided but future releases of Log Analytics will include the necessary changes and fixes.

Log Analytics 7.0.6

Released: 22 June 2021

New features

These are the new features of this release:

Module or component Release description
Alert Added five new alerts to detect SUNBURST attack.
Incidents Added the ability of transferring the calculated risk_value to be sent in any alarm method.
Incidents Added visibility of unassigned incidents based on the security-tenant user role.
install.sh Added the ability to update with ./install.sh -u.
   

Improvements

These are the improvements of this release:

Module or component Release description
Object permission Object filtering optimisation.
Reports Date verification with scheduler enabled tasks.
Reports UI optimisation.
   

Issues fixed

These are the issues we have fixed in this release:

Module or component Release description
Agents Addressed the security vulnerability issue: CVE-2020-28168
Alert Fixed the problem with Syslog notifications.
Alert Fixed the problem with Test Rule functionality.
Alert Addressed the security vulnerability issue: CVE-2020-28168
Archive Addressed the security vulnerability issue: CVE-2020-28168
Cerebro Addressed the security vulnerability issue: CVE-2019-12384
Kibana-xlsx-import Addressed the security vulnerability issue: CVE-2020-28168
Login Addressed the security vulnerability issue: CVE-2020-28168
Reports Addressed the security vulnerability issue: CVE-2020-28168
Reports Fixed errors related to background tasks.
Sync Addressed the security vulnerability issue: CVE-2020-28168
   

Log Analytics 7.0.5

Released: June 2021

New features

These are the new features of this release:

Module or component Release description
Agents Added an index rotation using a roll-over function.
Alert Added a counter which displays information about the number of rules there are in a given group.
Alert Added an index rotation using a roll-over function.
Alert The first group is now expanded by default.
Alert A new alert method for Syslog was added to GUI.
Archive Added a compression level support: archive.compressionOptions [kibana.yml]
Archive Added a mapping or template import support.
Archive Added a number of matches in files.
Archive Added regexp and extended regexp support.
Archive A size of the created archive is now displayed on the list of files for selection.
Archive Added support for archiving a selected field from the index.
Archive Added timestamp field for custom time-frame fields.
Audit Added an index rotation using a roll-over function.
Config Added configuration possibility for Rollover for audit, alert and .agents indexes in the Settings tab.
iFrame embedding support Added a new directive in kibana.yml: login.isSameSite ["Strict" or "None"].
Object permission When deleting an object to a role in the Object permission, it is now possible to delete related objects at the same time.
Plugin A new plugin: Wiki — integration with wiki.js.
Reports Possibility to delete multiple tasks at once.
Reports Added a details field for each task that includes information about user, time range, and query.
Reports Added Scheduler for the Data Export tab.
Reports Fields that can be exported are now alphabetical, searchable list.
Reports Scheduled tasks now support the following: enable, disable, delete.
Reports Scheduled tasks now support the following: Logo, Title, Comments, PDF, JPEG, CSV, HTML.
Other Installation support for CentOS  7/8, Red Hat 7/8, Oracle Linux 7/8, Scientific Linux 7, CentOS Stream.
   

Improvements

These are the improvements of this release:

Module or component Release description
Access management Plugin Login for app management is now displayed as Config.
Alert Added support for nested fields in the blacklist-ioc alert type.
Alert Alert Dashboard was rewritten to the alert_status pattern. This allows you to filter visible alarms per use.
Alert Cardinality — a fix has been applied to the _thread._local object not having an alerts_sent attribute.
Alert Chain/Logical — a few improvements for output content.
Alert Rule type example is now hidden by default.
Alert RunOnce — improved results output.
Alert RunOnce — information that the process has finished is now displayed.
Alert TestRule — improved error output.
Archive Added document sorting which speeds up the Elasticsearch response.
Archive Only an admin can now use the API security (previously this was only visual information).
Archive Archiving process uses a direct connection, bypassing the elastfilter - proxy.
Archive Changed UTC time to local time.
Archive Information about problems with reading or writing to the archive directory.
Archive Optimised function for loading large files — improved loading time.
Archive Optimised saving method to a temporary flat file.
Archive Optimised scroll time which speeds up Elasticsearch response.
Audit Converted SEARCH _id: auditselection to GET _id: auditselection.
Audit Removed background task used for refresh audit settings.
Beats Updated to v6.8.14.
Blacklist-IOC Added Duplicates removal mechanism.
Blacklist-IOC Automatic configuration of repository access during installation [install.sh].
Cerebro Updated to v0.9.3.
Config Character validation for user names and roles can now only consist of letters a-z, A-Z, numbers 0-9 and the following characters: underscore _ and dash -.
Config Deleting a user deletes their tokens or cookies immediately and causes logging out.
Config Securing the default administrator account against deletion.
Config Session time-out redirects into login screen from all modules.
Config Workaround for automatic filling of fields with passwords in modern browsers.
Curator Updated to v5.8.3 and added support for Python 3 as default.
ElasticDump Updated to v6.65.3 and added support for backing up all templates at once.
Elasticsearch Removed the default user "scheduler" with the admin role.
Elasticsearch Removed indices.query.bool.max_clause_count from the default configuration as it was causing performance issues.
Elasticsearch Role caching improvements.
GEOIP Automatic configuration of repository access during installation [install.sh].
Incidents Switching to the Incidents tab creates pattern alert if one does not already exist.
install.sh Added workaround for cluster.max_shards_per_node=1000 bug.
Kibana Removed kibana.autocomplete from default configuration as it was causing performance issues.
License Revision and update of license files in all system modules.
Logstash Updated logstash-codec-sflow to v2.1.3.
Logstash Updated logstash-input-beats to v6.1.0.
Logstash Updated to v6.8.14.
Logtrail Added default action file for curator to clean logtrail indexes after 2 days.
Network visualization Corrected the legend and improved colours.
Reports Added the Switch button for filtering only the scheduled tasks.
Reports Admin users now see all scheduled reports from every other user.
Reports Changed Export Dashboard to Report Export.
Reports Changed Export Task Management to Data Export.
Reports Crontab format validated before Submit in Schedule.
Reports Default task list is now sorted by start time.
Reports Improved security by using kernel namespaces. Dropped suid permissions for chrome_sandbox.
Reports Moved the Schedule Export Dashboard to the Report Export tab.
Reports Try catch for async getScheduler function.
Skimmer

Added the following alerts:

  • High_lag_on_Kafka_topic

  • High_node_CPU_usage

  • High_node_HEAP_usage

  • High_Flush_duration

  • High_Indexing_time
Skimmer New metric: _cat/shards.
Skimmer New metric: _cat/tasks.
Skimmer Updated to v1.0.17.
small_backup.sh Added sync, archive, and wiki support.
small_backup.sh Information about the completed operation is now logged.
Wazuh Searching in the rule description field.
   

Issues fixed

These are the issues we have fixed in this release:

Module or component Release description
Access management Fixed some UI related issues in the apps select box for default roles (admin, alert, intelligence, and kibana).
Alert Category name now appears on the Risk list.
Alert Description update for find_match alert type.
Alert Fixed a bug where after renaming the alert it was not immediately visible on the list of alerts.
Alert Fixed a bug where editing an alert caused it to return to the Other group.
Alert Fixed an incorrect function alertMethodData a problem with TestRule operation [itrs op5 alert-method].
Alert Fixed a problem with [] in rule names.
Alert Fixed a process status in the Alert Status tab.
Alert Fixed a problem in groups: if there is pagination, it was not possible to change the page because it did not occur with the default group Others.
Alert Missing op5_url directive in /opt/alert/config.yaml [itrs op5 alert-method].
Alert Missing smtp_auth_file directive in /opt/alert/config.yaml [itrs op5 alert-method].
Alert Missing username directive in /opt/alert/config.yaml [itrs op5 alert-method].
Alert Overwrite config files after updating; it now creates /opt/alert/config.yml.rpmnew.
Archive Fixed an exception during connection problems to Elasticsearch.
Archive Missing symlink to runTask.js.
Cerebro Fixed problems with PID file after cerebro crash.
Cerebro Overwrite config files after updating, now it should create /opt/cerebro/conf/application.conf.rpmnew.
Config Fixed the issue with SSO login misreading application names entered in Access Management.
Elasticsearch Fixed No value present message log when not using a radius auth [properties.yml].
Elasticsearch Fixes nullPointerException by adding default value for licenseFilePath [properties.yml].
Incidents Fixed a problem with vanishing status.
install.sh Opens the ports required by logstash via firewall-cmd.
install.sh Set openjdk11 as the default Javafor the operating system.
Kibana

Fixed an exception during connection problems to Elasticsearch.
Kibana Fixed URL shortening when using Store URLs in session storage.
Logtrail Fixed missing logrotate definitions for Logtrail logfiles.
Logtrail Fixed the problem with overwriting config files after update. Now it will create /usr/share/kibana/plugins/logtrail/logtrail.json.rpmnew.
Object Permission Fixed permission verification error if the overwritten object title changes.
Reports Fixed Image Creation failed exception.
Reports Fixed permission problem for checkpass Reports API.
Reports Fixed problems with AD, Radius, and LDAP users.
Reports Fixed a problem with choosing the date for export.
Reports Fixed setting default index pattern for technical users when using HTTPS.
Skimmer Changed kafka.consumer_id to number in default mapping.
Skimmer Fixed in indices stats monitoring.
Skimmer Fixed the problem with overwriting config files after update. Now it will create /opt/skimmer/skimmer.conf.rpmnew.
   

Log Analytics 7.0.4

Released: 15 December 2020

New features

These are the new features of this release:

Module or component Release description
Alert New Alert method for the OP5 Monitor added to GUI.
Alert New Alert method for Slack added to GUI.
Alert The ability to rename an already created rule was added.
Alert Groups for different alert types.
Alert Possibility to modify all alarms in a selected group.
Alert Calendar for managing notifications.
Alert Escalate the alarm after a specified time.
Alert The Hive integration.
Beats Beats added to the installation package.
Central Agents Management (masteragent) Stop, start, and restart for each registered agent.
Central Agents Management (masteragent) Status of detected beats and master agent in each registered agent.
Central Agents Management (masteragent) Tab with the list of agents can be grouped.
Central Agents Management (masteragent) Auto rolling documents from .agents index based on a Settings in the Config tab.
Dashboards Possibility to play a sound in the dashboard.
QualysGuard Integration with the dedicated dashboard.
Tenable.SC Integration with the dedicated dashboard.
Wazuh Added the installation package.
Other New plugin: Archive specified indices.
Other Applications access management based on roles.
   

Improvements

These are the improvements of this release:

Module or component Release description
Alert Added sorting of labels in comboxes.
Alert Chain/Logical Introduced a few improvements.
AD integration Domain selector on the login page.
Audit Cache for audit settings (performance).
Diagnostic-tool.sh Added cerebro to audit files.
Incidents New field was added: ToSkipForVerify. This is an option for skipping false-positives.
Installation script The setup script validates the license.
Installation script Support for CentOS 8.
Object permission When adding an object to a role in Object permission it is now possible to add related objects at the same time.
Skimmer New metric added: increase of documents in a specific index.
Skimmer New metric added: size of a specific index.
Skimmer New metric added: expected data nodes.
Skimmer New metric added: Kafka offset in Kafka cluster.
User roles Alphabetical, searchable list of roles.
User roles List of users assigned to a given role.
   

Issues fixed

These are the issues we have fixed in this release:

Module or component Release description
Alert Aggregation schedule time.
Alert Loading new_term fields.
Alert RecursionError: maximum recursion depth exceeded in comparison.
Alert Match_body.kibana_discover_url malfunction in aggregation.
Alert Dashboard Recovery from the Alert Status tab.
Dashboards Logserver_table removed in 7.x.x. It has been replaced with basic table.
Elasticsearch-auth Forbidden — not authorized when querying an alias with a wild card.
Logstash Mikrotik pipeline — failed to start pipeline.
Reports Black bars after JPEG dashboard export.
Reports Problems with Scheduled reports.
Other Role caching fix for working in multiple node setup.
   

Log Analytics 7.0.3

Released: 23 September 2020

New features

These are the new features of this release:

Module or component Release description
Alert New alert type: Chain. It creates alerts from underlying rules triggered in a defined order.
Alert New alert type: Logical. It creates alerts from underlying rules triggered with defined logic (OR,AND,NOR).
Alert Correlate alerts for Chain and Logical types. An alert is triggered only if each rule returns thesame value (for example, IP, username, process).
Alert Each triggered alert is indexed with unique alert_id — the field added to the default field schema.
Alert Processing Time visualization on Alert dashboard — it is now easier to identify badly designed alerts.
Alert Support for automatic search link generation.
Auditing Added an IP address field for each action.
Auditing Added the possibility to exclude values from auditing.
Input Added MikroTik parsing rules.
MasterAgent Added the possibility for beat agent restart and the master agent itself (GUI).
Skimmer Indexing rate visualization.
Skimmer New metric: offset in Kafka topics.
Skimmer New metric: expected-datanodes.
   

Improvements

These are the improvements of this release:

Module or component Release description
Alert Improved performance with multi thread support (now default).
Alert Validation of email addresses in the Alerts plugin.
Alert Difference rule description include examples for alert recovery function.
Blacklist Name field and Field names in the Fields column & Default field exclusions
Blacklist runOnce is now only terminated on a fatal Alert failure.
Blacklist IOC excludes threats marked as false-positive.
Incidents New design for Preview.
Incidents A new feature was added: Note. It provides the ability to add notes to incidents.
Logstash MasterAgent pipeline shipped by default
Logtrail Improved the beauty and readability of the plugin
MasterAgent Possibility to exclude older SSL protocols.
MasterAgent Now supports Centos 8 and related distros.
Risks Possibility to add new custom value for risk without the need to index that value.
Security jquery updated to 3.5.1.
Security Bootstrap updated to 4.5.0.
Skimmer Service status check was rewritten to dbus API.
XLSX import Updated to 7.6.1.
Other The Help button in Kibana now leads to the official product documentation.
Other Centralization of previous alert code changes to a single module.
Other Adding sample data and web sample dashboard from the home page was fixed. Changes were made in the default-base-template.
Other Copy/Sync now supports insecure mode (operations without certificates).
Other Search and sort support was added for the User List in the Config section.
   

Issues fixed

These are the issues we have fixed in this release:

Module or component Release description
Alert .alertrules is not a required index for proper system operation.
Alert /opt/alerts/testrules is not a required directory for proper system operation.
Alert .riskcategories is not a required index for proper system operation
Alert Overwriting an alert when trying to create a new alert with the same name.
Alert Wrong Alert status in the alert status tab.
Blacklist Removal of the doc type in blacklist template.
Blacklist Problem with generate_kibana_discover_url: true directive.
Reports Export to CSV supports the STOP action.
Reports Scroll errors CSV csv exports.
Reports When exporting dashboards, PDF generates only one page or cuts the page.
Skimmer Forcemerge caused under 0 values for cluster_stats_indices_docs_per_sec metric.
Other Individual special characters caused problems in user passwords.
Other Bad permissions for scheduler of Copy/Sync module has been corrected
Other diagnostic-tool.sh: wrong name for the archive in output..
Other Malfunction in Session Timeout.
Other Missing directives service_principal_name in bundled properties.yml.
Other Wrong product logo when viewing dashboards in full screen mode.
   

Log Analytics 7.0.2

Released: 29 June 2020

New features

These are the new features of this release:

  • Creating manual incidents from the Discovery section.

  • New Kibana plugin — Sync/Copy between clusters.

  • Analyzing historical data with a defined alert.

  • Indicators of compromise (IoC) — providing blacklists based on Malware Information Sharing Platform (MISP).

  • Automatic update of MaxMind GeoIP Databases [asn, city, country].

  • Extended LDAP support.

  • Cross cluster search.

  • Diagnostic script to collect information about the environment, log files, configuration files — utils/diagnostic-tool.sh.

  • New beat: op5beat — dedicated data shipper from OP5 Monitor.

Improvements

These are the improvements of this release:

  • Added _license API for Elasticsearch (it replaces the license path which is now deprecated and will stop working in future releases)

  • _license API now shows expiration_date and days_left.

  • Visual indicator on the Config tab for expiring license (for 30 days and less).

  • Creating a new user now requires re-entering the password.

  • Complexity check for password fields.

  • Incidents can be supplemented with notes.

  • Alert Spike: more detailed description of usage.

  • ElasticDump added to base installation — /usr/share/kibana/elasticdump.

  • Alert plugin updated — frontend.

  • Reimplemented session timeout for user activity.

  • Skimmer: new metrics and dashboard for Cluster Monitoring.

  • Wazuh config/keys added to the small_backup.sh script.

  • Logrotate definitions for Logtrail logfiles.

  • Incidents can be sorted by Risk value.

  • UTF-8 support for credentials.

  • Wazuh: wrong document_type and timestamp fields.

Issues fixed

These are the issues we have fixed in this release:

  • Audit: Missing Audit entry for successful SSO login

  • Report: "stderr maxBuffer length exceeded" — export to CSV.

  • Report: "Too many scroll contexts" — export to CSV.

  • Intelligence: incorrect work in updated environments.

  • Agents: fixed wrong document type

  • Kibana: "Add Data to Kibana" from Home Page.

  • Incidents: the preview button uses the wrong index-pattern.

  • Audit: Missing information about login errors of ad/ldap users.

  • Netflow: fix for netflow v9.

  • MasterAgent: none/certificade verification mode should work as intended.

  • Incorrect CSS injections for dark theme.

  • The role could not be removed in specific scenarios.

Log Analytics 7.0.1

Released: 19 March 2020

New features

These are the new features of this release:

  • Major update is now based on Elasticsearch 7.3.2, Kibana 7.3.2 and Logstash 6.8.6.

  • Migration of all existing features from version 6.1.8 and below.

  • New plugin for working with XLSX import.

  • Embedded curator for index management.

  • New design for system and underlying plugins.

Improvements

These are the improvements of this release:

  • All node_modules Kibana dependencies updated.

  • Report plugin redesigned.

Issues fixed

These are the issues we have fixed in this release:

  • Fixed the alert type description.

  • Fixed empty report.

 

Disclaimer: The information contained in this document is for general information and guidance on our products, services, and other matters. It is only for information purposes and is not intended as advice which should be relied upon. We try to ensure that the content of this document is accurate and up-to-date, but this cannot be guaranteed. Changes may be made to our products, services, and other matters which are not noted or recorded herein. All liability for loss and damage arising from reliance on this document is excluded (except where death or personal injury arises from our negligence or loss or damage arises from any fraud on our part).

Other releases

ITRS Log Analytics 7.x Release Notes

Released: September 2020

Last updated: June 2020
ITRS Log Analytics 6.x Release Notes

Released: September 2018

Last updated: February 2020
ITRS Log Analytics 2.x Release Notes

Released: June 2018

Last updated: August 2018