Data Sets (Time series)
Overview Copied
Data sets represent a placeholder for different kinds of data collected by the Gateway. Time series is the only type available.
Time Series data sets can then be used as part of Anomaly Detection and Breach predictor functionality.
Create a Time Series Copied
To create a Time Series:
- Select Data sets from the navigation tree in the GSE.
- Click New Time series.
- Select the type of the time series. The type specifies if the time series is database driven or gathers data from Gateway Hub.
- Provide all required information. See Configuration reference for more information about available options.
- Validate and click Save current document to save the changes.
Gateway Hub driven Time series Copied
Geneos can use data sets generated by Gateway Hub. The specification for these data sets is set up using GSE, and the Gateway automatically manages their generation and retrieval.
To use Anomaly Detection rules with data sets generated from Gateway Hub you should enable SSO authentication in Gateway Hub. For more information, see Connecting to Gateway Hub
The Gateway uses SSO tokens to access the data set endpoints in Gateway Hub. It is possible to connect to a Gateway Hub without authentication but this is insecure and should not be used in production environments. See Unauthenticated usage with an insecure Gateway Hub in Centralised Gateways User Guide.
To use this method, select Gateway Hub driven as Type when setting up time series. For more information, see Type — Hub.
Database driven Time Series Copied
The time series can be set up by a process external to the Gateway and stored in two tables in the database used for database logging. The tables are:
The schema for these tables is available in the Gateway resources directory provided as part of the Gateway bundle. The data is read from the database at Gateway start time and at the reload time defined in the setup.
It is up to an external process to maintain and update the time series tables. This can be controlled using the Gateway scheduled command.
To use this method, select databaseDriven as Type when setting up time series. For more information, see Configuration reference.
Table name | Description |
---|---|
time_series_user_table
|
Stores a set of names and unique IDs. The names are used to map the names
of the time series defined in the Gateway setup to the IDs used in the There are two values per row:
|
time_series_data_user_table
|
Stores the time series data. There are three values per row:
|
Prerequisites Copied
Before you can configure the database driven time series, you need to configure the database tables:
- Configure database logging. For more information, see MySQL configuration in Gateway Database Logging.
- Ensure that the tables defined in
<gateway directory>/resources/database/<database type>/time-series-schema-1.0.sql
exist. - Insert data into those tables.
Configuration reference Copied
Type — Gateway Hub driven Copied
This section provides more information about configuration options if you select your time series to be generated from Gateway Hub.
Algorithm — Seasonal-quick Copied
You can specify the algorithm to use to generate the dataset. There is currently one algorithm available, Seasonal-quick:
Setting | Description |
---|---|
Entity query |
Specifies the entity query using the entities filter syntax. For example, For more information on how to use the entity filter syntax, see Entity Filter Syntax. Note: In GSE you should only use quotes. Do not escape the quotes with backslashes because the JSON formatter in Gateway does that for you. |
Metrics |
Data you want to query. This is a sequence of raw metric names to be included in the resulting metric time series. Example: For more information about retrieving metric data, see Metric Query Example Note: In GSE you should only use quotes. Do not escape the quotes with backslashes because the JSON formatter in Gateway does that for you. |
Aggregations |
Aggregations are calculations you wish to perform on the data. The options are:
For aggregations that can be set to a specified level (for example, Aggregations are provided by Gateway Hub, for more information see Aggregations in Retrieve data from Gateway Hub. Note: The percentage aggregation available here uses the |
Granularity |
Seasonal granularity setting is used to create Time Series buckets of selected granularity. It allows you to choose how long an interval each value in the Time Series represents. You can choose from the drop-down menu, or enter a number followed by a length of time. The options in the drop-down menu are:
|
Period |
The period of cycle which defines seasonality. It allows you to choose the period over which you want the values to repeat. The options are:
|
Periods | Number of periods. It allows you to say how many recent periods (days or weeks) you want the data to be based on. |
Period settings example Copied
Here’s an example of how to use period settings (granularity, period, and periods):
You want to write a rule which will compare the current value of an item with a typical value for this time of day, based on data from the last 60 days.
- Granularity allows you to choose how long an interval each value in the time series represents. In this case, you should choose
5 minutes
or15 minutes
. - Period allows you to choose the period over which you expect the values to repeat: does your time series represent a typical day or a typical week? In this use case, you want to choose a period of
day
. If the value you are monitoring varies a lot between working and non-working days, and you have enough historical data available, you should chooseweek
. - Periods setting allows you to say how many recent days or weeks you want the data to be based on. In this use case, you want 60 days, so you should set this parameter to
60
.
Time to live Copied
This specifies the length of time the time series is valid for. The options are:
1 day
1 hour
1 week
3 hours
When the data is passed from Gateway to Gateway Hub, the default value is 1 day
.
The data set is automatically deleted after the time to live value expires.
Setting | Description | Mandatory | XML schema path |
---|---|---|---|
Time series |
Time series model a day's worth of data uploaded from the database. |
No |
timeSeries |
Name |
Specifies a name that you want to identify each time series with. If your time series are database driven, the name must correspond to one of the database tables you have configured. |
Yes |
timeSeries > name |
Description |
Specifies additional information about the time series. You can enter multi-line comments in the description field. |
No |
timeSeries > description |
External |
Specifies how the external data access is managed. |
timeSeries > external | |
External > Reload time |
Specifies the time of the day that
data should be uploaded from the database |
No (default value is the current time during time series creation). |
timeSeries > external > reloadTime |
Type |
Specifies if the time series is database driven or gathers data from Gateway Hub. This setting has two options:
|
No | timeSeries > type |