--- title: "Data Sets (Time series)" description: "" url: "https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/data-access-and-publishing/data_sets-time_series/index.html" --- ## Overview Data sets represent a placeholder for different kinds of data collected by the Gateway. Time series is the only type available. Time Series data sets can then be used as part of [Anomaly Detection](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/gateway-hub/anomaly-detection/index.html) and [Breach predictor](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/configuration/geneos_gateway_plugins_tr/index.html#breach-predictor) functionality. ## Create a Time Series To create a Time Series: 1. Select **Data sets** from the navigation tree in the GSE. 2. Click **New Time series**. 3. Select the type of the time series. The type specifies if the time series is database driven or gathers data from Gateway Hub. 4. Provide all required information. See [Configuration reference](#configuration-reference) for more information about available options. 5. Validate and click **Save current document** ![](https://docscdn.itrsgroup.com/geneos/images/Icons/save16x16.png "icon") to save the changes. ## Gateway Hub driven Time series Geneos can use data sets generated by Gateway Hub. The specification for these data sets is set up using GSE, and the Gateway automatically manages their generation and retrieval. To use Anomaly Detection rules with data sets generated from Gateway Hub you should enable SSO authentication in Gateway Hub. For more information, see [Connecting to Gateway Hub](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/gateway-hub/gateway-hub-quick/index.html) The Gateway uses SSO tokens to access the data set endpoints in Gateway Hub. It is possible to connect to a Gateway Hub without authentication but this is insecure and should not be used in production environments. See [Unauthenticated usage with an insecure Gateway Hub in Centralised Gateways User Guide](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/gateway-hub/gateway-centralised-config/index.html#unauthenticated-usage-with-an-insecure-gateway-hub). To use this method, select **Gateway Hub driven** as **Type** when setting up time series. For more information, see [Type — Hub](#type--gateway-hub-driven). ## Database driven Time Series The time series can be set up by a process external to the Gateway and stored in two tables in the database used for database logging. The tables are: | Table name | Description | |-------|-------| | `time_series_user_table` | Stores a set of names and unique IDs. The names are used to map the names of the time series defined in the Gateway setup to the IDs used in the `time_series_data_user_table`. There are two values per row: - `name` — corresponds to the data set name defined in the Gateway and is unique. - `time_series_id` — a unique ID used in the `time_series_data_user_table`. | | `time_series_data_user_table` | Stores the time series data. There are three values per row: - `time_series_id` — ID used to link the data back to the name in the `time_series_data_user_table`. - `start_time` — start time for the `value` of a point in the time series data. Time in seconds since the start of the day. - `value` — value of the time series at `start_time`. | The schema for these tables is available in the Gateway resources directory provided as part of the Gateway bundle. The data is read from the database at Gateway start time and at the reload time defined in the setup. It is up to an external process to maintain and update the time series tables. This can be controlled using the Gateway scheduled command. To use this method, select **databaseDriven** as **Type** when setting up time series. For more information, see [Configuration reference.](#configuration-reference) ### Prerequisites Before you can configure the database driven time series, you need to configure the database tables: 1. Configure database logging. For more information, see [MySQL configuration in Gateway Database Logging](https://docs.itrsgroup.com/docs/geneos/7.11.0/processing/data-access-and-publishing/geneos_dblogging_tr/index.html#mysql-configuration). 2. Ensure that the tables defined in `/resources/database//time-series-schema-1.0.sql` exist. 3. Insert data into those tables. ## Configuration reference ![](https://docscdn.itrsgroup.com/geneos/images/gateway/gw-data-sets-time-series.png) | Setting | Description | Mandatory | XML schema path | |-------|-------|-------|-------| | Time series | Time series model a day's worth of data uploaded from the database. | No | timeSeries | | Name | Specifies a name that you want to identify each time series with. If your time series are database driven, the name must correspond to one of the database tables you have configured. | Yes | timeSeries > name | | Description | Specifies additional information about the time series. You can enter multi-line comments in the description field. | No | timeSeries > description | | External | Specifies how the external data access is managed. | | timeSeries > external | | External>Reload Time | Specifies the time of the day that data should be uploaded from the database or Gateway Hub every day. | No (default value is the current time during time series creation) | timeSeries > external > reloadTime | | Type | Specifies if the time series is database driven or gathers data from Gateway Hub. This setting has two options:- Database driven - Gateway Hub driven. See [Type Hub](#type--gateway-hub-driven) | No | timeSeries > type | ### Type — Gateway Hub driven This section provides more information about configuration options if you select your time series to be generated from Gateway Hub. #### Algorithm — Seasonal-quick You can specify the algorithm to use to generate the dataset. There is currently one algorithm available, **Seasonal-quick**:
Setting Description
Entity query

Specifies the entity query using the entities filter syntax. For example, user.COMPONENT=EMS. This query finds all entities with the user-defined managed entity attribute COMPONENT=EMS.

For more information on how to use the entity filter syntax.

Note: In GSE you should only use quotes. Do not escape the quotes with backslashes because the JSON formatter in Gateway does that for you.

Metrics

Data you want to query. This is a sequence of raw metric names to be included in the resulting metric time series.

Example: cpu/cpu/Average_cpu/percentUtilisation

Note: In GSE you should only use quotes. Do not escape the quotes with backslashes because the JSON formatter in Gateway does that for you.

Aggregations

Aggregations are calculations you wish to perform on the data. The options are:

  • count
  • sum
  • min
  • max
  • stddev
  • avg
  • var
  • percentile

For aggregations that can be set to a specified level (for example, percentile), you must also specify an alias to distinguish between multiple entries. Percentage levels are set between 0 and 100.

Aggregations are provided by Gateway Hub.

Note: The percentage aggregation available here uses the itrs.tdigest.percentile operation provided by the Gateway Hub API.

Granularity

Seasonal granularity setting is used to create Time Series buckets of selected granularity. It allows you to choose how long an interval each value in the Time Series represents.

You can choose from the drop-down menu, or enter a number followed by a length of time. The options in the drop-down menu are:

  • 1 minute
  • 5 minutes
  • 15 minutes
  • 1 hour
  • 3 hours
  • 12 hours
  • 1 day
Period

The period of cycle which defines seasonality. It allows you to choose the period over which you want the values to repeat.

The options are:

  • Day
  • Week
Periods Number of periods. It allows you to say how many recent periods (days or weeks) you want the data to be based on.
##### Period settings example Here's an example of how to use **period settings** (granularity, period, and periods): You want to write a rule which will compare the current value of an item with a typical value for this time of day, based on data from the last 60 days. - Granularity allows you to choose how long an interval each value in the time series represents. In this case, you should choose `5 minutes` or `15 minutes`. - Period allows you to choose the period over which you expect the values to repeat: does your time series represent a typical day or a typical week? In this use case, you want to choose a period of `day`. If the value you are monitoring varies a lot between working and non-working days, and you have enough historical data available, you should choose `week`. - Periods setting allows you to say how many recent days or weeks you want the data to be based on. In this use case, you want 60 days, so you should set this parameter to `60`. #### Time to live This specifies the length of time the time series is valid for. The options are: - `1 day` - `1 hour` - `1 week` - `3 hours` When the data is passed from Gateway to Gateway Hub, the default value is `1 day`. The data set is automatically deleted after the time to live value expires.