Plan the cluster

Overview

A Gateway Hub installation consists of a number of individual servers called nodes. A set of nodes collectively form a cluster.

The first step in deploying Gateway Hub is determining the number of clusters are required and where these should be physically located. For a global organisation, you should deploy one cluster in each geographical region (for example, Americas, EMEA, or APAC) to reduce network latency.

Next, you must plan the number of servers that are required to form the cluster. This is done based on the work that Gateway Hub is expected to perform and the specifications of the nodes. The hubsize tool provides an estimate of the hardware requirements for a specified workload. Different regions may have different workloads and therefore require different cluster sizes.

To ensure resilience, at least 3 nodes are required.

It is possible to install on a single node for testing or proof of concept development. However, in production systems a cluster is required for redundancy and scaling.

To determine whether an individual server is capable of contributing to the cluster, check Hardware requirements and Software requirements. One of the most common reasons for installation failure is the unsuitability of a node in the cluster.

Cluster sizing tool

You can use the hubsize tool to estimate the hardware requirements of your installation environment.

The hubsize tool can be downloaded separately from ITRS Downloads.

Prerequisites

The hubsize tool has the following dependencies:

Requirement	Versions Supported
Python	3.6 or newer
PyYAML	5.3.1 or newer

You can install all dependencies using the included requirements.txt file:

pip3 install --user --requirement requirements.txt

Define the requirements

The requirements of a Gateway Hub installation vary between use cases. The hubsize tool provides an estimate of the requirements of your installation based on the following criteria:

Parameter	Description
`configuration > hardware > clusterSize`	Number of nodes in the cluster.
`configuration > geneos > estimatedNrGeneosMessagesPerSec`	Estimated rate of Geneos metric messages. Alternatively, specify the expected number of Netprobes.
`configuration > geneos > estimatedNrGeneosEventMessagesPerSec`	Estimated rate of Geneos event messages. Alternatively, specify the expected number of Netprobes.
`configuration > geneos > numberOfGeneosProbes`	Number of connected Netprobes. Specifying message estimates provides more accurate results.
`configuration > hub > metrics > retentionPeriodDays`	Duration, in days, for which historical event and metric data is stored.
`configuration > hub > storageBuffer`	Specify a buffer factor that `hubsize` will use to account for growth when performing calculations.
`configuration > hub > kafka > retentionPeriodHours`	Duration, in hours, for which historical Kafka data is stored. You can alternatively specify the retention period in minutes using the `retentionPeriodMinutes` parameter or days using the `retentionPeriodDays` parameter.
`configuration > hub > kafka > replicationFactor`	Number of Kafka replicas.

You must specify these parameters in a definition.yml file that the hubsize tool will read. A human-readable example file is included in the /hubsize/definition.yml directory.

Run the hubsize tool

To estimate the cluster requirements run:

./hubsize definition.yml

The hubsize tool has the following command line options:

Option	Description
`--help`	Print a help text to standard out.
`--version`	Print the tool version to standard out.
`--out`	Specify a file to write the estimated hardware requirements to.
`--metrics`	Write the performance metrics used to a file.