Plan the cluster
Overview
A Gateway Hub installation consists of a number of individual servers called nodes. A set of nodes collectively form a cluster.
The first step in deploying Gateway Hub is determining the number of clusters are required and where these should be physically located. For a global organisation, you should deploy one cluster in each geographical region (for example, Americas, EMEA, or APAC) to reduce network latency.
Next, you must plan the number of servers that are required to form the cluster. This is done based on the work that Gateway Hub is expected to perform and the specifications of the nodes. The hubsize tool provides an estimate of the hardware requirements for a specified workload. Different regions may have different workloads and therefore require different cluster sizes.
To ensure resilience, at least 3 nodes are required.
It is possible to install on a single node for testing or proof of concept development. However, in production systems a cluster is required for redundancy and scaling.
To determine whether an individual server is capable of contributing to the cluster, check Hardware requirements and Software requirements. One of the most common reasons for installation failure is the unsuitability of a node in the cluster.
Cluster sizing tool
You can use the hubsize tool to estimate the hardware requirements of your installation environment.
The hubsize tool can be downloaded separately from ITRS Downloads.
Prerequisites
The hubsize tool has the following dependencies:
| Requirement | Versions Supported |
|---|---|
| Python | 3.6 or newer |
| PyYAML | 5.3.1 or newer |
You can install all dependencies using the included requirements.txt file:
pip3 install --user --requirement requirements.txt
Define the requirements
The requirements of a Gateway Hub installation vary between use cases. The hubsize tool provides an estimate of the requirements of your installation based on the following criteria:
| Parameter | Description |
|---|---|
configuration > hardware > clusterSize
|
Number of nodes in the cluster. |
configuration > geneos > estimatedNrGeneosMessagesPerSec
|
Estimated rate of Geneos metric messages. Alternatively, specify the expected number of Netprobes. |
configuration > geneos > estimatedNrGeneosEventMessagesPerSec
|
Estimated rate of Geneos event messages. Alternatively, specify the expected number of Netprobes. |
configuration > geneos > numberOfGeneosProbes
|
Number of connected Netprobes. |
configuration > hub > metrics > retentionPeriodDays
|
Duration, in days, for which historical event and metric data is stored. |
configuration > hub > storageBuffer
|
Specify a buffer factor that hubsize will use to account for growth when performing calculations. |
configuration > hub > kafka > retentionPeriodHours
|
Duration, in hours, for which historical Kafka data is stored. You can alternatively specify the retention period in minutes using the retentionPeriodMinutes parameter or days using the retentionPeriodDays parameter. |
configuration > hub > kafka > replicationFactor
|
Number of Kafka replicas. |
You must specify these parameters in a definition.yml file that the hubsize tool will read. A human-readable example file is included/hubsize/definition.yml directory
Run the hubsize tool
To estimate the cluster requirements run:
./hubsize definition.yml
The hubsize tool has the following command line options:
| Option | Description |
|---|---|
--help
|
Print a help text to standard out. |
--version
|
Print the tool version to standard out. |
--out
|
Specify a file to write the estimated hardware requirements to. |
--metrics
|
Write the performance metrics used to a file. |