The end of life (EOL) date for this module is on 31 January, 2020.

Gateway Distribution

Introduction

Gateways connections are dynamically divided amongst the nodes in the cluster. This page describes how the gateways are distributed.

To change the algorithm used, edit application.conf and set geneos.gateway-assigner to one of:

  • geneos.consistent-hash-assigner (default)
  • geneos.divide-by-nodes-assigner
  • geneos.round-robin-assigner

Warning

It is important that all the nodes have the same algorithm configured, otherwise they will disagree about which gateway will be assigned to which node!

The various algorithms have default configurations defined in application.conf:

geneos {
    ...
    gateway-assigner: geneos.consistent-hash-assigner
    ...
    divide-by-nodes-assigner {
        type: DivideByNodes
    }

    round-robin-assigner {
        type: RoundRobin
    }

    consistent-hash-assigner {
        type: ConsistentHash
        virtual-nodes-factor: 10
    }
}

To change the algorithm used simply change geneos.gateway-assigner. For instance, to set it to the round robin assigner:

geneos {
    ...
    gateway-assigner: geneos.round-robin-assigner
    ...
}

Distribution Algorithms

All the algorithms key the nodes using their full cluster addresses, which take the form:

akka.tcp://ClusterSystem@<host_or_ip>:<port>

So if a node is on host nodes.datacenter.com port 2551, its key in the distribution algorithm would be:

akka.tcp://ClusterSystem@nodes.datacenter.com:2551

Gateways are assigned using the host and port of the primary gateway, so given gateways configured like this:

gateways {
  Demo Gateway {
    host = "uk1.gateways.datacenter.com"    port = 7039
  }
  HotStandby Gateway {
    primary {
      host = "us1.gateways.datacenter.com"      port = 7039
    }
    secondary {
      host = "us2.gateways.datacenter.com"      port = 7039
    }

}

This would yield 2 gateways for assignment:

uk1.gateways.datacenter.com:7039
us1.gateways.datacenter.com:7039

Each node will update its gateway assignments when one of the following events occurs

  • The cluster changes: a node is added or removed intentionally or because of failure.
  • The gateways are changed in settings.conf: a gateway is either added or removed.

When either of these events occur the node will re-calculate the gateway assignments and add or remove gateways as appropriate. In some cases this means that gateways effectively move from one node to another, which can be a costly exercise.

Consistent Hashing

Consistent Hashing is the default distribution algorithm and the only one prior to 2.0. This article gives good insight into how consistent hashing is implemented.

The keys for the hash are the node addresses.

AdvantagesDisadvantages
  • Minimises the amount of gateway ‘moves’ when a node is added or removed.
  • Adding gateways is guaranteed not to ‘move’ any existing gateways.
  • You cannot easily determine which gateways will be assigned to which nodes (See Assignment Checker Utility).
  • When using small numbers of nodes/gateways, the distribution may not be very uniform.

Round Robin

Note: This was added to version 2.0 and is not available in earlier versions.

Nodes and gateways are sorted by their addresses, then each gateway is round robin assigned to each node in turn.

AdvantagesDisadvantages
  • Easy to predict how gateways will be assigned
  • Gateways will uniformly assigned around the cluster
  • Changes to the cluster or gateway list will result in more gateway moves than consistent hashing

Divide-By-Node

Note: This was added to version 2.0 and is not available in earlier versions.

Nodes and gateways are sorted by their addresses, then the gateways are divided by the nodes, the last node taking the remainder.

AdvantagesDisadvantages
  • Easy to predict how gateways will be assigned
  • Apart from the last node, gateways will uniformly assigned around the cluster
  • Changes to the cluster or gateway list will result in more gateway moves than consistent hashing

Assignment Checker Utility

AssignmentChecker is utility program for checking how gateways will be assigned in an Open Access cluster.

From the node directory, run:

> java -cp "lib/*" com.itrsgroup.oanode.assigner.AssignmentChecker

You should see some output like this:

2015-03-04 15:57:38,069  INFO - AssignmentChecker$ - <node_path>/assignmentchecker.conf does not exist, creating it...
2015-03-04 15:57:38,397  INFO - ClusterGatewayAssignerFactory$ - Using consistent hash gateway assigner
2015-03-04 15:57:38,401  INFO - AssignmentChecker - Assigning using class com.itrsgroup.oanode.assigner.ConsistentHashAssigner
2015-03-04 15:57:38,622  INFO - AssignmentChecker -
akka.tcp://ClusterSystem@nodes.datacenter.com:2551 (1)
    gateways.datacenter.com:7039
akka.tcp://ClusterSystem@nodes.datacenter.com:2552 (1)
    gateways.datacenter.com:7040
akka.tcp://ClusterSystem@nodes.datacenter.com:2553 (0)

The first time you run it, it will create a file called assignmentchecker.conf. You can now edit the file; each time you save it the checker will output the new distribution.

assignmentchecker.conf has 3 sections:

  • assigner: this is the the same as the node in application.conf described above
  • nodes: this is a list of node addresses, the members of the cluster
  • gateways: this is the same format as settings.conf (See Connecting to a Gateway)

Note: The precise hosts/IPs and ports of the nodes and gateways will affect the assignment algorithms. You should use the exact same hostnames/IPs/Ports as your production system to get accurate results.

Example assignmentchecker.conf:

assignmentchecker {
  assigner {
    type: ConsistentHash // Or DivideByNodes or RoundRobin
    virtual-nodes-factor: 10
  }

  nodes = [
    "nodes.datacenter.com:2551"    "nodes.datacenter.com:2552"    "nodes.datacenter.com:2553"  ]

  gateways {
    gateway1 {
      host = "gateways.datacenter.com"      port = 7039
    }
    gateway2 {
      host = "gateways.datacenter.com"      port = 7040
    }
  }
}