Connect Geneos to Obcerv

Overview Copied

Obcerv is an observability platform for data storage and analytics. You can configure Gateways and Netprobes to publish data to Obcerv so you can store and analyse metrics, logs and events in Obcerv.

Obcerv concepts Copied

Obcerv adopts a data model that includes many features that may sound familiar to Geneos users. However, it is important to recognise that these concepts are fundamentally different.

Entities Copied

Unlike the user-defined Managed Entities in Geneos, in Obcerv an Entity is automatically created based on the dimensions that describe the data points. See the Data model for more information.

Entities can also have associated attributes but this does not affect the definition of an Entity. Obcerv allows users to search for Entities metrics using both dimensions and attributes.

Geneos structures are mapped to Obcerv entities, attributes, and metrics.

Consider an example:

item	Entity dimensions	Attributes	Metrics
Managed Entity	probe=itrspc101	ConState=Up
Managed Entity	managedEntity=pc101	HostName=itrspc101.itsgroup.com
Managed Entity		COUNTRY=UK
Managed Entity		OS=Linux
Managed Entity		DEPT=SALES
Sampler	probe=itrspc101	Plugin Name=CPU
Sampler	managedEntity=pc101	Group Name=Infra
Sampler	sampler=CPU
Sampler	type=LinuxInfra
Dataview	probe=itrspc101		numCpuCores
Dataview	managedEntity=pc101		numOnlineCpus
Dataview	sampler=CPU		loadAverage1Min
Dataview	type=LinuxInfra		loadAverage5Min
Dataview	dataview=CPU
Row	probe=itrspc101		type
Row	managedEntity=pc101		clockSpeed
Row	sampler=CPU		percentUtilisation
Row	type=LinuxInfra		percentUserTime
Row	dataview=CPU		percentIdle
Row	row=cpu_0

In general, traditional data generates the following entities:

An entity to represent each Geneos Managed Entity.
An entity to represent each Geneos sampler.
An entity to represent each dataview (where headline metrics exist).
An entity to represent each row in a dataview (or column if the dataview is pivoted).

Cells in dataview rows (and headlines) are mapped to metrics that exist on these Obcerv entities according to the following rules:

If the cell is marked as having a numeric data type in its schema then its contents will be published to Obcerv as a GAUGE metric.
If the cell is marked as having a non-numeric data type in its schema then its contents will be published to Obcerv as a STATUS_METRIC.
If no schema has been defined, the Gateway will attempt to convert the cell to a GAUGE metric. If this succeeds the value will be published as a GAUGE, otherwise, it will be published as a STATUS_METRIC.

For data that is sent to Geneos by a Collection Agent, the type of the metric will be preserved. As Collection Agent v2 does not use STATUS_METRIC, non-numeric data from Collection Agent will be sent and published as ATTRIBUTES.

Obcerv closely follows the data model used by the Collection Agent. For more information, see Data collection in an orchestrated environment.

Data labels Copied

Gateway sends two types of labels that allow you to search through Obcerv entities:

Dimensions: these are the name-value pairs used to uniquely identify an Obcerv Entity.
Attributes: these are observed pieces of information about an Obcerv entity, such as the osType of a Geneos Managed Entity.

Every metric has a set of dimensions that uniquely identify it. These are derived from the Geneos hierarchy of gateway, managedEntity, sampler, type, dataview, and row.

Geneos Managed Entity attributes set in the Gateway Setup Editor are sent as ENTITY_ATTRIBUTE metrics on the corresponding Obcerv entities in the itrsgroup.com/geneos/gse namespace. Gateway also sends each sampler’s Group and PluginName as ENTITY_ATTRIBUTE metrics on the Obcerv entity representing that sampler.

Netprobe parameters, such as HostName, Port, ConState, OS, or Version, are sent as ENTITY_ATTRIBUTE metrics on each Obcerv entity representing a Geneos Managed Entity running on that Netprobe. The Netprobe also sends a sanitised version of OS called osType.

Signals Copied

Gateway sends severity changes to Obcerv as signals, and a signal stream is provided for every metric. Gateway sends the severity and the value on the cell at the time that the severity changed, and snooze events to Obcerv.

Audit Copied

Gateway sends audit events to Obcerv when a connection is established. Audit events originating from a Gateway will have a dimension of gateway. Audit events originating from a Netprobe (like running a command on a Netprobe) will have dimensions of both gateway and probe.

Publish from Gateway to Obcerv Copied

You can publish data from Gateway to Obcerv via the Ingestion service. This service is installed as part of your Obcerv installation.

By default, Gateway will publish all metrics, severity events, and audit events to Obcerv. You can configure filtering strategies to control what data is published to Obcerv. This can help minimise downstream data processing requirements in Obcerv. To configure include and exclude filters, see Strategies.

To start the connection between Gateway and Obcerv, you must configure the Obcerv connection in the Publishing settings in the Gateway Setup Editor. For more information, see Obcerv connection configuration.

Gateway self-monitoring, which is enabled by default, creates a dataview called Obcerv Connection that shows the current state of Obcerv publishing if the Obcerv Connection has been configured. For more information, see Self-monitoring.

Publish from Netprobes to Obcerv Copied

You can publish log data from a Netprobe to Obcerv via the Obcerv Ingestion service. Currently, the log data collected by the FKM plugin can be published to Obcerv. To do this, you must:

Configure the Obcerv connection in your Gateway.
Enable publishing for each FKM file in the Gateway Setup Editor. See FKM Obcerv publishing.

Obcerv publishing is enabled: FKM file publishes to Obcerv

Obcerv publishing is not enabled: FKM file does not publish to Obcerv

View Obcerv data in Active Console Copied

If the Gateways connected to your Active Console publish data to Obcerv, then those Gateways will share the Obcerv connection details with Active Console so you can create history charts by querying data in Obcerv.

To allow a connected Active Console to access Obcerv data, you must configure the Obcerv connection in the Data access settings in the Gateway Setup Editor.

In Active Console, you must also ensure that Tools > Settings > Geneos > Default data source is set to Obcerv.

Self-monitoring Copied

When Obcerv publishing is configured on a Gateway, self-monitoring statistics are reported in two dataviews under Gateway Info > Obcerv.

Detail dataview Copied

Name	Description
name	Name of the data type.
sendState	Either `SENDING` or `BUFFERING`.
deliveryStatus	Delivery status of the last message. This can be either `NONE`, `SUCCEEDED`, `RETRYING` or `FAILED`.
byteRate	Transfer rate in bytes per second.
messagingRate	Total transfer rate in messages per second across all three queues.
maxBufferSize	Maximum number of buffered messages.
messagesInBuffer	Current number of messages in the buffer.
messagesDroppedPerSample	Number of messages dropped in the last sampler.

Summary dataview Copied

Name	Description
enabled	The Obcerv connection can be either `Enabled` or `Disabled`
connectionStatus	Shows the connection status. This can be either `Statistics only`, `Idle`, `Ready`, `Connecting`, `Transient Failure`, or `Shutdown`.
messageRate	Transfer rate per queue in messages per second.
byteRate	Transfer rate in bytes per second.
messagesDroppedPerSample	Average number of messages dropped per sampler.
dataviewsPublished	Number of dataviews published.
dataviewsUnpublished	Number of dataviews unpublished.
dataviewsWithErrors	Number of dataview with errors.
accessTokenUser	Obcerv user associated with the access token used to connect to Obcerv. Note, this is not an application key. Access tokens are short lived tokens used to secure connections.
accessTokenAvailable	Status of access tokens.

Obcerv connection configuration Copied

Basic configuration Copied

The Obcerv connection section of the GSE provides the following options:

Setting	Description	Default
Enabled	Enables or disables publishing to Obcerv.	Enabled
Mode	Specify the publishing mode, choose from: connection — publish Gateway data to Obcerv. statisticsOnly — perform a dry run using the current publishing settings. Statistics are recorded to the self-monitoring dataviews and to log files.	connection

Setting

Description

Default

Enabled

Enables or disables publishing to Obcerv.

Enabled

Mode

Specify the publishing mode, choose from:

connection — publish Gateway data to Obcerv.
statisticsOnly — perform a dry run using the current publishing settings. Statistics are recorded to the self-monitoring dataviews and to log files.

connection

If the Mode is set to connection, then you can set up:

Publishing — configure publishing from this Gateway to Obcerv.
Data Access — allow an Active Console connected to this Gateway to access Obcerv data.

Section	Setting	Description
Connection	Verify server certificate	Enables or disables the server certificate verification. If this parameter is set to `false` and the TLS is enabled, then the server certificate will not be checked when the connection is made, and the provided certificate will be accepted. Default: True Note If the certificate supports Application-Layer Protocol Negotiation (ALPN), make sure that ALPN is set to `enabled` for Gateway and Obcerv connection to work properly.
Connection	Root certificates	Specify the root CA certificate used to sign the ingestion service certificate. You can provide: pemString — provide the full PEM string of the CA certificate. This can be found on the certificates page of the Obcerv Admin page in your Web Console. pemFile — provide the path to a file in the Gateway’s current working directory. This file should contain the full PEM string of the CA certificate. A root certificate is not required if the certificate is already trusted by the host.
Connection > Publishing	Service address	Specify the Obcerv ingestion service hostname with the `http://` prefix. For example: `https://ingestion.my-obcerv.com`. Note The service address field defaults to `https` and port `443`, so that `https://ingest.hub.local:443` can be entered as `ingest.hub.local`.
Connection > Publishing	Credentials	Specify the credentials used to access the Obcerv ingestion service. You should provide a username and password in the Gateway Setup Editor. For more information about password configuration, see Secure Passwords.
Connection > Data access	Service address	Specify the Obcerv Web Console hostname. For example: `https://my-obcerv.com`.
Connection > Data access	Credentials	Specify the Obcerv user credentials to access data from Obcerv. The Obcerv username must have or should be part of a user role. You should provide a username and password in the Gateway Setup Editor. For more information about password configuration, see Secure Passwords.

Advanced configuration Copied

The Additional settings allow you to specify the following additional publishing settings that you may set individually:

Settings	Description
grpc.rpc.timeout	The time the Gateway waits for a gRPC response. Default: `5000` Unit: milliseconds Mandatory: No
grpc.batch.timeout	The time the Gateway waits before sending a batch of datapoints to Obcerv. Default: `500` Unit: milliseconds Mandatory: No
grpc.batch.size	The maximum size a datapoint batch can have. Once the batch reaches the given size, it will be sent to Obcerv. Default: `1000` Unit: datapoints Mandatory: No
grpc.queue.size	The number of messages the Gateway can hold before it starts dropping message. This queue holds messages before being sent to Obcerv. The Gateway will attempt to keep this queue as small a possible. Default: `50000` Mandatory: No

Strategies Copied

Setting	Description	Default
Name	Specifies a name to uniquely identify the strategy.	New Strategy
Targets	One or more XPaths identifying the data items to which the strategy applies. These XPaths must point to one or more Netprobes, Managed Entities, samplers, or dataviews. XPaths that reference run-time values such as severity or connection state, or XPaths that point to individual cells are not supported.
Options	Specifies what type of strategy to use. The following options are available: include filter — publish only the data items specified by the targets, their ancestors, and their descendants. The metrics, severity messages, snooze messages, and user assignment messages of all data items not explicitly or implicitly targeted are not published. Severity is propagated through published data based only on included data items. exclude filter — do not publish the data items specified by the targets or their descendants. Metrics, severity messages, snooze messages, and user assignment messages are not published for any targeted data items. Severity is propagated through published data based only on data items which are not excluded. Note: If you specify both include filters and exclude filters, a data item is published if it is selected (directly or as an ancestor or descendant) by at least one include target and it is not selected (directly or as a descendant) by any exclude target.	filter

Strategy Group Copied

Strategies may be grouped for organisational purposes. This has no effect on their application but is useful for grouping categories and making them easier to find in the navigation tree.

You must specify a name when creating a strategy group.

Previous article Next article