Google Stackdriver Integration User Guide

Overview

Google Stackdriver integration enables you to retrieve all metrics and metadata available in your Stackdriver setup and display them as dataviews in Geneos.

The integration provides a Gateway configuration file that enables monitoring of Stackdriver through a set of pre-configured Toolkit samplers. You can query and list the available metrics for your Google Cloud environment and running instances.

This guide walks you through the integration process to get the Toolkit sampler running with default Stackdriver views.

Audience

This user guide is intended for experienced Geneos users who want to set up and configure an integration between Geneos and Stackdriver.

As a user, you should be familiar with Stackdriver as well as the Google Cloud Platform (GCP) services.

Prerequisites

The following requirements must be met prior to the installation and setup of the template:

Python 3.6 or higher.
Python package manager (pip).
Google API Client for Python.
Google Authentication Library.
Google Cloud Platform service account key in a form of a JSON file.
This requires an additional licence to use. Please contact your ITRS Account Manager for more information.

Installation procedure

The Stackdriver integration is largely pre-configured and involves three primary steps:

Download the integration files from the resources site.
Load the integration file in the Gateway Setup Editor.
Configure the environment.
Associate the samplers with a managed entity.

Note: The include files are intended to be read-only and should not be edited (unless the integration specifically requires for them to be modified). This is because include files can be shared between a few instances of a Gateway and best practice is to share them in their default state. It is also to make the upgrades and maintenance process of the integration files easier. All configuration should be carried out in your local Gateway files instead.

Download the integration files

Download the integration package geneos-integration-gcp-<version>.zip from the ITRS Downloads site.
As a best practice, save the files on the same machine where your Gateway is running.

The integration package includes a number of files. These are the files that determine the integration setup:

include/gcpMonitoring.xml — Gateway configuration file (also called an include file) containing Toolkit samplers configured for Stackdriver integration.
scripts/list_instances.py — Python script that calls Stackdriver API and return a JSON format. The JSON values are then processed in a CSV format and displayed in Active Console. This displays available metrics for your services.
scripts/requirements.txt — list of the required Google libraries. They can be installed by running the pip command against this requirements file:
```
$ pip install -r requirements.txt
```

Load the integration files

To load the integration files into the Gateway Setup Editor, follow these steps:

Open Gateway Setup Editor.
In the Navigation panel, click Includes to create a new file.
Enter the location of the file to include in the Location field. In this example, it is the include/gcpMonitoring.xml.
Update the Priority field. This can be any value except 1. If you input a priority of 1, the Gateway Setup Editor will return an error.
Expand the file location in the Include section.
Select Click to load.
Click Yes to load the new include file.

Success: Gateway configuration now appears in the Includes section.

For more information on the dataviews associated with the Stackdriver integration, see the Google Stackdriver Integration Technical Reference.

Configure the environment

The default environment configuration is created by the gcpMonitoring.xml script. You should create a new environment in your main Gateway setup based on the default one.

In the Gateway Setup Editor, locate your Gateway under the Includes section.
Copy the default GCP environment.
In the Gateway Setup Editor, create a new environment by right-clicking the Environment folder. Paste the copied environment.
Rename the new environment to distinguish it from the default one. Example: GCP_env
Modify the environment variables according to your system.

Note: Standard variable scoping rules apply when variable reference is used in include files. Please take note of that when creating your own variables. For more information, see Variable Scoping and Resolution in User Variables and Environments.

For more information on the environment variables used in this integration, see the Environment variables in Google Stackdriver Integration Technical Reference.

Associate the samplers with a managed entity

After you have loaded the configuration file, the Stackdriver monitoring samplers become available for use. These samplers are associated with a pre-defined Type. To associate the samplers with a managed entity, follow these steps:

In the Gateway Setup Editor, create a new managed entity by right-clicking the Managed entities folder.
Under the Basic tab for the new managed entity, add a pre-configured GCP type.
Specify the environment associated with the samplers. This is the new environment that you created in your main Gateway setup. Example: GCP_env.
Apply the managed entity to the running netprobe.
Under the Basic tab for the new managed entity, add a pre-configured GCP type.
Apply the managed entity to the running Netprobe.
Click Validate current document to check your configuration.
Click Save current document to apply the changes.

Success: After you save your changes to the managed entity, the Google Stackdriver dataviews appear under the managed entity on the state tree.

For more information about include files, see the 'Include' files in Gateway Setup Editor.

Dataviews configuration

This integration provides a number of default dataviews, but you can add more based on which metrics and services your Stackdriver is monitoring. The default StackDriver-Monitoring dataview lists all the available metrics and their respective key paths. These key paths are used to configure the aggregated mode and detail views.

Configure a sampler with a single metric

To configure a sampler with a particular Stackdriver metric, use a GOOGLE_DRILL_METRIC environment variable. The value of that variable is a key path of the metric you want to see the details of. Key paths are displayed by the StackDriver-Monitoring dataview.

Configure a sampler with multiple metrics

To configure a sampler with a set of Stackdriver metrics, use a GOOGLE_DRILL_VIEW environment variable. The value of that variable is a sub-string of the target key paths. Key paths are displayed by the StackDriver-Monitoring dataview.

For example, to display a set of metrics referring to CPU, you can use the compute.googleapis.com/instance/cpu sub-string and it will cross reference the following metrics:

compute.googleapis.com/instance/cpu/reserved_cores
compute.googleapis.com/instance/cpu/usage_time
compute.googleapis.com/instance/cpu/utilization