Google Cloud Platform
Overview
This Collection Agent plugin monitors Google Services through the Google Cloud Platform metrics.
Metrics gathered depend on the services specified in the plugin configuration.
You can use the following collector in the Collection Agent YAML file:
GoogleCloudMonitoringCollector
- Monitors Google Cloud services. Supported services can be found at Monitored GCP Services.
Prerequisites
Geneos environment
The GCP Collection Agent plugin requires the following versions of Geneos components:
- Gateway and Netprobe 6.5.x or higher. The same version must be used for the GSE schema.
- Collection Agent 3.4.x or higher. To run a Collection Agent, see Collection Agent setup.
The GCP binaries are packaged with Netprobe, and are stored in the collection_agent folder. Alternatively, you can download separate binaries for the GCP plugin from ITRS Downloads.
GCP environment
The GCP plugin requires valid Google credentials to use.
Google application default credentials searches for credentials in the following locations:
- Google Application Credentials environment variable
- User credentials set up by using the Google Cloud CLI
- The attached service account, returned by the metadata server
The order of the locations ADC checks for credentials is not related to the relative merit of each location.
Google Application Credentials environment variable
You can use the GOOGLE_APPLICATION_CREDENTIALS
environment variable to provide the location of a credential JSON file. This JSON file can be one of the following types of files:
-
A credential configuration file for workload identity federation. Workload identity federation enables you to use an external identity provider to access Google Cloud resources. For more information, see Authenticating by using client libraries, the gcloud CLI, or Terraform in the Identity and Access Management (IAM) documentation.
-
A service account key. Service account keys create a security risk and are not recommended. Unlike the other credential file types, compromised service account keys can be used by a bad actor without any additional information. For more information, see Best practices for using and managing service account keys.
User credentials provided by using the gcloud CLI
You can provide user credentials to ADC by running the gcloud auth application-default login
command. This command places a JSON file containing the credentials you provide (usually from your own Google Account) in a well-known location on your file system. The location depends on your operating system:
- Linux, macOS:
$HOME/.config/gcloud/application_default_credentials.json
- Windows:
%APPDATA%\gcloud\application_default_credentials.json
The credentials you provide to ADC by using the gcloud CLI are distinct from your gcloud credentials—the credentials the gcloud CLI uses to authenticate to Google Cloud.
The attached service account
Many Google Cloud services let you attach a service account that can be used to provide credentials for accessing Google Cloud APIs. If ADC does not find credentials it can use in either the GOOGLE_APPLICATION_CREDENTIALS environment variable or the well-known location for Google Account credentials, it uses the metadata server to get credentials for the service where the code is running.
Using the credentials from the attached service account is the preferred method for finding credentials in a production environment on Google Cloud. To use the attached service account, follow these steps:
- Create a user-managed service account.
- Grant that service account the least privileged IAM roles possible.
- Attach the service account to the resource where your code is running.
Please refer to Google Application Default Credentials for more information.
Required Google Cloud Permissions
For all services, the following monitoring permissions are needed to gather metrics from Cloud Monitoring:
monitoring.metricDescriptors.list
monitoring.monitoredResourceDescriptors.list
monitoring.timeSeries.list
Some services require additional permissions to get resource information (e.g. user labels):
- compute
compute.disks.list
compute.instances.list
- file
file.instances.list
- storage
storage.buckets.list
- vpn
compute.vpnGateways.list
Configure Geneos to deploy the GCP plugin
The GCP plugin supports Collection Agent publication into Geneos using dynamic Managed Entities. Setting up this plugin in Geneos involves these primary steps:
-
Configure your mappings.
-
Configure your other Dynamic Entities in the Gateway, see Create Dynamic Entities in Collection Agent setup for a more detailed procedure.
Set up your Collection Agent plugin
Set up your collector in the Gateway Setup Editor by adding the following configuration in Dynamic Entities > Collectors. For more information, see Collectors in Dynamic Entities.
Below are the available collectors for the GCP plugin:
Collectors | Description |
---|---|
GoogleCloudMonitoringCollector | Enables the GCP collector. By default, all supported services are monitored. If you want to monitor specific services, use the enabled-services: option |
Configure your GCP plugin using the following reference configuration:
collectors:
# GCP collector configuration
- name: gcp
type: plugin
class-name: GoogleCloudMonitoringCollector
# Interval (in millis) between collections (optional, defaults to five minutes).
collection-interval: 300000
# Project id to monitor (required)
project-id: itrsdev
# Google cloud service metrics to be gathered (optional, default to all supported services)
#enabled-services:
# - compute
# Plugin self monitoring (optional, disabled by default)
#self-monitoring:
# Collection interval of the self-monitoring metrics (required if self-monitoring is used)
# interval: 300000
# Enable detailed self-monitoring metrics reporting (optional, false by default)
# Two levels of monitoring are :
# - Detailed (true): API Call counts are reported per monitored service, per resource, per specific api
# - Detailed (false): API Call counts are reported total per moonitored service
# detailed-mode: false
# Google cloud service launch stage configuration (optional, default is GA)
#enabled-stages:
# - ALPHA
The plugin contains an optional self-monitoring-interval
configuration to enable or disable self monitoring, which counts the number of Google APIs called. This feature is disabled by default and can be enabled by setting the interval value in seconds.
You can also set the interval
value to determine how often metrics are published.
By default, a summary view where only the total calls per service are published by the plugin. If detailed-mode
is true
, the plugin publishes a summary view with more detailed metrics by breaking down the API calls per service, per resource, and per API called.
Optional enabled-stages
configuration to include specific metric launch stages is also available.
Configure your mappings
Use one of the following options listed below to configure your dynamic mappings.
-
Adding
templates/gcp_mapping.xml
as an include file in your Gateway. -
Setting up a custom mapping in Dynamic Entities > Mapping. For more information, see Mapping and mapping group in Dynamic Entities.
Example Gateway-SQL Summary view
The template also includes a Gateway-SQL sampler named GCP
that shows a summary view of your GCP project.
This view provides information on all the instances, disks, and networks under the monitored project.