--- title: "AWS" description: "Enhance your cloud monitoring with the AWS plugin for Geneos, enabling seamless integration with AWS CloudWatch for comprehensive metric tracking and alerting." url: "https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws/index.html" --- ## Overview The AWS plugin is a Collection Agent plugin that gathers metrics through AWS CloudWatch. This plugin also provides an API Destination that can interact with AWS services, such as EventBridge and SNS. The AWS plugin improves Geneos cloud monitoring capabilities by building a more easy-to-use and scalable solution to interface with AWS CloudWatch to monitor various services being deployed in AWS. In addition, the AWS plugin allows you to: - Combine metrics from AWS with on-premise, multi-cloud, and hybrid environments for end-to-end visibility. - Monitor real-time alerts using rule and alert capabilities. - Minimize cost by using a single enterprise tool for monitoring. ### Monitored services, logs, and events You can use the AWS plugin to monitor different services by using the following collectors in the Collection Agent YAML file: - `AwsCollector`— for the list of collected services, see [AWS Plugin services](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws-services/index.html#overview). - `AwsBillingCollector` — for billing services, providing a breakdown of estimated charges by service. This includes estimated charges from CloudWatch, budget metrics from AWS Budgets, and cost/usage metrics from AWS Cost Explorer. See [AWS Billing collector](#aws-billing-collector). - `ApiDestinationCollector` — for receiving the following real-time data through its API Destination collector for Amazon Events, Logs, and Alarms services. Logs and events that are reported to Netprobe will result in stream messages that can be monitored by the FKM plugin of Netprobe. See [AWS API destination collector](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws/index.html#aws-api-destination-collector). - `AwsSdkUsageMetricsCollector` — for publishing the aggregated SDK Usage metrics from the last 5-minute window. See [AWS SDK Usage metrics collector](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws/index.html#aws-sdk-usage-metrics-collector). - `AwsCustomNamespaceCollector` — for collection and publishing custom metrics published into custom namespaces created in AWS CloudWatch. - `AwsMetricStreamCollector` — for exposing an HTTPS endpoint to connect to Data Firehose, wherein data received is processed to provide similar datapoints as AwsCollector. - `AwsCloudwatchLogsCollector` — for directly monitoring CloudWatch logs. See [AWS CloudWatch Logs collector](#aws-cloudwatch-logs-collector). - `AwsRedShiftDataCollector` — for directly querying Redshift data. See [AWS Redshift Data collector](#aws-redshift-data-collector). ### Deployment recommendations #### Launch ITRS Geneos EC2 instance in AWS Marketplace Use the [Plugin in AWS Marketplace](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws-marketplace/index.html) deployment option for the following reasons: - To deploy the AWS plugin through the AWS Marketplace with built-in dynamic entity mapping to minimize Gateway configuration. - To deploy the AWS plugin through the marketplace with minimal configuration. #### Configure Geneos to deploy the AWS plugin Use the [Configure Geneos to deploy AWS CloudWatch](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws/index.html#configure-geneos-to-deploy-the-aws-plugin-1) deployment option in the following cases: - When you require a native deployment option where you configure the Gateway and Netprobe in Geneos on your local machine. ## Prerequisites ### Geneos environment The AWS Collection Agent plugin requires the following versions of Geneos components: - Gateway and Netprobe 7.1.x or higher. The same version must be used for the GSE schema. - Collection Agent 5.x or higher. To run a Collection Agent, see [Collection Agent setup](https://docs.itrsgroup.com/docs/geneos/current/collection/collection-agent/collection-agent-setup/index.html). The AWS binaries are packaged with Netprobe, and are stored in the `collection_agent` folder. Alternatively, you can download separate binaries for the AWS plugin from the [ITRS Downloads](https://resources.itrsgroup.com/downloads). **Caution** Collection Agent and its plugins is no longer packaged with the Netprobe in Geneos 5.14.7 and the subsequent 5.x versions. If you want to run Collection Agent via Netprobe, please upgrade to the current version of Geneos. ### AWS environment The AWS plugin requires valid AWS credentials to use, such as an Access Key ID and a Secret Access Key which can be stored through environment variables or in the `~/.aws/credentials` file on your machine. For example: ```credentials [default] aws_access_key_id = aws_secret_access_key = ``` Please refer to the [Setting the default credentials](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/setup.html#setup-credentials) page for how to specify your AWS credentials on your machine. To see the required permissions for some of the monitored services, see [Required AWS plugin permissions in Plugin services](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws-services/index.html#required-aws-plugin-permissions). #### Role assumption The AWS plugin supports role assumption following standard AWS configuration methods. This allows you to assume different IAM roles to access resources across multiple AWS accounts or resources restricted to specific roles, which is especially useful in enterprise monitoring setups. When assuming a role, the original user's permissions are replaced entirely by those of the assumed role. This provides a secure access model by limiting actions strictly to what the assumed role allows. Before configuring role assumption, ensure the following: - The original user must have the `sts:AssumeRole` permission for the target role. - The target role must have a trust policy allowing the original user to assume the role. To configure role assumption: 1. Create an AWS profile in the `~/.aws/config` file that specifies the role to assume. For example, this configuration allows the profile `demo-role` to assume the role `DemoAssumeRole`. ```config [profile demo-role] region = ap-southeast-1 output = json role_arn = arn:aws:iam:::role/DemoAssumeRole source_profile = default ``` **Note** When using profile-based role assumption, ensure that the environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are not set, as these will override profile configuration. 2. Set the `AWS_PROFILE` environment variable to use the profile created. On Linux or macOS: ```command export AWS_PROFILE=demo-role ``` On Windows: ```command set AWS_PROFILE demo-role ``` 3. The AWS SDK will automatically handle the role assumption process. If the AWS CLI is installed within the container where the AWS plugin is running, you can verify that the role assumption is successful by running the command: ```command aws sts get-caller-identity ``` The caller identity must display the ARN of the assumed role instead of the ARN of the original user. For more information on AWS SDK credentials provider, refer to [AWS credentials provider](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials-chain.html). #### CloudWatch API usage Since the AWS plugin interacts with AWS CloudWatch using Amazon-provided APIs, you should be aware of [CloudWatch services quotas](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html). Otherwise, you might encounter an error similar to this: `2021-12-02 14:05:11.833 [EC2Service-Processor-0] ERROR com.itrsgroup.collection.plugins.aws.AwsCollector(awsSG) - CloudwatchMetricDataSource Get Metrics Error: Rate exceeded (Service: CloudWatch, Status Code: 400, Request ID: 31d9a57e-c6c5-46fd-94da-7fe62e74010c, Extended Request ID: null)` **Caution** There are cost implications when monitoring a large estate. Since the AWS plugin retrieves data using the `cloudwatch:GetMetricData` API, be aware of the associated costs per 1,000 metrics requested. See [Amazon CloudWatch Pricing](https://aws.amazon.com/cloudwatch/pricing/) for more information. To minimize costs, reduce the number of queries sent to CloudWatch by decreasing the number of monitored services or increasing the collection interval of the plugin. #### CloudWatch query time windows The values obtained from AWS are the values averaged over the last complete 5-minute window. This is because Cloudwatch makes the data available with a 5-minute latency. For example, if the collection time is `2021-07-04T03:31:12.34Z`, then the time window used to query the data in CloudWatch is: - `StartTime`: `2021-07-04T03:25:00.00Z` - `EndTime`: `2021-07-04T03:30:00.00Z` In the case where the collection interval is less than `5` minutes, the AWS plugin will query CloudWatch using the adjusted time windows at first. Then for the succeeding samples where the adjusted time window is the same as the previous window, no queries will be done so the plugin generates no data. For example, when the `collectionInterval` is set to `1` minute, the AWS plugin will query CloudWatch at first, then until it hits the next complete 5-minute window (for example, after `5` minutes), the plugin will not return any data. ## Configure Geneos to deploy the AWS plugin The AWS plugin supports Collection Agent publication into Geneos using dynamic Managed Entities. Setting up this plugin in Geneos involves these primary steps: 1. [Set up your Collection Agent plugin](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws/index.html#set-up-your-collection-agent-plugin). 2. [Configure your mappings](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws/index.html#configure-your-mappings). 3. Configure your other Dynamic Entities in the Gateway, see [Create Dynamic Entities in Collection Agent setup](https://docs.itrsgroup.com/docs/geneos/current/processing/configuration/dynamic_entities/index.html#create-dynamic-entities) for a more detailed procedure. ### Set up your Collection Agent plugin Use one of the following options listed below to configure the plugin. - Setting up your collector in the Gateway Setup Editor by adding the following configuration in **Dynamic Entities** > **Collectors**. For more information, see [Collectors in Dynamic Entities](https://docs.itrsgroup.com/docs/geneos/current/processing/configuration/dynamic_entities/index.html#collectors). Below are the available collectors for the AWS plugin: | Collectors | Description | |-------|-------| | `AwsCollector` | Enables AWS collector configuration. To add more AWS services to monitor, you can add them in `enabledServices` following this format: ``/``. Without this setting, the AWS plugin will pick up metrics for all available AWS services. See [Monitored AWS Services](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/current/user-guide/aws-services/index.html#monitored-aws-services). | | `AwsBillingCollector` | Enables AWS Billing collector configuration to monitor billing services, including estimated charges from CloudWatch, budget metrics from AWS Budgets, and cost/usage metrics from AWS Cost Explorer. See [AWS Billing collector](#aws-billing-collector). | | `ApiDestinationCollector` | Enables AWS API destination configuration to monitor the following AWS logs and events:- [EventBridge event and CloudWatch log streaming](#eventbridge-event-and-cloudwatch-log-streaming) - [Alarm Notifications](#alarm-notifications) | | `AwsSdkUsageMetricsCollector` | Publishes the aggregated SDK Usage metrics from the last 5-minute window. See [AWS SDK Usage metrics collector](#aws-sdk-usage-metrics-collector) . | | `AwsCustomNamespaceCollector` | Collects and publishes custom metrics published into custom namespaces created in AWS CloudWatch. See [AWS Custom namespace collector](#aws-custom-namespace-collector). | | `AwsMetricStreamCollector` | Exposes an HTTPS endpoint to connect to Data Firehose and the received data is processed to provide similar datapoints as AwsCollector. See [CloudWatch Metric Streaming](#cloudwatch-metric-streaming). | | `AwsCloudwatchLogsCollector` | Monitors CloudWatch logs directly by starting a live tail session on the selected log groups and log streams, filtering based on the pattern provided. See [AWS CloudWatch Logs collector](#aws-cloudwatch-logs-collector). | | `AwsRedShiftDataCollector` | Queries Redshift data directly. See [AWS Redshift Data collector](#aws-redshift-data-collector). | - Adding the following configuration in `collection-agent.yml` file on your local machine. ```yaml collectors: # AWS collector configuration - name: aws type: plugin className: AwsCollector # Interval (in millis) between collections (optional, defaults to five minutes). collectionInterval: 300000 # AWS regions from which metrics will be collected regions: - ap-southeast-1 - ap-southeast-2 # List of services to collect metrics from (optional, case-sensitive, and defaults to all metrics) enabledServices: - AWS/EC2 - AWS/EBS - AWS/EKS - AWS/RDS - AWS/ECS # Publish SDK usage metrics sdkMetrics: true # FOR ENGINEERING USE ONLY - Override AWS to connect to the endpoint of your choice (optional). awsUrl: "http://localhost:4566" # AWS custom metric collector configuration - name: awscustom type: plugin className: AwsCustomNamespaceCollector # Interval (in millis) between collections (optional, defaults to five minutes). collectionInterval: 300000 # AWS regions from which metrics will be collected regions: - ap-southeast-1 - ap-southeast-2 # Publish SDK usage metrics sdkMetrics: true # FOR ENGINEERING USE ONLY - Override AWS to connect to the endpoint of your choice (optional). awsUrl: "http://localhost:4566" # List of custom namespaces to collect metrics from customMonitoredNamespaces: - namespace: ECS/ContainerInsights # List of metrics to collect from the custom namespace (optional, defaults to collect all available metrics if not specified) customMonitoredMetrics: # Regex pattern to match the metrics to collect from the namespace - nameIncludes: Network* # Length of time used to aggregate the metric (optional, defaults to 300 seconds) period: 300 # Aggregation to be used (optional, defaults to "average") statistic: average # AWS Billing collector configuration - name: aws-billing type: plugin className: AwsBillingCollector # Interval (in millis) between collections (optional, defaults to 6 hours). collectionInterval: 21600000 # Publish SDK usage metrics sdkMetrics: true # AWS SDK Metrics collector configuration # Publishes the aggregated SDK Usage Metrics from the last 5-minute window - name: aws-sdk-metrics type: plugin className: AwsSdkUsageMetricsCollector # Interval (in millis) between collections (optional, defaults to five minutes). collectionInterval: 300000 # AWS API destination configuration - name: apidestination type: plugin className: ApiDestinationCollector # AWS region from which alarm notifications are expected. snsRegion: ap-southeast-1 # Port on which to receive API destination events port: ${env:API_DESTINATION_PORT} # Acceptor thread pool size (optional, defaults to 2) acceptorThreadPoolSize: 2 # Worker thread pool size (optional, defaults to 4) workerThreadPoolSize: 4 # TLS configuration (TLS is required by AWS API Destination to be a valid endpoint) tlsConfig: certFile: ${env:CERT_FILE} keyFile: ${env:KEY_FILE} trustChainFile: ${env:TRUST_CHAIN_FILE} # Authentication type (at the moment, only basic authentication for EventBridge is supported) authentication: # The basic authentication credentials here should match the ones set in EventBridge basicAuthentication: username: ${env:BASIC_AUTH_USERNAME} password: ${env:BASIC_AUTH_PASSWORD} # FOR ENGINEERING USE ONLY - Override AWS to connect to the endpoint of your choice (optional). awsUrl: "http://localhost:4566" # FOR ENGINEERING USE ONLY # This option specifies whether the signatures of SNS messages will be verified first. # Verifying SNS message signatures ensures that the message was sent from Amazon SNS. # Set this option to false in order to handle SNS messages that are not from Amazon SNS. # (optional, defaults to true) verifySnsSignature: true # AWS Metric Stream configuration - name: metricstream type: plugin className: AwsMetricStreamCollector # TLS configuration (TLS is required by AWS Metric Stream to be a valid endpoint) tlsConfig: certFile: ${env:CERT_FILE} keyFile: ${env:KEY_FILE} # Metric stream format (optional, defaults to otel-10) # Options are case-insensitive: json, otel-07, and otel-10 metricFormat: json # Statistics configuration for each metric (optional, default uses internal table statistics from aws-collector) # This configuration can overwrite or add to the default internal table of statistics # Options are case-insensitive: average, sum, minimum, and maximum # statistics: # AWS/EC2: # CPUUtilization: Sum # AWS Cloudwatch Log Collector configuration - name: cloudwatchlogs type: plugin className: AwsCloudwatchLogsCollector # AWS regions that contains the logs to monitor (required) regions: - ap-southeast-1 - ap-southeast-2 # List of log groups to monitor (required) logGroups: - arn:aws:logs:us-east-2:164181677543:log-group:airflow-mwaa-environment-public-network-MwaaEnvironment-WebServer - arn:aws:logs:us-east-2:164181677543:log-group:airflow-mwaa-environment-public-network-MwaaEnvironment-DAGProcessing - arn:aws:logs:us-east-2:164181677543:log-group:airflow-mwaa-environment-public-network-MwaaEnvironment-Worker - arn:aws:logs:us-east-2:164181677543:log-group:airflow-mwaa-environment-public-network-MwaaEnvironment-Scheduler - arn:aws:logs:us-east-2:164181677543:log-group:airflow-mwaa-environment-public-network-MwaaEnvironment-Task # List of streams to monitor (optional) # - dag_processor_manager.log # Log filter pattern (optional) # filterPattern: ELB ``` ### Configure your mappings To be able to show metrics and events in Geneos, dynamic mappings must be configured and attached to the Netprobe receiving the data from the Collection Agent. Use one of the following options listed below to configure your dynamic mappings. - Adding `templates/aws_mapping.xml` as an include file in your Gateway. This mapping template also includes sample FKM streams that can be used as a template which can then be customized to handle your intended Events, Logs, and Alarms. - Choosing a built-in mapping in > **Mapping**. For more information, see [Mapping and mapping group in Dynamic Entities](https://docs.itrsgroup.com/docs/geneos/current/processing/configuration/dynamic_entities/index.html#mapping-and-mapping-group). - Setting up a custom mapping in **Dynamic Entities Health** > **Mapping**. For more information, see [Mapping and mapping group in Dynamic Entities](https://docs.itrsgroup.com/docs/geneos/current/processing/configuration/dynamic_entities/index.html#mapping-and-mapping-group). ### Access AWS cloud through a proxy host The AWS Collection Agent plugin can connect to AWS through a proxy server by configuring the Collection Agent's JVM arguments. This setting can be modified in the Gateway Setup Editor, under **Dynamic Entities** > **Collection agent parameters** > **Collection agent** > **Jvm args**. 1. Route both `HTTP` and `HTTPS` traffic through a proxy server: ```args -Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttps.proxyHost= -Dhttps.proxyPort= ``` For example: ```args -Dhttp.proxyHost=webcache.example.com -Dhttp.proxyPort=8080 ``` 2. Authenticate to the proxy server using mTLS (client certificate): ```args -Djavax.net.ssl.keyStore=/client-keystore.p12 -Djavax.net.ssl.keyStorePassword= -Djavax.net.ssl.keyStoreType=PKCS12 ``` 3. Trust the proxy server and AWS endpoints by providing a truststore that contains both: - The proxy server’s Certificate Authority (CA) – used to verify the proxy server certificate - The AWS’s public CA – required to trust their service certificates ```args -Djavax.net.ssl.trustStore=/proxy-truststore.jks -Djavax.net.ssl.trustStorePassword= -Djavax.net.ssl.trustStoreType=JKS ``` These settings enable the AWS Collection Agent plugin to perform the following operations automatically and transparently: 1. Perform a mutual TLS handshake with the proxy on the `HTTPS` port. 2. Establish an `HTTP CONNECT` tunnel to AWS services. 3. Validate AWS’s certificates. For more information on adding JVM arguments, see [Managed Collection Agent](https://docs.itrsgroup.com/docs/geneos/current/processing/configuration/dynamic_entities/index.html#managed-collection-agent). To learn more about the available properties to enable proxy access, see [Java Networking and Proxies](https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html). ### Dealing with large volumes of data When the AWS plugin collects and publishes a large volume of data, the Collection Agent may perform garbage collection more frequently. This can lead to performance issues. To address this, you can increase the heap size of the Collection Agent by adding the following JVM arguments, for example: ``` -Xms1024m -Xmx1024m ``` This increases the initial heap size to 1024 MB and the maximum heap size to 1024 MB. Note that the default values are 512 MB for both the initial and maximum heap sizes. For more information on adding JVM arguments, see [Managed Collection Agent](https://docs.itrsgroup.com/docs/geneos/current/processing/configuration/dynamic_entities/index.html#managed-collection-agent). ## AWS API destination collector Aside from the [AWS Plugin services](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws-services/index.html#overview), you can also use this collector to monitor the following AWS services: - [EventBridge event and CloudWatch log streaming](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws/index.html#eventbridge-event-and-cloudwatch-log-streaming) - [Alarm Notifications](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws/index.html#alarm-notifications) You can use the sample mapping included in the Gateway package: `templates/aws_mapping.xml`, that contains the sample FKM streams to handle AWS Events, Logs, and Alarms services. ![](https://docscdn.itrsgroup.com/geneos-aws-cloudwatch/images/aws-gateway-config-eventbridge.png) **Note**  Ensure that you enable the AWS API destination configuration in the Collection Agent YAML file before you set up these services in the AWS Management Console site. ### EventBridge event and CloudWatch log streaming Amazon EventBridge, a serverless event bus service, streams real-time events, and logs from various AWS resources and applies its rules to route events to its targets, one of which is the `ApiDestinationCollector` collector. The `ApiDestinationCollector` collector exposes an HTTPS endpoint to connect to EventBridge, and the received data can then be processed as FKM streams. Logs and events are not formatted and are displayed as it is. A source dimension is available with the following behavior: - For logs, the value is always `lambda` since logs are passed through a lambda function. - For events, value is the source of the event. For example, for EC2 Instance State-change events, the source is `aws.ec2`. ### HTTPS and authentication For the `ApiDestinationCollector` to be considered a valid endpoint by AWS, it needs to use the HTTPS protocol. This can be achieved by using the TLS configuration of the `ApiDestinationCollector`. Using self-signed TLS certificates will not work. Events sent by AWS can only be read by the collector if certificates from a trusted CA (certificate authority) are used. For EventBridge events, the `ApiDestinationCollector` endpoint requires basic authentication. An event can only reach the API destination endpoint if it has the proper basic authentication credentials. #### Set up EventBridge event bus events To receive events from EventBridge, create an FKM stream in your Gateway then set up the following in your AWS Management Console. If you need more information about EventBridge, see [Amazon EventBridge](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-setup.html). 1. Navigate to **[Amazon EventBridge > Event buses > Create an event bus](https://console.aws.amazon.com/events/)** where you will stream the logs or events. 2. In the **Integration > Connections**, click **Create connection**. - In the **Authorization type**, select **Basic (Username/Password)** to input your desire username and password. - Authorization for EventBridge is required, but at the moment only basic authentication is supported by the plugin. 3. In the **Integration > API destinations**, click **Create API destination**. - In the **API destination endpoint**, enter `https://:`. - In the **HTTP method**, choose **POST**. 4. Navigate to **Amazon EventBridge > Rules** to create a rule. - In the **Define pattern**, select the type of event that the rule will apply to. - For lambda functions, the source is `lambda` and **detail-type** is either `Lambda Function Invocation Result - Success` or `Lambda Function Invocation Result - Failure` - In the **Target**, select **API destination**. Select the API destination that you created. #### API destination Events FKM dataview ![](https://docscdn.itrsgroup.com/geneos-aws-cloudwatch/images/aws-events-collector-dataview.png) #### Set up CloudWatch Logs CloudWatch logs can be sent to the `ApiDestinationCollector` collector through a subscription filter to a Lambda function, which then passes the logs to EventBridge. Similar to events, data is then sent from the EventBridge event bus to the collector. ##### Prerequisites - An EventBridge event bus and rule to the API destination. To create an event bus, follow the steps in [Set up EventBridge event bus events](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws/index.html#set-up-eventbridge-event-bus-events). - A lambda function. - When creating a lambda function, make sure that the execution role used has [EventBridge PutEvents](https://docs.aws.amazon.com/eventbridge/latest/APIReference/API_PutEvents.html) permission. - Sample code from AWS is available [here](https://aws.amazon.com/blogs/compute/introducing-aws-lambda-destinations/). Ensure that the lambda function's destination points to the event bus created in [Set up EventBridge event bus events](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws/index.html#set-up-eventbridge-event-bus-events). To receive CloudWatch logs, create an FKM stream in your Gateway then set up the following in AWS Management Console. If you need more information about AWS CloudWatch, see [CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/GettingSetup.html). 1. Navigate to **[CloudWatch > Log groups](https://console.aws.amazon.com/cloudwatch/)** to select the AWS resource log group that you want to stream. 2. In the **Subscription filters**, click **Create > Create Lambda subscription filter**. - In the **Choose destination**, select the lambda function you have set up as prerequisite. - In the **Configure log format and filters**, choose a filter pattern to match the logs that you want to stream. - Click **Start streaming**. 3. Navigate to **Lambda > Functions**. 4. Select the lambda function you have set up as prerequisite, then verify that the subscription filter is under **Configuration > Triggers** and that the correct EventBridge event bus is set as the destination under **Configuration > Destinations**. #### API destination Logs FKM dataview ![](https://docscdn.itrsgroup.com/geneos-aws-cloudwatch/images/aws-api-destination-logs-fkm.png) ### Alarm Notifications The `ApiDestinationCollector` collector can handle alarm notifications from SNS messages that are sent from standard topics. These alarm notifications are then published to the Collection Agent as log events. **Note** The alarm notifications only from the `snsRegion` specified in the `ApiDestinationCollector` collector configuration will be handled. The following describes some of the parameters of the log events related to Alarm notifications: | Parameter | Description | |-------|-------| | name | Name of the alarm that triggered the notification. This corresponds to the stream name of the FKM plugin in the Netprobe. | | message | Shows the details regarding the alarm that triggered the notification. This corresponds to the **triggerDetails** column of the FKM plugin in the Netprobe. Format: `TIMESTAMP [Namespace=NAMESPACE] [Metric=METRIC_NAME] State=STATE Reason=REASON [Dimensions=DIMENSIONS]` - `TIMESTAMP` - timestamp of the state update. This timestamp is in UTC. - `NAMESPACE` - namespace of the metric associated with the alarm. - `METRIC_NAME` - name of the metric associated with the alarm. - `STATE` - state value for the alarm. Possible values: `ALARM`, `OK`, or `INSUFFICIENT_DATA`. - `REASON`- explanation for the alarm state. For metric alarms, the reason contains the value and timestamp of the datapoints that caused the alarm state update. For composite alarms, the reason may contain the metric alarm that caused the state update of the composite alarm. - `DIMENSIONS`- dimensions for the metric associated with the alarm. For composite alarms, there are no `Namespace`, `Metric`, and `Dimensions` fields. **Sample message for Metric alarm:** 2021-12-03T06:38:23.047Z Namespace=AWS/EC2 Metric=CPUUtilization State=ALARM Reason="Threshold Crossed: 6 out of the last 30 datapoints were less than or equal to the threshold (0.7). The most recent datapoints which crossed the threshold: [0.0650449497620362 (03/12/21 06:31:00), 0.0327868852458998 (03/12/21 06:26:00), 0.0666666666666628 (03/12/21 06:21:00), 0.09947207557655299 (03/12/21 06:16:00), 0.0333333333333314 (03/12/21 06:11:00)] (minimum 30 datapoints for OK -> ALARM transition)." Dimensions=[{"value":"i-07d7e8d7dd0ee675f", "name":"InstanceId"}] **Sample message for Composite alarm:** s2021-12-03T08:18:20.602Z State=OK Reason="arn:aws:cloudwatch:ap-southeast-2: 164181677543:alarm:alarm_when_greaterthan_0.07 transitioned to INSUFFICIENT_DATA at Friday 03 December, 2021 08:18:20 UTC" | | entity dimensions | Entity dimensions parameters:- `namespace` - set to `AWS/Alarms`. - `region` - set to the value of the `snsRegion` option. - `source` - name of the SNS topic where the alarm notification came from. - `alarm_name` - name of the alarm that triggered the notification. | #### Set up for Alarm Notifications To receive SNS messages related to alarm notifications, create an FKM stream in your Gateway then set up the following in AWS Management Console. If you need more information about Alarm notifications, see [Amazon SNS](https://docs.aws.amazon.com/sns/latest/dg/sns-getting-started.html) and [Amazon CloudWatch alarms](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html). 1. Navigate to [Amazon SNS > Topic](https://console.aws.amazon.com/sns/home) to create an HTTPS subscription under the SNS topic that receives the alarm notifications that will be collected. 2. In the **Create subscription > Details**, set the **Endpoint** of this subscription to the URL corresponding to the server started by the collector. 3. Keep the **Enable raw message delivery** unchecked. The `ApiDestinationCollector` will not handle SNS messages where raw message delivery is enabled. For more information, see [Subscribing to an Amazon SNS topic](https://docs.aws.amazon.com/sns/latest/dg/sns-create-subscribe-endpoint-to-topic.html). Once an HTTPS subscription is confirmed, the collector will be able to receive alarm notifications from this subscription. - The collector should log `Successfully confirmed subscription to the topic TOPIC_NAME`, and in the AWS SNS Console, the status of the subscription should be **Confirmed**. - If the collector was not running when the HTTPS subscription is created, the status of the subscription will remain as **Pending confirmation**. To confirm this subscription, ensure that the collector is running and then request confirmation for this subscription. To request this confirmation: - Go to either the topic page or in the **AWS SNS Console > Subscription Page**. - Select the subscription with pending confirmation, and then click the **Request confirmation** button. #### API destination Alarms FKM dataview ![](https://docscdn.itrsgroup.com/geneos-aws-cloudwatch/images/aws-api-destination-alarms-fkm.png) ## AWS Billing collector The `AwsBillingCollector` collects budget and cost metrics from AWS to provide visibility into your cloud spending and budget management. This collector integrates with AWS Budgets and the AWS Cost Explorer API to deliver comprehensive cost and usage data directly into Geneos. The AWS Billing collector is essential for: - Budget management — track budget limits, actual spend, and forecasted spend to prevent cost overruns. **Note** Budgets with either an unset end date or an end date that has not yet passed will appear in the `AWS/Billings` dataviews, while expired budgets are excluded. - Cost visibility — monitor estimated charges, actual spending, and forecasted costs across services, regions, and accounts. - Cost optimization — identify cost trends and anomalies by analyzing amortized, blended, and unblended costs. - Usage tracking — monitor normalized usage amounts to compare resource consumption across different instance types and services. Normalized usage converts consumption into comparable units based on the smallest instance size in each family (for example, `t3.nano` = 1 unit, `t3.micro` = 2 units). For a complete list of available metrics, see [AWS/Billing in Plugin services](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws-services/index.html#awsbilling). The `AWS/Billing` collector configuration allows you to filter and group data by service, region, tags, or other dimensions to analyze costs at the appropriate level. #### Sample configuration: ```yaml # AWS Billing collector configuration - name: aws-billing type: plugin className: AwsBillingCollector # Interval (in millis) between collections (optional, defaults to 5 minutes). collectionInterval: 300000 # Publish SDK usage metrics sdkMetrics: true #Day of month where collection starts (optional, defaults to 1) monthlyStartDay: 1 #Metrics to be collected (optional, defaults to all metrics) #valid values https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/costexplorer/model/Metric.html #metrics: # - "AMORTIZED_COST" # - "USAGE_QUANTITY" # - "BLENDED_COST" #Data Collection Filter (optional, if empty data will be unfiltered) #valid dimension names (https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_DimensionValues.html) #includeFilter: # dimensions: # - name: "REGION" # values: # - "ap-southeast-1" # tags: # - name: "Team" # values: # - "Collection" # - "QA" #Data Grouping (required, maximum of 2 groups) #valid dimension names (https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_DimensionValues.html) grouping: dimensions: - "SERVICE" tags: - "Team" ``` #### Sample cost and usage metrics displayed in ITRS Analytics Dashboards: ![AWS Billing collector metrics in IAX](https://docscdn.itrsgroup.com/geneos-aws-cloudwatch/images/aws-billing-collector-in-iax.png) ## AWS SDK Usage metrics collector Since the AWS plugin interacts with AWS CloudWatch using Amazon provided APIs, take note of the [CloudWatch service quotas](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html). To monitor the SDK Usage of the `AwsCollector`, `AwsBillingCollector`, and `AwsCustomNamespaceCollector`, follow these steps: 1. Enable the `sdkMetrics` configuration under each collector. See the `collection-agent.yml` in [Configure Geneos to deploy the AWS plugin](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws/index.html#configure-geneos-to-deploy-the-aws-plugin-1). 2. Configure the `AwsSdkUsageMetricsCollector` to publish the SDK Usage metrics under the namespace `AWS/SdkUsage`. The `AwsSdkUsageMetricsCollector` reports the SDK Usage from the last 5-minute window. Below is an example dataview for AWS SDK Usage metrics: ![](https://docscdn.itrsgroup.com/geneos-aws-cloudwatch/images/aws-sdkusage-metrics-dataview.png) For more information, see [AWS/SDK Usage in Plugin services](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws-services/index.html#awssdkusage). ## AWS Custom namespace collector The AWS plugin can collect custom metrics for custom namespaces created in AWS CloudWatch. To monitor the custom namespace for the AWS plugin, follow these steps: 1. Add a `AwsCustomNamespaceCollector` class name in your `collection-agent.yml` file. For more information, see [Configure Geneos to deploy the AWS plugin](https://docs.itrsgroup.com/docs/geneos/collection/aws-cloudwatch/6.6.0/user-guide/aws/index.html#configure-geneos-to-deploy-the-aws-plugin-1). 2. (Optional) You can configure the `AwsSdkUsageMetricsCollector` to monitor the SDK usage of this collector by setting the sdkMetrics configuration under the `AwsCustomNamespaceCollector` to `true`. See [AWS SDK Usage metrics collector](#aws-sdk-usage-metrics-collector) for details on how to configure an `AwsSdkUsageMetricsCollector`. 3. Add and configure your custom namespace and the metrics to be collected under the `AwsCustomNamespaceCollector` class. 4. Set up and configure your custom mappings and other Dynamic Entities. Follow the steps in [Create Dynamic Entities in Collection Agent setup](https://docs.itrsgroup.com/docs/geneos/current/processing/configuration/dynamic_entities/index.html#create-dynamic-entities). Sample configuration: ```yaml # AWS custom metric collector configuration - name: awscustom type: plugin className: AwsCustomNamespaceCollector # Interval (in millis) between collections (optional, defaults to five minutes). collectionInterval: 300000 # AWS regions from which metrics will be collected. regions: - eu-west-1 # Publish SDK usage metrics sdkMetrics: true # List of custom namespaces to collect metrics from customMonitoredNamespaces: - namespace: ECS/ContainerInsights # List of metrics to collect from the custom namespace (optional, defaults to collect all available metrics if not specified) customMonitoredMetrics: # Regex pattern to match the metrics to collect from the namespace - nameIncludes: Network* - nameIncludes: Storage* # Length of time used to aggregate the metric (optional, defaults to 300 seconds) period: 300 # Aggregation to be used (optional, defaults to "average") statistic: average ``` Sample dataview: ![](https://docscdn.itrsgroup.com/geneos-aws-cloudwatch/images/aws-custom-namespace-dataview.png) ## CloudWatch Metric Streaming Amazon CloudWatch Metric Streams, together with Amazon Data Firehose, can send real-time metrics from various AWS resources to its configured HTTPS destination endpoint where the `AwsMetricStreamCollector` collector is located. The collector exposes an HTTPS endpoint to connect to Data Firehose, and the received data is processed to provide similar datapoints as `AwsCollector`. **Note** Failed communication between AWS Data Firehose and `AwsMetricStreamCollector` results in [logs being stored in S3 buckets](https://docs.aws.amazon.com/firehose/latest/dev/create-configure.html#create-configure-backup), leading to increased storage usage. ### HTTPS and Authentication For the `AwsMetricStreamCollector` to be considered a valid endpoint by AWS, it needs to use the HTTPS protocol. This can be achieved by using the TLS configuration of the `AwsMetricStreamCollector`. **Note** Using self-signed or free TLS certificates, like Let's Encrypt, will not work. Metrics sent by AWS can only be read by the collector if certificates from a trusted CA (certificate authority) are used. ### Setup for CloudWatch Data Firehouse 1. Select the source and destination: - **Source** — Direct PUT - **Destination** — HTTP Endpoint 2. Transform records — Not supported under the current revision 3. Destination settings: - **HTTP endpoint URL** — Endpoint of the collector - **Authentication/Access key** — Not supported under the current revision - **Content encoding** — GZIP is not supported under the current revision 4. Buffer hints — Lower **buffer size** and **interval values** will notify firehose to send metrics more frequently. 5. Advanced settings - **Server-side encryption** — Not supported under the current revision ### Setup for CloudWatch Metric Streams CloudWatch metrics can be sent to the `AwsMetricStreamCollector` collector through an AWS Data Firehose stream. 1. Destination: - Custom setup with Firehose - Select your **Amazon Data Firehose stream** — Select the created Data Firehose 2. Change output format — Ensure this configuration matches with collector configuration. 3. Metrics to be streamed: - Select either **All metrics** or **Select metrics**. For **Select Metrics**, select individual metrics to be included or excluded. - Add **additional statistics** — Only default statistics are currently supported (Minimum, Maximum, Sample Count, and Sum). ## AWS CloudWatch Logs Collector CloudWatch logs can be monitored directly using the `AwsCloudwatchLogsCollector`. This collector starts a live tail session on the selected log groups and log streams, filtering based on the specified `filterPattern`. Events received are then published as log events. This collector requires the following permissions: 1. logs:StartLiveTail 2. logs:DescribeLogGroups To monitor CloudWatch logs using this collector, configure the `AwsCloudwatchLogsCollector`: 1. Specify the `regions` containing the logs you want to monitor. 2. Define log groups to monitor using the following optional parameters: - `logGroupNamePattern` — a case-sensitive substring used to match log group names. Log groups that contain this pattern will be monitored. - `logGroups` — a list of specific log groups to monitor, identified by their full Amazon Resource Names (ARNs). Ensure to remove any special characters (for example, `*`) at the end of the logGroup name. If `logGroups` is specified, the value of `logGroupNamePattern` will be ignored. - `logGroupRefreshInterval` — the interval (in milliseconds) at which to refresh the `logGroups` or `logGroupNamePattern` matches. The default value is five minutes. **Note** If neither `logGroups` nor `logGroupNamePattern` is defined, the collector will monitor all available log groups in the specified regions. It will automatically partition the log groups across multiple monitoring services, grouping 10 log groups per service, to efficiently handle any number of log groups. #### Sample configuration: ```yaml # AWS Cloudwatch Log Collector configuration - name: cloudwatchlogs type: plugin className: AwsCloudwatchLogsCollector # AWS regions that contains the logs to monitor (required) regions: - ap-southeast-1 - ap-southeast-2 # List of log groups to monitor (optional) logGroups: - arn:aws:logs:us-east-2:164181677543:log-group:airflow-mwaa-environment-public-network-MwaaEnvironment-WebServer - arn:aws:logs:us-east-2:164181677543:log-group:airflow-mwaa-environment-public-network-MwaaEnvironment-DAGProcessing - arn:aws:logs:us-east-2:164181677543:log-group:airflow-mwaa-environment-public-network-MwaaEnvironment-Worker - arn:aws:logs:us-east-2:164181677543:log-group:airflow-mwaa-environment-public-network-MwaaEnvironment-Scheduler - arn:aws:logs:us-east-2:164181677543:log-group:airflow-mwaa-environment-public-network-MwaaEnvironment-Task # List of streams to monitor (optional) # - dag_processor_manager.log # Log filter pattern (optional) # filterPattern: ELB # Log Group name pattern (optional) # logGroupNamePattern: public-network # Log group refresh interval (in millis) (optional) # Default is 5 minutes # Minimum is 1 minute, maximum is 1 day # logGroupRefreshInterval: 30000 ``` #### Sample metrics published in the ITRS Analytics Entity Viewer ![AWSCloudwatchLogsCollector metrics in IAX](https://docscdn.itrsgroup.com/geneos-aws-cloudwatch/images/aws-cloudwatch-logs-collector-iax-ev.png) ## AWS Redshift Data Collector Redshift data can be queried directly using the `AwsRedshiftDataCollector`. This collector queries single statements on a specified Redshift cluster and database. The result set is then published according to the datapoint configuration defined for each column. Configure the `AwsRedshiftDataCollector` by defining a query configuration. This includes the following required values: - A unique query name - A query statement - A datapoint configuration to define how the query results are displayed Sample configuration: ```yaml # AWS Redshift Data Collector configuration - name: redshiftdata type: plugin className: AwsRedshiftDataCollector # Interval (in millis) between collections (optional, defaults to five minutes). #collectionInterval: 300000 # Database connection (required) database: region: ${env:REGION} clusterId: ${env:REDSHIFTDATA_CLUSTER} dbName: ${env:REDSHIFTDATA_DATABASE} user: ${env:REDSHIFTDATA_USER} # Generic query timeout (optional, in millis, defaults to collectionInterval) # This value must be less than or equal to collectionInterval #timeout: 300000 # Query configuration (required) queries: # Query name (required) # The name must be unique between different queries - name: queues # Query statement (required) # Only supports single statements query: SELECT COALESCE(s.service_class, q.service_class) AS service_class, s.num_queued_queries, q.max_exec_time FROM stv_wlm_service_class_state s FULL OUTER JOIN ( SELECT service_class, MAX(total_exec_time) AS max_exec_time FROM stl_wlm_query GROUP BY service_class) q ON s.service_class = q.service_class ORDER BY service_class; # Query timeout (optional, in millis, defaults to generic query timeout) # This value must be less than or equal to generic query timeout #timeout: 300000 # Dimensions (optional) # A list of columns to serve as dimensions. Restrictions on dimensions: # - Values within the dimension can not have duplicates # - Values within the dimension should not be found in the datapoints # - A dimension set must be unique between different queries dimensions: [service_class] # Datapoint (required) # A list of columns to serve as datapoints. # - Datapoint names should not be found within the dimension datapoints: # Column name num_queued_queries: # Column datapoint type (gauge, attribute, status, counter) type: counter # Column datapoint unit, possible values are as described in Unit class unit: none # Column name max_exec_time: # Column datapoint type (gauge, attribute, status, counter) type: gauge # Column datapoint unit, possible values are as described in Unit class unit: microseconds # Query name (required) # The name must be unique between different queries - name: blocked # Query statement (required) # Only supports single statements query: SELECT COUNT(DISTINCT lock_owner) AS blocked FROM stv_locks ORDER BY blocked DESC; # Query timeout (optional, in millis, defaults to generic query timeout) # This value must be less than or equal to generic query timeout #timeout: 300000 # Dimensions (optional) # A list of columns to serve as dimensions. Restrictions on dimensions: # - Values within the dimension can not have duplicates # - Values within the dimension should not be found in the datapoints # - A dimension set must be unique between different queries #dimensions: [] # Datapoint (required) # A list of columns to serve as datapoints. # - Datapoint names should not be found within the dimension datapoints: # Column name blocked: # Column datapoint type (gauge, attribute, status, counter) type: counter # Column datapoint unit, possible values are as described in Unit class unit: none ``` #### Sample metrics published in the ITRS Analytics Entity Viewer ![AWS RedshiftDataCollector metrics in IAX](https://docscdn.itrsgroup.com/geneos-aws-cloudwatch/images/aws-redshift-data-collector-iax-ev.png)