--- title: "Ansible configuration" description: "Master Ansible playbook configuration for Netprobe installations, updates, and diagnostics across AWS, Azure, and Google Cloud environments." url: "https://docs.itrsgroup.com/docs/geneos/7.11.0/collection/cloud-native-deployments/deploying-via-terraform/ansible-configuration/index.html" --- This section explains the general structure of the Ansible playbook configuration. For settings specific to each environment, such as file paths, cloud options, or bootstrap commands, refer to the corresponding Terraform environment guides for [AWS](https://docs.itrsgroup.com/docs/geneos/7.11.0/collection/cloud-native-deployments/deploying-via-terraform/instance-provisioning/aws-instance-provisioning/index.html), [Azure](https://docs.itrsgroup.com/docs/geneos/7.11.0/collection/cloud-native-deployments/deploying-via-terraform/instance-provisioning/azure-instance-provisioning/index.html), and [Google Cloud](https://docs.itrsgroup.com/docs/geneos/7.11.0/collection/cloud-native-deployments/deploying-via-terraform/instance-provisioning/google-cloud-instance-provisioning/index.html). The Ansible playbook supports the following operations: - Install - Update - Setup file - Rollback - Environment - Uninstall - Fetch archive ## Prerequisites - Ansible version 2.15 or higher installed. The installation should automatically include `ansible-core`. - Compatible Ansible and Python in control and target machines. Refer to the `ansible-core` [support matrix](https://docs.ansible.com/ansible/7/reference_appendices/release_and_maintenance.html#ansible-core-support-matrix) for more information. - Ansible collections for [AWS](https://docs.ansible.com/ansible/latest/collections/amazon/aws/index.html#plugins-in-amazon-aws), [Azure](https://docs.ansible.com/ansible/latest/collections/azure/azcollection/index.html#plugins-in-azure-azcollection), or [Google Cloud](https://docs.ansible.com/ansible/latest/collections/google/cloud/index.html#plugins-in-google-cloud). - Resources stored either in the localhost or cloud storages. Ensure you have proper access permissions for virtual machines if the resources are stored in cloud. - Netprobe - Netprobe tarball - Netprobe setup file - Cloud CLI tools (installed automatically during bootstrap): - AWS CLI for S3 access - Azure CLI for Azure Blob Storage access - Google Cloud SDK for Google Cloud Storage (GCS) access ### SSH configuration for Ansible Configure `startup.yml` to define the SSH configurations required for Ansible to securely connect to your target servers. | Parameter | Description | |-------|-------| | `ansible_user` | Username for SSH connections to target hosts. | | `ansible_ssh_private_key_file` | Path to SSH private key file for authentication. | | `ansible_ssh_common_args` | Additional SSH connection arguments. | | `connection_timeout` | Connection timeout in seconds. | ### Bootstrap for Python and cloud CLI installation for Ansible **Important** Only configure bootstrap when there is a mismatch observed in your environment. For example, when you are unsure about Ansible compatibility between the control node (where Ansible is running) and target node (where Netprobe is being installed). By default, bootstrap is set to `false`. You must be familiar with what to configure when deciding to enable bootstrap. Otherwise, please reach out to ITRS for support. | Parameter | Description | |-------|-------| | `perform_bootstrap` | Flag to enable/disable bootstrap operations (boolean). | | `bootstrap_packages` | OS-specific packages for Ansible compatibility (system packages). | | `pip_packages` | List of Python packages to install via pip. | | `python_path` | Python executable path on target host. | | `bootstrap` | Shell script commands to execute during bootstrap phase. | | `additional_bootstrap` | Additional bootstrap commands (cloud-specific CLI tools, etc.). | ### Cloud storage authentication Before running any Ansible operation, ensure you have access to the cloud storage (whether in AWS, Azure, or Google Cloud) where your resources are stored. #### AWS For Amazon Web Services integration, the Ansible playbooks support multiple authentication methods for S3 access. The authentication follows the AWS SDK credential precedence chain. ##### Required S3 permissions Regardless of authentication method, the credentials must have the following S3 permissions: - S3 Object Access (`s3:GetObject`) — for downloading deployment resources. - S3 Bucket List (`s3:ListBucket`) — for listing bucket contents. ##### Supported authentication methods - Instance profile — You must have an instance profile with an IAM role granting `read` permissions on the S3 bucket. Refer to the following AWS documentation for more information: - [Create new IAM role](https://docs.aws.amazon.com/cli/latest/reference/iam/create-role.html) - [Add or update inline policy document within the IAM role](https://docs.aws.amazon.com/cli/latest/reference/iam/put-role-policy.html) - [Create new instance profile](https://docs.aws.amazon.com/cli/latest/reference/iam/create-instance-profile.html) - [Add an IAM role to the instance profile](https://docs.aws.amazon.com/cli/latest/reference/iam/add-role-to-instance-profile.html) For example: ```bash aws iam create-role --role-name MyEC2S3ReadRole --assume-role-policy-document '{ "Version":"2012-10-17", "Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}] }' aws iam put-role-policy --role-name MyEC2S3ReadRole --policy-name S3Read-MyBucket --policy-document '{ "Version":"2012-10-17", "Statement":[ {"Effect":"Allow","Action":["s3:GetObject"],"Resource":"arn:aws:s3:::/*"}, {"Effect":"Allow","Action":["s3:ListBucket"],"Resource":"arn:aws:s3:::"} ] }' aws iam create-instance-profile --instance-profile-name MyEC2InstanceProfile aws iam add-role-to-instance-profile --instance-profile-name MyEC2InstanceProfile --role-name MyEC2S3ReadRole ``` - When to use: This is the default behavior for EC2 instances with attached instance profiles. - Named AWS profile — uses a named AWS CLI profile configured on the target machine. - Prerequisites: - The AWS CLI configured with named profile on target machine. - The profile must have the required S3 permissions. - Configuration ```yaml resources: source: aws: profile: non_default_profile # Name of AWS CLI profile region: us-east-1 # Optional region parameter ``` - When to use: This is suitable when you are using specific AWS CLI profiles for credential management. - Session token (temporary credentials) — uses temporary AWS credentials with session token. - Prerequisites - A valid AWS access key, secret key, and session token. - Temporary credentials that have required S3 permissions. - Configuration ```yaml resources: source: aws: sessiontoken: accesskey: AKIAIOSFODNN7EXAMPLE secretkey: wJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY token: AQoEXAMPLEH4aoAH0gNCAPyJxz4BlCFFxWNE1OPTgk5TthT+FvwqnKwRcOIfrRh3c/L region: us-east-1 ``` - When to use: Use this when you need temporary access only or when using AWS STS assumed roles. #### Azure The Ansible playbooks only support the [Managed Identity authentication](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview) for Azure Blob Storage access. ##### Required Azure permissions The user-assigned identity must have the following Azure permissions: - `Microsoft.Storage/storageAccounts/read` — for accessing storage account information. - `Microsoft.Storage/storageAccounts/listKeys/action` — for retrieving storage account access keys. ##### Supported authentication methods - Managed Identity — uses the User Assigned Identity attached to the Azure Virtual Machine. No credential parameters are needed. - Prerequisites - Running on an Azure virtual machine with attached user-assigned identity. - The identity must have the required Azure storage permissions. - Configuration ```yaml resources: source: azure: resource_group: "my-resource-group" storage_account_name: "mystorageaccount" # No credential parameters required - uses managed identity automatically ``` - When to use: This is the required authentication method for Azure VMs with attached managed identity. #### Google Cloud Platform (GCP) The Ansible playbooks support multiple authentication methods based on the [Google Cloud Ansible collection](https://docs.ansible.com/ansible/latest/collections/google/cloud/gcp_storage_object_module.html). The authentication type is configured using the `auth_kind` parameter in your GCP resource configuration. ##### Required service account permissions Regardless of authentication method, the service account must have the following IAM roles: - Storage Object Viewer (`roles/storage.objectViewer`) - for accessing cloud storage objects. - Storage Bucket Viewer (`roles/storage.bucketViewer`) - for listing storage buckets. ##### Supported authentication methods - Application default credentials (`auth_kind: "application"`) — uses Google Cloud Application Default Credentials (ADC). No credential parameters are needed. - Prerequisites - `GOOGLE_APPLICATION_CREDENTIALS` environment variable must be set to the service account key file path, OR - The virtual machine must have a service account attached (recommended for GCP compute instances). - Configuration ```yaml resources: source: gcp: auth_kind: "application" # No credential parameter required ``` - When to use: This is recommended for deployments on GCP compute instances with attached service accounts. - Machine account (`auth_kind: "machineaccount") — uses the default service account of the compute instance. - Prerequisites - Must be running on a GCP compute instance. - The instance has a service account attached. - Configuration: ```yaml resources: source: gcp: auth_kind: "machineaccount" credential: "service-account@project-id.iam.gserviceaccount.com" # Service account email ``` - When to use: This is suitable when you want to explicitly specify the service account email for a compute instance. - Service account key file (`auth_kind: "serviceaccount"`) — uses a service account JSON key file for authentication. - Prerequisites: - The service account JSON key file must be accessible on the target machine. - Configuration: ```yaml resources: source: gcp: auth_kind: "serviceaccount" credential: "/path/to/service-account-key.json" # Path to JSON key file ``` - When to use: This is suitable for deployments from non-GCP environments or when using explicit service account keys. - Access token (`auth_kind: "accesstoken"`) — uses a Google Cloud access token for authentication. - Prerequisites - A valid Google Cloud access token. - Token management (renewal) handled externally. - Configuration ```yaml resources: source: gcp: auth_kind: "accesstoken" credential: "ya29.c.Kp6B9Q..." # Access token string ``` - When to use: This is suitable for temporary access or when integrating with external token management systems. **Unsupported authentication method**   `service_account_contents` - inline service account JSON content is not supported for security reasons. ## Netprobe configuration Use the following configuration to deploy the Netprobe: ```yaml # ============================================================================= # NETPROBE CONFIGURATION # ============================================================================= # ============================================================================= # ENVIRONMENT SETUP # Configuration for setting up the environment for initial netprobe usage # ============================================================================= # List of packages needed by netprobe to run packages: # Method to install the package, available options at the moment are 'system', 'pip', and 'shell' - method: system packages: [ # JAVA package if java-plugins or c2-plugins will be used, sample packages below #{ name: openjdk-21-jre-headless }, #{ name: java-21-openjdk }, # RedHat 8/9 packages #{ name: openssl-libs }, { name: glibc }, { name: libnsl2 }, { name: libxcrypt }, { name: zlib }, { name: libstdc++ }, { name: libgcc }, { name: libtirpc }, #{ name: krb5-libs }, { name: libcom_err }, { name: keyutils-libs }, { name: libselinux }, { name: pcre2 }, { name: libcurl } ] # Workspace directory (required) workspace_dir: "/opt/workspace" # Environment variables (required) env_vars: # Component log file (required, for rollback mechanism) LOG_FILENAME: "{{ workspace_dir }}/logs/{{ component }}/{{ component }}.log" MAX_LOG_FILE_SIZE_MB: "10" # Gateway information (required) GATEWAY_HOST: "0.0.0.0" GATEWAY_PORT: "7039" # Netprobe information (required) SERVICE_PARAM: "-port 7036 -setup {{ workspace_dir }}/{{ resources.setupfile.path | basename }}" # ============================================================================= # RESOURCE CONFIGURATION # Configuration where resources come from when installing or updating the system # ============================================================================= resources: # Remove source to use localhost as default #source: # Valid configurations are gcp, azure, aws #aws: # Default authentication is instance profile (no parameters needed) # Optional region parameter, defaults to instance metadata #region: us-east-1 # Alternative: Named AWS CLI profile #profile: non_default_profile # Alternative: Session token for temporary credentials #sessiontoken: # accesskey: accesskey # secretkey: secretkey # token: token #azure: # resource_group: resource_group # storage_account_name: storage_account_name # account_key: account_key #gcp: # Default authentication is application, see Ansible readme.md for more details #auth_kind: application #credential: "please see Ansible readme.md if needed" # Cloud storage container where resource is located # container: geneos # Path where resource is located (required) path: ../../resources/netprobe.tar.gz # Netprobe executable file (required) executable: netprobe.linux_64 setupfile: # Cloud storage container where resource is located (optional, defaults to resources.container) #container: geneos # Path where resource is located in (required) path: ../../resources/netprobe.setup_template.xml # ============================================================================= # UPDATE OPERATIONS # Configuration specific to update operations # ============================================================================= # List of files to import over during Netprobe updates (only used for update operation) import_config: - dir: "collection_agent" patterns: - 'collection-agent.yml' - 'logback.xml' - dir: "Workflow" # ============================================================================= # VERIFICATION # Configuration for verification after every operation # ============================================================================= # List of patterns for netprobe verification after every operation verification: # Initialization delay after update before logs verification (optional, defaults to 5 seconds) #initialization_delay: 5 # Log strings to indicate system behavior (required) # List of strings are patterns used for Extended Regular Expressions success: - "SelfAnnouncing Connected to Gateway" errors: - "ERROR.*FATAL.*" # ============================================================================= # DIAGNOSTICS # Configuration for diagnostic operations (only needed on fetch_archive operation) # ============================================================================= # Diagnostic configuration (only needed on fetch_archive operation) diagnostic: # A list of files to archive for diagnostic purpose files: # Netprobe log files, as configured in env_vars.LOG_FILENAME - "{{ env_vars.LOG_FILENAME | dirname }}*" # Collection agent log files and log configuration - "{{ workspace_dir }}/{{ component }}/collection-agent.log" - "{{ workspace_dir }}/{{ component }}/collection_agent/collection-agent.log" - "{{ workspace_dir }}/{{ component }}/collection_agent/logback.xml" # Collection agent configurations - "{{ workspace_dir }}/{{ component }}/collection-agent.yml" - "{{ workspace_dir }}/{{ component }}/collection_agent/collection-agent.yml" dest: "./" ``` ### Configure package dependencies | Parameter | Description | |-------|-------| | `packages` | List of package groups to be installed, organized by installation method. | | `packages.method` | Installation method: system, pip, or shell. | | `packages.packages` | List of packages to be installed using the specified method. | | `packages.packages.name` | Name of the package to be installed. | | `packages.packages.script` | Shell script to execute (when method is shell). | ### Configure the Netprobe deployment | Setting | Description | |-------|-------| | `component` | Service component name (typically `netprobe`) | | `workspace_dir` | Remote target destination directory for Netprobe installation | | `env_vars` | Environment variables required for Netprobe operation - LOG_FILENAME - Path to error and log file - MAX_LOG_FILE_SIZE_MB - Log file rollover size in megabytes - GATEWAY_HOST - Gateway server IP address - GATEWAY_PORT - Gateway service port number - SERVICE_PARAM - Netprobe CLI flags (for example, `-port 7036 -setup `). See [Netprobe command-line options](https://docs.itrsgroup.com/docs/geneos/7.11.0/collection/netprobe/configuration/netprobe-command-line-options/index.html#binary-command-line-options). | | `import_config` | Configuration for importing files during Netprobe updates - dir - Directory name to import from previous installation - patterns - File patterns to import (defaults to `*` for all files) | | `resources` | Resource configuration for Netprobe binaries and setup files - source - Cloud storage source configuration, remove if local - `aws` - AWS S3 configuration (see [AWS authentication types](#aws) section) - `azure` - Azure Blob Storage configuration (`resource_group`, `storage_account_name`, `account_key`) - `gcp` - Google Cloud Storage configuration (see [GCP authentication types](#google-cloud-platform-gcp) section) - Empty/undefined - Uses local file paths (no cloud access) - container - Cloud storage container/bucket name - path - Path to netprobe executable archive - executable - Name of netprobe executable file within the archive - setupfile - Setup file configuration - container - Container/bucket for setup file (optional, defaults to `resources.container`) - path - Path to netprobe setup file | | `verification` | Post-deployment verification configuration - initialization_delay - Delay in seconds before checking logs for verification - success - List of log message patterns indicating successful operation - errors - List of log message patterns indicating operation failure | | `diagnostic` | Diagnostic file collection configuration - files - List of file patterns to collect for diagnostics - dest - Local destination path for diagnostic archive | ## Installing the Netprobe The Netprobe can be installed either on a remote host or a local machine by running the following Ansible commands. - Remote host installation ```command ansible-playbook -i "," netprobe.yml ``` **Important** Replace `` with a comma-separated host list. A comma is needed at the end of the list. - Local machine installation ```command ansible-playbook -i "localhost," -c local netprobe.yml ``` **Important** A comma is needed at the end of `localhost`. Running these commands will perform the following: 1. Install all required packages. 2. Set up the Netprobe system service. 3. Install the Netprobe. 4. Run the Netprobe. ### Ansible-supported operations The Ansible playbook supports the following operations: | Operation | Definition | |-------|-------| | `install` | Install and configure Netprobe service. | | `update`| Update Netprobe to newer version (creates backup for rollback). | | `rollback` | Rollback to previous version after failed update. | | `setupfile` | Update Netprobe setup file configuration. | | `environment`| Update environment variables and reinstall packages. | | `fetch_archive` | Download diagnostic files and logs. | | `uninstall` | Remove Netprobe installation and service. | Specify your desired operation in the command by appending `-e "operation=`. For example: `ansible-playbook -i "hosts," netprobe.yml -e "operation=update"`. ### Custom commands #### Update Netprobe ```shell ansible-playbook -i "," netprobe.yml -e "operation=update" ``` The `update` operation downloads and installs the Netprobe package configured in `resources.container` (if from Cloud Storage) and `resources.path`. A backup copy of the current version will be stashed for a potential `rollback` operation. The list of files in `import_config` will be imported to the new package after installation. Post verification will be performed based on log patterns configured in `verification.success` and `verification.error`. #### Rollback Netprobe ```shell ansible-playbook -i "," netprobe.yml -e "operation=rollback" ``` The `rollback` operation will only work after a previous `update` operation. A copy of the current Netprobe installation will be stashed with suffix `_failed` for further analysis. Post verification will be performed based on log patterns configured in `verification.success` and `verification.error`. #### Update Netprobe setup file ```shell ansible-playbook -i "," netprobe.yml -e "operation=setupfile" ``` The `setupfile` operation downloads the Netprobe setup file configured in `resources.setupfile.container` (if from Cloud Storage) and `resources.setupfile.path`. Post verification will be performed based on log patterns configured in `verification.success` and `verification.error`. User can rerun the `setupfile` operation on a working version in case of errors. #### Update environment variables and reinstalls required Netprobe packages ```shell ansible-playbook -i "," netprobe.yml -e "operation=environment" ``` The `environment` operation rewrites environment variables configured in `env_vars` and reinstalls packages listed in `packages`. Post verification will be performed based on log patterns configured in `verification.success` and `verification.error`. User can rerun the `environment` operation on working values and packages in case of errors. (eg. Gateway host: `env_vars.GATEWAY_HOST`) #### Fetch diagnostic files ```shell ansible-playbook -i "," netprobe.yml -e "operation=fetch_archive" ``` The `fetch_archive` operation will archive files listed in `diagnostic.files` and archive them in the format `netprobe_diagnostics___`. This operation does not stop the service to ensure continuous monitoring during diagnostic collection. #### Uninstall Netprobe ```shell ansible-playbook -i "," netprobe.yml -e "operation=uninstall" ``` The `uninstall` operation deletes everything inside `workspace_dir` and removes the systemd service file.