Capacity Planner ["Capacity Planner"]

Azure data collection

Overview

In order for the Data Collector to access Azure resources, it needs an application and service principal to be added to the Azure tenancy.

Note: When extracting data from cloud providers, the Capacity Planner data collectors are run from the ITRS environment using secure read-only credentials provided by the customer. This avoids unnecessary network transfer, the need to upgrade and maintain on-premise data collectors, and ensures that data collection is always at the most up-to-date release.

Access and security

Once retrieved, data is stored on the Data Collector hosting machine inside the ITRS firewall. It is segregated in separate folders by customer and project. Once collected, the data is zipped and encrypted using AES encryption and uploaded to a database exclusively created for the given customer's data.

ITRS is ISO 27001 accredited. For more information. see our Security policies.

Create Service Principal

To create Service Principal, follow the steps:

  1. Sign in to your Azure account through the Azure portal.

  2. Navigate to Azure Active Directory.

  3. Select App registrations, and then click New registration.

  4. Name the new application ICP-DataCollector.

  5. Under Supported account types, select the option for the Single tenant.

  6. Set the Redirect URI to be Public client/mobile (desktop & mobile).

  7. Click Register to complete the registration.

Complete instructions on how to create a new role can be found in MicrosoftAzure documentation.

Create custom role

To limit the permissions only to the ones that Data Collector needs, create a new role.

  1. Create an empty text file called QueryRateCardsRole.json.

  2. Copy the following into the text file and replace <subscriptionID> (including the <>) with the ID of the subscription this role is to be located in.

Complete instructions on how to create a new role can be found in Microsoft Azure documentation.

Assign roles

To assign roles, follow the steps:

  1. In the Azure portal, click Subscriptions.

  2. Select Access Control (IAM).

  3. Select Add role assignment either from Grant access to this resource or in the Add menu.

  4. In the Role drop-down menu, select QueryRateCardsRole.

  5. Select ICP-DataCollector that you created as the app.

  6. Click Save.

Repeat the steps for the Billing Reader and Monitoring Reader roles.

Gather credentials for Azure collection task

You need to gather the following values from the Azure portal to set up the collection task. They are available from the following locations:

  • Subscription Id — available from Home > Subscription > Subscription ID

  • Tenant Id — available from Home > Azure Active directory > App registrations

  • Search for or select ICP-DataCollector > Tenant ID

  • Client Id — available from Home > Azure Active directory > App registrations

  • Search for or select ICP-DataCollector > Client ID

  • Secret key — for instructions, see Generate a secret key.

Generate a secret key

To generate a secret key, follow the steps:

  1. Navigate to Azure Active Directory.

  2. From App registrations, select your application.

  3. Select Certificates & secrets.

  4. Select Client secrets > New client secret.

  5. Provide a description and a duration of the secret, then click Add.

Note: After saving the client secret, the value of the client secret is displayed. You must copy this value because you will not be able to retrieve the key later.

Frequency of collection

Connectivity for data collection is every 12 hours, but can be configured to be more frequent.

What data is collected

Once connectivity is established, properties and metrics are collected from the estate using Azure API and RestAPI.

The collected data includes:

  • Virtual machine instances and their properties.

  • Scale set details and associated virtual machines.

  • Metrics for virtual machines, specifically:

    • Percentage CPU

    • Network In

    • Network Out

    • OS Per Disk Read Bytes/sec

    • OS Per Disk Write Bytes/sec

    • CPU Credits Remaining

    • CPU Credits Consumed

  • Events relating to virtual machines and scale set virtual machines, specifically:

    • write events

    • deallocate/action events

    • delete events

    • start/action events

  • Reference data

  • Resource SKUs

  • Rate cards

  • Catalogues

  • Azure Cloud Costs, including usage details and usage aggregates

  • Azure Savings Plans

  • Azure Reserved Instances