Skip to content

Metrics Agent Script


Overview

Metrics agent is usually an executable script/command responsible to collect metrics from the required services running on the cloud and stream them to the respective metrics service created on CloudAEye.

This topic provides complete details about the the metrics agent script deployed on your cloud environment to collect metrics from the selected services.

⚓ Prerequisites

CloudAEye makes use of python to automate the process of deploying an agent on to your cloud environment.

  • User must have python3 installed on the system

How it Works?

AWS ECS(Fargate)

CloudAEye's ECS metrics agent script does the following:

  1. Fetches the subnet and security group details of the VPC
  2. Regsiters a new task definition for the agent
  3. Runs the agent as a standalone task on the ECS cluster

How boto3 is used

To perform the above steps automatically, the agent script makes use of AWS boto3 python SDK. The AWS services accessed by the SDK are:

AWS Permissions Required

To perform the above mentioned steps, the AWS SDK (boto3) needs the below set of permissions to be enabled

These permissions needs to be added to the AWS account used to deploy the agent. See how to configure an AWS account

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ecs:ListTaskDefinitionFamilies",
                "ecs:RegisterTaskDefinition",
                "ecs:DescribeTaskDefinition"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "iam:PassRole",
                "ecs:ListTasks",
                "ecs:DescribeTasks"
            ],
            "Resource": [
                "arn:aws:ecs:AWS_REGION:AWS_ACCOUNT_ID:container-instance/*",
                "arn:aws:ecs:AWS_REGION:AWS_ACCOUNT_ID:task/*",
                "arn:aws:iam::AWS_ACCOUNT_ID:role/*"
            ]
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeSubnets",
                "ec2:DescribeSecurityGroups",
                "ec2:CreateSecurityGroup"
            ],
            "Resource": [
                "arn:aws:ec2:AWS_REGION:AWS_ACCOUNT_ID:security-group/*",
                "arn:aws:ec2:AWS_REGION:AWS_ACCOUNT_ID:vpc/VPC_ID"
            ]
        }
    ]
}

AWS EKS(on EC2)

CloudAEye's EKS metrics agent script does the following:

  1. Adds the agent as a daemonset on the EKS cluster using helmv3 chart

    This steps needs helmv3 to be installed as a pre-requisite.

  2. (Optional) Collects metrics of other AWS resources from Cloudwatch

    To enable this Enable cloud services input needs to be provided.

How boto3 is used

To perform the above steps automatically, the agent script makes use of AWS boto3 python SDK. The AWS services accessed by the SDK are:

  • AWS CloudFormation

    Applicable only if Enable cloud services is enabled

    • Describe Stacks to check if a stack already exists for CloudAEyeServiceAccountStack
    • Create Stack creates a stack CloudAEyeServiceAccountStack > This stack provisions a dedicated IAM user responsible to collect and ship metrics from the Cloudwatch to the log service
  • AWS IAM

    Applicable only if Enable cloud services is enabled

    • List Policy Versions Lists all available policies
    • Get Policy Fetch the policy CloudAEyeServiceAccountPolicy (if exists)
    • Create Policy Creates policy CloudAEyeServiceAccountPolicy (if not already exists)
    • Delete Policy Deletes a given policy
    • Get User Fetch user CloudAEyeServiceAccount (if exists)
    • Create User Creates a programmatic user CloudAEyeServiceAccount (if not alrady exists)
    • List AccessKeys List all keys available for the programmatic IAM user CloudAEyeServiceAccount
    • Create AccessKey Create access key for the programmatic IAM user CloudAEyeServiceAccount
    • Attach User Policy Attach user policy CloudAEyeServiceAccountPolicy to user CloudAEyeServiceAccount
  • AWS Cloudwatch

    Applicable only if Enable cloud services is enabled

AWS Permissions Required

To perform the above mentioned steps, the AWS SDK (boto3) needs the below set of permissions to be enabled

These permissions needs to be added to the AWS account used to deploy the agent. See how to configure an AWS account

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "iam:GetUser",
                "iam:CreateUser",
                "iam:CreateAccessKey",
                "iam:ListAccessKeys"
                "iam:AttachUserPolicy",
                "iam:ListPolicyVersions",
                "iam:GetPolicy",
                "iam:CreatePolicy",
                "iam:DeletePolicy",
                "logs:DescribeLogGroups",
            ],
            "Resource": [
                "arn:aws:logs:AWS_REGION:AWS_ACCOUNT_ID:log-group:*",
                "arn:aws:iam::AWS_ACCOUNT_ID:user/CloudAEyeServiceAccount",
                "arn:aws:iam::AWS_ACCOUNT_ID:policy/CloudAEyeServiceAccountPolicy"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "cloudformation:CreateStack",
                "cloudformation:DescribeStacks"
            ],
            "Resource": "arn:aws:cloudformation:AWS_REGION:AWS_ACCOUNT_ID:stack/CloudAEyeServiceAccountStack/*"
        }
    ]
}