Skip to content

Create a Logs Service

Overview

CloudAEye SaaS provides centralized logs service. It is a managed service that makes logs collection, parsing, storing and analytics easier. It enables operating and analyzing logs from distributed cloud applications at scale in a cost effective manner. It is built on highly scalable infrastructure.

This topic describes how user may create this service.

Videos

Create

From left navigation menu, select Logs Service. You will see list of logs services that are already created. The table will be empty if there are no logs services being created.

To create a new logs service, click on Create on the top right corner. A new form will appear under Logs-Service > Create.

LogsServiceCreate

Provide the following informaiton:

  • Name: Name of the logs service. This is usually an alpha-numeric string. For example, "sockshop-logs".
  • Group: Pick the Service Group for the new service. You may create a new Service Group if necessary. Learn more about Service Groups here.
  • Expected Usage: Specify expected amount amount of logs in GB per day. The storage disks are provisioned based on this information and retention period you choose in the next option.
  • Retention: Choose the # of days the logs are retained in Hot storage.
  • Encryption at REST: Optional true/false field indicating if you wish to turn on data encryption at REST for your domain. The feature uses AWS Key Management Service (AWS KMS) to store and manage your encryption keys and the Advanced Encryption Standard algorithm with 256-bit keys (AES-256) to perform the encryption.
  • Instance Type: Choose from drop-down option of Small, Medium, Large and Custom.
    • Small: This option creates a single instance domain with small instance type. Choose this option for 0 to 1 GB of logs per day with retention period of 3 days.
    • Medium: This option creates a single instance domain with medium instance type. Choose this option for 1-2 GB of logs per day with retention period of 3 or 7 days.
    • Large: This option creates a single instance domain with large instance type. Choose this option for 2-3 GB of logs per day with retention period of 3 or 7 days.
    • Extral Large: This option creates a single instance domain with extra large instance type. Choose this option for 3-5 GB of logs per day with retention period of 3 or 7 days.
    • Custom: This option creates a domain with 2 or more master nodes and 2 or more data nodes. Choose this option for any production setup with 5+ GB of logs per day with retention period of 7 days or more.

Click Submit to create the logs service.

Custom Domain

Custom option provides a few more features.

LogsServiceCreateCustom

  • Enable UltraWarm: This allows you to use UltraWarm storage. This is a more cost-effective way of keeping large amount of log data in retention for a longer period of time without paying the high cost of “Hot” storage. CloudAEye provides policy-based data aging that enables you to move data from “Hot” to “UltraWarm” to “Cold” and eventually delete the data.
  • Enable Cold Storage: This option keeps log data in Cold storage. This option provides a cheaper option to store logs data in archive format for a longer period. You may re-hydrate the logs from Cold to Hot at any time.
  • Node Type: Choose the type of UltraWarm node type you want to use. Default is ultrawarm1.medium. Please note that ultrawarm1.large instance type is 11x more expensive than ultrawarm1.medium. One ultrawarm1.medium may handle 1.5 TB of logs while ultrawarm1.large can handle 20 TB of logs.
  • Node Count: Choose number of UltraWarm nodes. Minimum is 2.
  • Master Node Choice: Choose the type of master node you want in the domain. Default option is c5.xlarge.
  • Master Node Count: Choose the number of master nodes in the domain. Minimum is 2.
  • Data Node Count: Choose the number of data nodes in the domain. Minimum is 2.
  • Data Node Choice: Choose the type of data node you want in the domain. Default option is r5.xlarge.

List All Logs Service(s)

To see list of already created logs services, select Logs Service from the left navigation.

LogsServiceList

You will see a list of logs services that are already created. You may filter the logs service based on Region, Service Group and Status.

Details of a Logs Service

To see details of a specific logs service, click on the name of the logs service under Service Name column. A new page will appear under Logs-Service > <Name-of-logs-service>.

This page shows the following information:

Summary

LogsServiceSummary

  • Name: Name of the logs service
  • Region: Name of the region where your service is running. Public cloud providers have the concept of a region, which is a physical locaiton around the world that consists of multilple, isolated and physically seperate availability zone within a geographic area. Availability zone is a logical grouping of data centers.
  • Created: Date when this service was created.
  • Group Name: Name of the Service Group.
  • Elasticsearch URL: URL of the domain.
  • Status: Health status of the domain.
  • Kibana Dashboard: Dashboard to view and search the logs.

Labels

LogsServiceLabels

  • Labels: Key/Value information associated with this service. Use Create Lable button to add a new label.

Applications Attached to the Logs Service

This section shows all the application associated to the logs service.

LogsServiceApps

  1. Click on the pencil icon to edit the application configuration.
  2. Click on the 2nd icon to see the install instructions.
  3. Click on the 3rd icon to view the dashboards associated with logs service.
  4. Click on the trash icon to delete this application associated with this logs service.

Next Steps

User may setup logs streaming from application environment to CloudAEye SaaS for centralized logging and viewing.