Create a Logs Service¶
Overview¶
CloudAEye SaaS provides centralized logs service. It is a managed service that makes logs collection, parsing, storing and analytics easier. It enables operating and analyzing logs from distributed cloud applications at scale in a cost effective manner. It is built on highly scalable infrastructure.
This topic describes how user may create this service.
Videos¶
- Tutorial: Create a Logs Service
- Log Management Overview
Create¶
From left navigation menu, select Logs Service
.
You will see list of logs services that are already created. The table will be empty if there are no logs services being created.
To create a new logs service, click on Create
on the top right corner. A new form will appear under Logs-Service > Create
.
Provide the following informaiton:
- Name: Name of the logs service. This is usually an alpha-numeric string. For example, "sockshop-logs".
- Group: Pick the
Service Group
for the new service. You may create a newService Group
if necessary. Learn more aboutService Groups
here. - Expected Usage: Specify expected amount amount of logs in GB per day. The storage disks are provisioned based on this information and retention period you choose in the next option.
- Retention: Choose the # of days the logs are retained in Hot storage.
- Encryption at REST: Optional true/false field indicating if you wish to turn on data encryption at REST for your domain. The feature uses AWS Key Management Service (AWS KMS) to store and manage your encryption keys and the Advanced Encryption Standard algorithm with 256-bit keys (AES-256) to perform the encryption.
- Instance Type: Choose from drop-down option of
Small
,Medium
,Large
andCustom
.Small
: This option creates a single instance domain with small instance type. Choose this option for 0 to 1 GB of logs per day with retention period of 3 days.Medium
: This option creates a single instance domain with medium instance type. Choose this option for 1-2 GB of logs per day with retention period of 3 or 7 days.Large
: This option creates a single instance domain with large instance type. Choose this option for 2-3 GB of logs per day with retention period of 3 or 7 days.Extral Large
: This option creates a single instance domain with extra large instance type. Choose this option for 3-5 GB of logs per day with retention period of 3 or 7 days.Custom
: This option creates a domain with 2 or more master nodes and 2 or more data nodes. Choose this option for any production setup with 5+ GB of logs per day with retention period of 7 days or more.
Click Submit
to create the logs service.
Custom Domain¶
Custom
option provides a few more features.
- Enable UltraWarm: This allows you to use
UltraWarm
storage. This is a more cost-effective way of keeping large amount of log data in retention for a longer period of time without paying the high cost of “Hot” storage. CloudAEye provides policy-based data aging that enables you to move data from “Hot” to “UltraWarm” to “Cold” and eventually delete the data. - Enable Cold Storage: This option keeps log data in
Cold
storage. This option provides a cheaper option to store logs data in archive format for a longer period. You may re-hydrate the logs fromCold
toHot
at any time. - Node Type: Choose the type of
UltraWarm
node type you want to use. Default isultrawarm1.medium
. Please note thatultrawarm1.large
instance type is 11x more expensive thanultrawarm1.medium
. Oneultrawarm1.medium
may handle 1.5 TB of logs whileultrawarm1.large
can handle 20 TB of logs. - Node Count: Choose number of
UltraWarm
nodes. Minimum is 2. - Master Node Choice: Choose the type of master node you want in the domain. Default option is
c5.xlarge
. - Master Node Count: Choose the number of master nodes in the domain. Minimum is 2.
- Data Node Count: Choose the number of data nodes in the domain. Minimum is 2.
- Data Node Choice: Choose the type of data node you want in the domain. Default option is
r5.xlarge
.
List All Logs Service(s)¶
To see list of already created logs services, select Logs Service
from the left navigation.
You will see a list of logs services that are already created. You may filter the logs service based on Region
, Service Group
and Status
.
Details of a Logs Service¶
To see details of a specific logs service, click on the name of the logs service under Service Name
column.
A new page will appear under Logs-Service > <Name-of-logs-service>
.
This page shows the following information:
Summary¶
- Name: Name of the logs service
- Region: Name of the region where your service is running. Public cloud providers have the concept of a region, which is a physical locaiton around the world that consists of multilple, isolated and physically seperate availability zone within a geographic area. Availability zone is a logical grouping of data centers.
- Created: Date when this service was created.
- Group Name: Name of the
Service Group
. - Elasticsearch URL: URL of the domain.
- Status: Health status of the domain.
- Kibana Dashboard: Dashboard to view and search the logs.
Labels¶
- Labels: Key/Value information associated with this service. Use
Create Lable
button to add a new label.
Applications Attached to the Logs Service¶
This section shows all the application associated to the logs service.
- Click on the pencil icon to edit the application configuration.
- Click on the 2nd icon to see the install instructions.
- Click on the 3rd icon to view the dashboards associated with logs service.
- Click on the trash icon to delete this application associated with this logs service.
Next Steps¶
User may setup logs streaming from application environment to CloudAEye SaaS for centralized logging and viewing.