AWS Kinesis¶
Overview¶
Amazon Kinesis makes it easy to collect, process, and analyze video and data streams in real time.
Prerequisites¶
- User must setup a logs service for the same application.
- Logs service must be added to a service group.
Account Access¶
This step is optional. We strongly recommend that you allow us to retreive info about your services because it will greatly minimize the chances of making a mistake
Configuration for Kinesis Logs Collection¶
User must select AWS Services field in application creation form and select Kinesis service from AWS Service dropdown.
Provide the s3 bucket associated with the Kinesis
Configuration for Kinesis Metrics Collection¶
User must select AWS Services field in application creation form and select Kinesis service from AWS Service dropdown.
Provide required dimensions and metrics to be collected.
If you have allow us to retreive service details from your account, the dimensions will shown as a dropdown.
Detailed description of metrics and dimension can be viewed at Kinesis metrics and dimensions
Supported Kinesis Dimensions¶
Kinesis provides below dimensions on which metrics are collected.
| Dimension | Description |
|---|---|
| StreamName | The name of the Kinesis stream. All available statistics are filtered by StreamName |
| ShardId | Unique identifier for an individual shard within a Kinesis data stream |
Supported Kinesis Metrics¶
SNS provides below metrics to be collected.
| Metrics | Description |
|---|---|
| GetRecords.Bytes | The number of bytes retrieved from the Kinesis stream, measured over the specified time period |
| GetRecords.IteratorAge | This metric is deprecated. Use GetRecords.IteratorAgeMilliseconds |
| GetRecords.IteratorAgeMilliseconds | The age of the last record in all GetRecords calls made against a Kinesis stream, measured over the specified time period. Age is the difference between the current time and when the last record of the GetRecords call was written to the stream |
| GetRecords.Latency | The time taken per GetRecords operation, measured over the specified time period |
| GetRecords.Records | The number of records retrieved from the shard, measured over the specified time period |
| GetRecords.Success | The number of successful GetRecords operations per stream, measured over the specified time period |
| IncomingBytes | The number of bytes successfully put to the Kinesis stream over the specified time period |
| IncomingRecords | The number of records successfully put to the Kinesis stream over the specified time period |
| PutRecord.Bytes | The number of bytes put to the Kinesis stream using the PutRecord operation over the specified time period |
| PutRecord.Latency | The time taken per PutRecord operation, measured over the specified time period |
| PutRecord.Success | The number of successful PutRecord operations per Kinesis stream, measured over the specified time period |
| PutRecords.Bytes | The number of bytes put to the Kinesis stream using the PutRecords operation over the specified time period |
| PutRecords.Latency | The time taken per PutRecords operation, measured over the specified time period |
| PutRecords.Records | This metric is deprecated. Use PutRecords.SuccessfulRecords |
| PutRecords.Success | The number of PutRecords operations where at least one record succeeded, per Kinesis stream, measured over the specified time period |
| PutRecords.TotalRecords | The total number of records sent in a PutRecords operation per Kinesis data stream, measured over the specified time period |
| PutRecords.SuccessfulRecords | The number of successful records in a PutRecords operation per Kinesis data stream, measured over the specified time period |
| PutRecords.FailedRecords | The number of records rejected due to internal failures in a PutRecords operation per Kinesis data stream, measured over the specified time period |
| PutRecords.ThrottledRecords | The number of records rejected due to throttling in a PutRecords operation per Kinesis data stream, measured over the specified time period |
| ReadProvisionedThroughputExceeded | The number of GetRecords calls throttled for the stream over the specified time period. The most commonly used statistic for this metric is Average |
| SubscribeToShard.RateExceeded | This metric is emitted when a new subscription attempt fails because there already is an active subscription by the same consumer or if you exceed the number of calls per second allowed for this operation |
| SubscribeToShard.Success | This metric records whether the SubscribeToShard subscription was successfully established. The subscription only lives for at most 5 minutes. Therefore, this metric gets emitted at least once every 5 minutes |
| SubscribeToShardEvent.Bytes | The number of bytes received from the shard, measured over the specified time period |
| SubscribeToShardEvent.MillisBehindLatest | The difference between the current time and when the last record of the SubscribeToShard event was written to the stream |
| SubscribeToShardEvent.Records | The number of records received from the shard, measured over the specified time period |
| SubscribeToShardEvent.Success | This metric is emitted every time an event is published successfully. It is only emitted when there's an active subscription |
| WriteProvisionedThroughputExceeded | The number of records rejected due to throttling for the stream over the specified time period |
| IncomingBytes | The number of bytes successfully put to the shard over the specified time period. This metric includes bytes from PutRecord and PutRecords operations |
| IncomingRecords | The number of records successfully put to the shard over the specified time period. This metric includes record counts from PutRecord and PutRecords operations |
| IteratorAgeMilliseconds | The age of the last record in all GetRecords calls made against a shard, measured over the specified time period |
| OutgoingBytes | The number of bytes retrieved from the shard, measured over the specified time period |
| OutgoingRecords | The number of records retrieved from the shard, measured over the specified time period |
| ReadProvisionedThroughputExceeded | The number of GetRecords calls throttled for the shard over the specified time period |
| WriteProvisionedThroughputExceeded | The number of records rejected due to throttling for the shard over the specified time period |