Many customers have built live streaming workflows using AWS Elemental MediaStore as a high performant live video origin in the cloud. For example, FOX recently used MediaStore as a low latency origin for its AWS-enabled production workflow for Super Bowl LIV.
In a previous blog post, I explained how to use MediaStore access logs to build monitoring dashboards. As MediaStore now supports Amazon CloudWatch metrics, this post describes how you can use CloudWatch metrics to build live streaming dashboards to monitor MediaStore performance.
The CloudWatch metrics used in this solution include RequestCount, 4xxErrorCount, 5xxErrorCount, BytesUploaded, BytesDownloaded,TotalTime and TurnaroundTime.
Set up
Review the following resources to familiarize yourself with AWS Elemental MediaStore, Amazon CloudWatch metrics and access logs, and how to send live video to MediaStore.
- AWS Elemental MediaStore User Guide: Enabling access logging for a container
- AWS Elemental MediaStore User Guide: Monitoring AWS Elemental MediaStore with Amazon CloudWatch metrics
- AWS Media Blog: How to Send Live Video to AWS Elemental MediaStore
Solution walkthrough
First, you will create a live channel using MediaStore as an origin and start a player to play the live stream directly from MediaStore to generate egress traffic. Next, you attach the MediaStore metric policy to your container and make sure the folder names are match your configuration correctly.
Prerequisites
For this walkthrough, you should have the following prerequisites:
- An AWS account
- An Encoder such as AWS Elemental Live that can push live stream to MediaStore, or an AWS Elemental MediaLive service to stream to MediaStore
- An HLS player that can play HLS streams
MediaStore and Live Stream Setup
Choose the MediaStore service from the management console, and create a container named ‘mediastore_live’. Under Containers, select the ‘mediastore_live’ container to display the container details property interface.
To play live stream directly from MediaStore and enable HTTP/S access, scroll to the Container policy section and click Edit policy. Copy and paste the following code in the editing text box. Be sure to modify the Resource ARN and region to match your container settings. Once finished, click Save.
| { “Version” : “2012-10-17”, “Statement” : [ { “Sid” : “MediaStoreFullAccess”, “Effect” : “Allow”, “Principal” : { “AWS” : “arn:aws:iam::xxxxxxxxxxxx:root” }, “Action” : “mediastore:*”, “Resource” : “arn:aws:mediastore:us-west-2:xxxxxxxxxxxx:container/mediastore_live/*”, “Condition” : { “Bool” : { “aws:SecureTransport” : “true” } } }, { “Sid” : “PublicReadOverHttpOrHttps”, “Effect” : “Allow”, “Principal” : “*”, “Action” : [ “mediastore:GetObject”, “mediastore:DescribeObject” ], “Resource” : “arn:aws:mediastore:us-west-2:xxxxxxxxxxxx:container/mediastore_live/*”, “Condition” : { “Bool” : { “aws:SecureTransport” : [ “true”, “false” ] } } } ] } |
To enable MediaStore to send CloudWatch metrics, scroll down to Metric policy, and click Edit metric policy. Copy and paste the following metric policy and click Save button.
|
{ “ContainerLevelMetrics”: “DISABLED”, “MetricPolicyRules”: [ { “ObjectGroup”: “msmetric1/”, “ObjectGroupName”: “msmetric1” }, { “ObjectGroup”: “msmetric2/”, “ObjectGroupName”: “msmetric2” } ] } |
Note the prefix name mnmetric1 and msmetric2. These prefix names are used in the AWS Elemental MediaLive configuration in the output path.
Scroll to the Container CORS policy section and click Edit CORS policy. Copy and paste the following CORS policy and click Save.
| [ { “AllowedHeaders”: [ “*” ], “AllowedMethods”: [ “GET”, “HEAD” ], “AllowedOrigins”: [ “*” ], “ExposeHeaders”: [ “*” ], “MaxAgeSeconds”: 3000 } ] |
Next, configure a live stream from AWS Elemental MediaLive to AWS Elemental MediaStore. Use the following URLs to configure HLS group destination A and HLS group destination B in MediaLive.
HLS group destination A: mediastoressl://<MediaStore data end point URL>/msmetric1/master, HLS group destination B: mediastoressl://<MediaStore data end point URL>/msmetric1/master
To learn more, review this blog post: How to Send Live Video to AWS Elemental MediaStore
Stream Ingest Dashboard
One of the most important aspects when monitoring a live stream from an origin perspective is the health status of the stream ingest into the origin. Now you will create dashboards by using MediaStore ingest CloudWatch metrics.
From your AWS Account Console, choose the CloudWatch service. Select Metrics in the left-side menu. Under the All metrics tab, choose MediaStore and then select Data Transfer Metrics by Object Group.
Under the ContainerName column, choose mediastore_live container. Under the ObjectGroupName column, choose msmetric1. For Metric Name, choose BytesUploaded.
Repeat this step and select the following:
- ContainerName: mediastore_live
- ObjectGroupName: msmetric2
- Metric Name: BytesUploaded
Select the Graphed metrics tab. Change the Statistic value to Average and change the Period value to 5 Minutes. On the top left corner of the diagram, change the name to Bytes Upload.
Click the Actions button and select Add to dashboard from the dropdown menu. In the Add to dashboard popup that appears, choose Create new under the Select a dashboard section. Name the dashboard mediastore_live_cw. Then click Add to dashboard at the bottom of the window.
Next, choose Metrics from left menu. Select the All metrics tab, click MediaStore followed by Request Metrics by Object Group. Input the metrics referenced in the following table:
|
ContainerName |
ObjectGroupName | RequestType |
Metric Name |
| mediastore_live | msmetric1 | DeleteRequests | 5xxErrorCount |
| mediastore_live | msmetric1 | PutRequests | 5xxErrorCount |
| mediastore_live | msmetric1 | DeleteRequests | 4xxErrorCount |
| mediastore_live | msmetric1 | PutRequests | 4xxErrorCount |
| mediastore_live | msmetric2 | DeleteRequests | 5xxErrorCount |
| mediastore_live | msmetric2 | PutRequests | 5xxErrorCount |
| mediastore_live | msmetric2 | DeleteRequests | 4xxErrorCount |
| mediastore_live | msmetric2 | PutRequests | 4xxErrorCount |
Choose Graphed metrics tab. Set the Statistic value to Sum and set the Period value to 5 Minutes.
To show a value of 0 in the dashboard when there are no errors, choose Math expression in the header of the table. Select All Functions and select FILL. This will add a math function to the top of the table.
Choose the Graph options tab, and, in Widget type section, select Number. Input ‘Ingest Error’ in the top left corner of the dashboard. Click the Actions button and select Add to dashboard from the dropdown menu. In the Add to dashboard dialog that appears, under the Select a dashboard section, choose mediastore_live_cw from the dropdown menu. When finished, click Add to dashboard at the bottom of the window.
This dashboard will help keep track of ingest errors. An increasing number of 4xx and 5xx errors should warrant a further investigation on stream ingest.
Using a similar process, you can create dashboard to monitor MediaStore PUT requests counts and delete request counts. These number should be relatively similar between pipelines and stay stable.
The last metrics for MediaStore ingest monitoring are the PutRequest TotalTime and PutRequest TurnaroundTime. In this example, you can set the Statistic value to p99, and set the Period value to 1 Minute.
Next, rearrange your dashboard to have a clear view of the MediaStore ingest operations based on CloudWatch metrics.
Following the same principles, you can also monitor MediaStore egress performance. Choose the following MediaStore metrics to graph your egress monitoring dashboard: BytesDownloaded, GetRequests TotalTime, GetRequests TurnaroundTime, GetRquest RequestCount and GetRequests 4xxErrorCount.
Unify your dashboards into one view with the ingest dashboard to the left and the egress dashboard to the right. We now have a full high-level view of your origin’s performance.
Clean up
To avoid incurring future charges, delete the MediaStore container and MediaLive services you created as part of this example.
Conclusion
This blog explained how to use CloudWatch metrics to create live streaming dashboards to track MediaStore performance and operations. Using CloudWatch metrics to monitor MediaStore origin performance provides comprehensive information on both ingress and egress operations. When MediaStore is used as a live streaming origin, these metrics help you create custom dashboards to monitor your MediaStore services, assess operational health and performance, and easily to identify potential issues in the workflow. Moreover, with support for object groups, you can pull unique metrics for each channel within the same MediaStore container, as long as each channel has a unique prefix.
To learn more about MediaStore dashboarding, check out this blog: Create a Live Streaming CloudWatch Dashboard using AWS Elemental MediaStore Access Logs