COST 3: How do you monitor usage and cost?

Establish policies and procedures to monitor and appropriately allocate your costs. This allows you to measure and improve the cost efficiency of this workload.

Resources

Managing AWS Cost and Usage Reports
AWS tagging strategies
Analyzing your costs with Cost Explorer
Analyzing your costs with AWS Budgets

Best Practices:

• Configure detailed information sources: Configure the AWS Cost and Usage Report, and Cost Explorer hourly granularity, to provide detailed cost and usage information. Configure your workload to have log entries for every delivered business outcome.

• Identify cost attribution categories: Identify organization categories that could be used to allocate cost within your organization.

• Establish organization metrics: Establish the organization metrics that are required for this workload. Example metrics of a workload are customer reports produced or web pages served to customers.

• Configure billing and cost management tools: Configure AWS Cost Explorer and AWS Budgets inline with your organization policies.

• Add organization information to cost and usage: Define a tagging schema based on organization, and workload attributes, and cost allocation categories. Implement tagging across all resources. Use Cost Categories to group costs and usage according to organization attributes.

• Allocate costs based on workload metrics: Allocate the workload's costs by metrics or business outcomes to measure workload cost efficiency. Implement a process to analyze the AWS Cost and Usage Report with Amazon Athena, which can provide insight and charge back capability.

Improvement Plan

Configure detailed information sources

Configure the cost and usage report : Using the billing console, configure at least one cost and usage report. Configure a report with hourly granularity that includes all identifiers and resource IDs. You can also create other reports with different granularities to provide higher-level summary information.
AWS Account Setup

Configure hourly granularity in cost explorer : Using the billing console, enable Hourly and Resource Level Data. NOTE: There will be associated costs with enabling this feature, please refer to the pricing.
AWS Account Setup
AWS Cost Management Pricing

Configure application logging : Ensure that your application logs each business outcome that it delivers so it can be tracked and measured. Ensure that the granularity of this data is at least hourly to be matched with the cost and usage data. Refer to the Operational Excellence pillar for more detail on logging and monitoring.
Well-Architected Operational Excellence Pillar

Identify cost attribution categories

Define your organization categories : Meet with stakeholders to define categories that reflect your organization's structure and requirements. These will directly map to the structure of existing financial categories, such as business unit, budget, cost center, or department. Look at the outcomes the cloud delivers for your business, such as training or education, as these are also organization categories. Multiple categories can be assigned to a resource, and a resource can be in multiple different categories, so define as many categories as needed.

Define your functional categories : Meet with stakeholders to define categories that reflect the functions that you have within your business. This may be the workload or application names, and the type of environment, such as production, testing, or development. Multiple categories can be assigned to a resource, and a resource can be in multiple different categories, so define as many categories as needed.

Establish organization metrics

Define workload outcomes : Meet with the stakeholders in the business and define the outcomes for the workload. These are a primary measure of customer usage and must be business metrics and not technical metrics. There should be a small number of high-level metrics (less than five) per workload. If the workload produces multiple outcomes for different use cases, then group them into a single metric.

Define workload component outcomes : Optionally, if you have a large and complex workload, or can easily break your workload into components (such as microservices) with well-defined inputs and outputs, define metrics for each component. The effort should reflect the value and cost of the component. Start with the largest components and work towards the smaller components.

Configure billing and cost management tools

Create a Cost Optimization group : Configure your account and create a group that has access to the required Cost and Usage reports. This group must include representatives from all teams that own or manage an application. This ensures that every team has access to their cost and usage information.
Well-Architected Labs - AWS Account Setup

Configure AWS Budgets : Configure AWS Budgets on all accounts for your workload. Set a budget for the overall account spend, and a budget for the workload by using tags.
Well-Architected Labs: Cost and Governance Usage

Configure AWS Cost Explorer : Configure AWS Cost Explorer for your workload and accounts. Create a dashboard for the workload that tracks overall spend, and key usage metrics for the workload.
Well-Architected Labs: Billing Visualization

Configure advanced tooling : Optionally, you can create custom tooling for your organization that provides additional detail and granularity. You can implement advanced analysis capability using Amazon Athena, and dashboards using Amazon QuickSight.
Well-Architected Labs: Cost and Usage Analysis
Well-Architected Labs: Cost and Usage Visualization

Add organization information to cost and usage

Define a tagging schema : Gather all stakeholders from across your business to define a schema. This typically includes technical, financial, and management people. Define a list of tags that all resources must have, as well as a list of tags that resources should have. Ensure that the tag names and values are consistent across your organization.
AWS tagging strategies

Tag resources : Using your defined cost attribution categories, place tags on all resources in your workloads according to the categories. Use tools such as the CLI, Tag Editor, or Systems Manager, to increase efficiency.
AWS tagging strategies

Implement Cost Categories : You can create Cost Categories without implementing tagging, Cost Categories use the existing cost and usage dimensions. Create category rules from your schema and implement it into cost categories.
AWS Cost Categories

Automate tagging : To ensure that you maintain high levels of tagging across all resources, automate tagging so that resources are automatically tagged when they are created. Use the features within the service, or services such as AWS CloudFormation, to ensure that resources are tagged when created. You can also create a custom microservice that scans the workload periodically and removes any resources that are not tagged, which is ideal for test and development environments.
AWS CloudFormation Resource Tag
Amazon EC2 and Amazon EBS add support for tagging resources upon creation

Monitor and report on tagging : To ensure that you maintain high levels of tagging across your organization, report and monitor the tags across your workloads. You can use AWS Cost Explorer to view the cost of tagged and untagged resources, or use services such as Tag Editor. Regularly review the number of untagged resources and take action to add tags until you reach the desired level of tagging.

Allocate costs based on workload metrics

Allocate costs to workload metrics : Using the defined metrics and tagging configured, create a metric that combines the workload output and workload cost. Use the analytics services such as Athena and QuickSight to create an efficiency dashboard for the overall workload, and any components.