COST 3: How do you monitor usage and cost?

Establish policies and procedures to monitor and appropriately allocate your costs. This allows you to measure and improve the cost efficiency of this workload.

Resources

Managing AWS Cost and Usage Reports
AWS tagging strategies
Analyzing your costs with Cost Explorer
Analyzing your costs with AWS Budgets

Best Practices:

Improvement Plan

Configure detailed information sources

  • Configure the cost and usage report : Using the billing console, configure at least one cost and usage report. Configure a report with hourly granularity that includes all identifiers and resource IDs. You can also create other reports with different granularities to provide higher-level summary information.
    AWS Account Setup
  • Configure hourly granularity in cost explorer : Using the billing console, enable Hourly and Resource Level Data. NOTE: There will be associated costs with enabling this feature, please refer to the pricing.
    AWS Account Setup
    AWS Cost Management Pricing
  • Configure application logging : Ensure that your application logs each business outcome that it delivers so it can be tracked and measured. Ensure that the granularity of this data is at least hourly to be matched with the cost and usage data. Refer to the Operational Excellence pillar for more detail on logging and monitoring.
    Well-Architected Operational Excellence Pillar
  • Identify cost attribution categories

  • Define your organization categories : Meet with stakeholders to define categories that reflect your organization's structure and requirements. These will directly map to the structure of existing financial categories, such as business unit, budget, cost center, or department. Look at the outcomes the cloud delivers for your business, such as training or education, as these are also organization categories. Multiple categories can be assigned to a resource, and a resource can be in multiple different categories, so define as many categories as needed.
  • Define your functional categories : Meet with stakeholders to define categories that reflect the functions that you have within your business. This may be the workload or application names, and the type of environment, such as production, testing, or development. Multiple categories can be assigned to a resource, and a resource can be in multiple different categories, so define as many categories as needed.
  • Establish organization metrics

  • Define workload outcomes : Meet with the stakeholders in the business and define the outcomes for the workload. These are a primary measure of customer usage and must be business metrics and not technical metrics. There should be a small number of high-level metrics (less than five) per workload. If the workload produces multiple outcomes for different use cases, then group them into a single metric.
  • Define workload component outcomes : Optionally, if you have a large and complex workload, or can easily break your workload into components (such as microservices) with well-defined inputs and outputs, define metrics for each component. The effort should reflect the value and cost of the component. Start with the largest components and work towards the smaller components.
  • Configure billing and cost management tools

  • Create a Cost Optimization group : Configure your account and create a group that has access to the required Cost and Usage reports. This group must include representatives from all teams that own or manage an application. This ensures that every team has access to their cost and usage information.
    Well-Architected Labs - AWS Account Setup
  • Configure AWS Budgets : Configure AWS Budgets on all accounts for your workload. Set a budget for the overall account spend, and a budget for the workload by using tags.
    Well-Architected Labs: Cost and Governance Usage
  • Configure AWS Cost Explorer : Configure AWS Cost Explorer for your workload and accounts. Create a dashboard for the workload that tracks overall spend, and key usage metrics for the workload.
    Well-Architected Labs: Billing Visualization
  • Configure advanced tooling : Optionally, you can create custom tooling for your organization that provides additional detail and granularity. You can implement advanced analysis capability using Amazon Athena, and dashboards using Amazon QuickSight.
    Well-Architected Labs: Cost and Usage Analysis
    Well-Architected Labs: Cost and Usage Visualization
  • Add organization information to cost and usage

  • Define a tagging schema : Gather all stakeholders from across your business to define a schema. This typically includes technical, financial, and management people. Define a list of tags that all resources must have, as well as a list of tags that resources should have. Ensure that the tag names and values are consistent across your organization.
    AWS tagging strategies
  • Tag resources : Using your defined cost attribution categories, place tags on all resources in your workloads according to the categories. Use tools such as the CLI, Tag Editor, or Systems Manager, to increase efficiency.
    AWS tagging strategies
  • Implement Cost Categories : You can create Cost Categories without implementing tagging, Cost Categories use the existing cost and usage dimensions. Create category rules from your schema and implement it into cost categories.
    AWS Cost Categories
  • Automate tagging : To ensure that you maintain high levels of tagging across all resources, automate tagging so that resources are automatically tagged when they are created. Use the features within the service, or services such as AWS CloudFormation, to ensure that resources are tagged when created. You can also create a custom microservice that scans the workload periodically and removes any resources that are not tagged, which is ideal for test and development environments.
    AWS CloudFormation Resource Tag
    Amazon EC2 and Amazon EBS add support for tagging resources upon creation
  • Monitor and report on tagging : To ensure that you maintain high levels of tagging across your organization, report and monitor the tags across your workloads. You can use AWS Cost Explorer to view the cost of tagged and untagged resources, or use services such as Tag Editor. Regularly review the number of untagged resources and take action to add tags until you reach the desired level of tagging.
  • Allocate costs based on workload metrics

  • Allocate costs to workload metrics : Using the defined metrics and tagging configured, create a metric that combines the workload output and workload cost. Use the analytics services such as Athena and QuickSight to create an efficiency dashboard for the overall workload, and any components.