REL 8: How do you implement change?

Controlled changes are necessary to deploy new functionality, and to ensure that the workloads and the operating environment are running known software and can be patched or replaced in a predictable manner. If these changes are uncontrolled, then it makes it difficult to predict the effect of these changes, or to address issues that arise because of them.

Resources

AWS Summit 2019: CI/CD on AWS
What Is AWS CodePipeline?
What Is CodeDeploy?
Overview of a Blue/Green Deployment
Deploying Serverless Applications Gradually
The Amazon Builders' Library: Ensuring rollback safety during deployments
The Amazon Builders' Library: Going faster with continuous delivery
Well-Architected lab: Level 300: Testing for Resiliency of EC2 RDS and S3
CanaryRelease
AWS Marketplace: products that can be used to automate your deployments
APN Partner: partners that can help you create automated deployment solutions

Best Practices:

• Use runbooks for standard activities such as deployment: Runbooks are the predefined steps used to achieve specific outcomes. Use runbooks to perform standard activities, whether done manually or automatically. Examples include deploying a workload, patching it, or making DNS modifications.

• Integrate functional testing as part of your deployment: Functional tests are run as part of automated deployment. If success criteria are not met, the pipeline is halted or rolled back.

• Integrate resiliency testing as part of your deployment: Resiliency tests (as part of chaos engineering) are run as part of the automated deployment pipeline in a pre-prod environment.

• Deploy using immutable infrastructure: This is a model that mandates that no updates, security patches, or configuration changes happen in-place on production workloads. When a change is needed, the architecture is built onto new infrastructure and deployed into production.

• Deploy changes with automation: Deployments and patching are automated to eliminate negative impact.

Improvement Plan

Use runbooks for standard activities such as deployment

Enable consistent and prompt responses to well understood events by documenting procedures in runbooks.
AWS Well-Architected Framework: Concepts: Runbook

Use the principle of infrastructure as code to define your infrastructure: By using AWS CloudFormation (or a trusted third party) to define your infrastructure, you can use version control software to version and track changes.

• Use AWS CloudFormation (or a trusted third-party provider) to define your infrastructure.
  What is AWS CloudFormation?
• Create templates that are singular and decoupled, using good software design principles.
  • Determine the permissions, templates, and responsible parties for implementation
    Controlling access with AWS Identity and Access Management
  • Use source control, like AWS CodeCommit or a trusted third-party tool, for version control.
    What is AWS CodeCommit?

Integrate functional testing as part of your deployment

Integrate functional testing as part of your deployment: Functional tests are run as part of automated deployment. If success criteria are not met, the pipeline is halted or rolled back.

• Invoke AWS CodeBuild during the ‘Test Action’ of your software release pipelines modeled in AWS CodePipeline. This capability enables you to easily run a variety of tests against your code, such as unit tests, static code analysis and integration tests.
  AWS CodePipeline Adds Support for Unit and Custom Integration Testing with AWS CodeBuild
• Use AWS Marketplace solutions for executing automated tests as part of your software delivery pipeline.
  Software test automation

Integrate resiliency testing as part of your deployment

Integrate resiliency testing as part of your deployment: Use Chaos Engineering, the discipline of experimenting on a workload in order to build confidence in the workload’s capability to withstand turbulent conditions in production.

• Resiliency tests inject faults or resource degradation to assess that your workload responds with its designed resilience
  Well-Architected lab: Level 300: Testing for Resiliency of EC2 RDS and S3
• These tests can be run regularly in pre-production environments in automated deployment pipelines.
• They should also be run in production, as part of scheduled game days.
• Using Chaos Engineering principles, propose hypotheses about how your workload will perform under various impairments, then test your hypotheses using resiliency testing.
  Principles of Chaos Engineering

Deploy using immutable infrastructure

Deploy using immutable infrastructure: Deploy using an immutable infrastructure approach such as Blue/Green or Canary Deployment
Overview of a Blue/Green Deployment
Deploying Serverless Applications Gradually
CanaryRelease

Deploy changes with automation

Automate your deployment pipeline: Deployment pipelines allow you to invoke automated testing and detection of anomalies, and either halt the pipeline at a certain step before production deployment, or automatically roll back a change.
The Amazon Builders' Library: Ensuring rollback safety during deployments
The Amazon Builders' Library: Going faster with continuous delivery

• Use AWS CodePipeline (or a trusted third-party product) to define and execute your pipelines.
  • Configure the pipeline to start when a change is committed to your code repository.
    What is AWS CodePipeline?
  • Use Amazon Simple Notification Service (SNS) and Amazon Simple Email Service (SES) to send notifications about problems in the pipeline or integrate with a team chat tool, like Amazon Chime.
    What is Amazon Simple Notification Service?
    What is Amazon SES?
    What is Amazon Chime?
    Automate chat messages with webhooks.