OPS 5: How do you reduce defects, ease remediation, and improve flow into production?

Adopt approaches that improve flow of changes into production, that enable refactoring, fast feedback on quality, and bug fixing. These accelerate beneficial changes entering production, limit issues deployed, and enable rapid identification and remediation of issues introduced through deployment activities.

Resources

CI/CD for Serverless Applications on AWS
Design with Ops in Mind
AWS Developer Tools

Best Practices:

Improvement Plan

Use version control

  • Use version control: Maintain assets in version controlled repositories. Doing so supports tracking changes, deploying new versions, detecting changes to existing versions, and reverting to prior versions (for example, rolling back to a known good state in the event of a failure). Integrate the version control capabilities of your configuration management systems into your procedures.
    Introduction to AWS CodeCommit
    What is AWS CodeCommit?
  • Test and validate changes

  • Test and validate changes: Changes should be tested and the results validated at all lifecycle stages (for example, development, test, and production). Use testing results to confirm new features and mitigate the risk and impact of failed deployments. Automate testing and validation to ensure consistency of review, to reduce errors caused by manual processes, and reduce the level of effort.
    What is AWS CodeBuild?
    Local build support for AWS CodeBuild
  • Use configuration management systems

  • Use configuration management systems: Use configuration management systems to track and implement changes, to reduce errors caused by manual processes, and reduce the level of effort.
    Infrastructure configuration management
    AWS Config
    What is AWS Config?
    Introduction to AWS CloudFormation
    What is AWS CloudFormation?
    AWS OpsWorks
    What is AWS OpsWorks?
    Introduction to AWS Elastic Beanstalk
    What is AWS Elastic Beanstalk?
  • Use build and deployment management systems

  • Use build and deployment management systems: Use build and deployment management systems to track and implement change, to reduce errors caused by manual processes, and reduce the level of effort. Fully automate the integration and deployment pipeline from code check-in through build, testing, deployment, and validation. This reduces lead time, enables increased frequency of change, and reduces the level of effort.
    What is AWS CodeBuild?
    Continuous integration best practices for software development
    Slalom: CI/CD for serverless applications on AWS
    Introduction to AWS CodeDeploy - automated software deployment with Amazon Web Services
    What is AWS CodeDeploy?
  • Perform patch management

  • Patch management: Patch systems to remediate issues, to gain desired features or capabilities, and to remain compliant with governance policy and vendor support requirements. In immutable systems, deploy with the appropriate patch set to achieve the desired result. Automate the patch management mechanism to reduce the elapsed time to patch, to reduce errors caused by manual processes, and reduce the level of effort to patch.
    AWS Systems Manager Patch Manager
  • Share design standards

  • Share design standards: Share existing best practices, design standards, checklists, operating procedures, and guidance and governance requirements across teams to reduce complexity and maximize the benefits from development efforts. Ensure that procedures exist to request changes, additions, and exceptions to design standards to support continual improvement and innovation. Ensure that teams are aware of published content so that they can take advantage of content, and limit rework and wasted effort.
    Delegating access to your AWS environment
    Share an AWS CodeCommit repository
    Easy authorization of AWS Lambda functions
    Sharing an AMI with specific AWS accounts
    Speed template sharing with an AWS CloudFormation designer URL
    Using AWS Lambda with Amazon SNS
  • Implement practices to improve code quality

  • Implement practices to improve code quality: Implement practices to improve code quality to minimize defects and the risk of their being deployed. For example, test-driven development, pair programming, code reviews, and standards adoption.
  • Use multiple environments

  • Use multiple environments: Provide developers sandbox environments with minimized controls to enable experimentation. Provide individual development environments to enable work in parallel, increasing development agility. Implement more rigorous controls in the environments approaching production to allow developers necessary freedom for innovation. Use infrastructure as code and configuration management systems to deploy environments that are configured consistent with the controls present in production to ensure systems operate as expected when deployed. When environments are not in use, turn them off to avoid costs associated with idle resources (for example, development systems on evenings and weekends). Deploy production equivalent environments when load testing to enable valid results.
    What is AWS CloudFormation?
    How do I stop and start Amazon EC2 instances at regular intervals using AWS Lambda?
  • Make frequent, small, reversible changes

  • Make frequent, small, reversible changes: Frequent, small, and reversible changes reduce the scope and impact of a change. This eases troubleshooting, enables faster remediation, and provides the option to roll back a change. It also increases the rate at which you can deliver value to the business.
  • Fully automate integration and deployment

  • Use build and deployment management systems: Use build and deployment management systems to track and implement change, to reduce errors caused by manual processes, and reduce the level of effort. Fully automate the integration and deployment pipeline from code check-in through build, testing, deployment, and validation. This reduces lead time, enables increased frequency of change, and reduces the level of effort.
    What is AWS CodeBuild?
    Continuous integration best practices for software development
    Slalom: CI/CD for serverless applications on AWS
    Introduction to AWS CodeDeploy - automated software deployment with Amazon Web Services
    What is AWS CodeDeploy?