Cloud native Continuous Delivery
Download as PPTX, PDF1 like604 views
The document discusses cloud-native continuous delivery, focusing on microservices architecture, deployment pipelines, and the importance of automation in managing infrastructure. It emphasizes fast, reliable, and traceable deployment processes that align with DevOps principles and highlights practices like feature toggles and canary releases. The document also details metrics for evaluating delivery performance and stresses the need for independent and elastic deployment pipelines.
![• Regent's Rowing 8 by Jmf3333 [CC BY 3.0]
https://en.wikipedia.org/wiki/File:Regents_rowing.JPG
• Aquapark Aquacolors by Pantharei.2017 (Own work) [CC BY-SA 4.0]
https://commons.wikimedia.org/wiki/File:Aquapark_Aquacolors.jpg
• The Key of a Chamberlain by Niklitov [CC BY-SA 4.0]
https://commons.wikimedia.org/wiki/File:The_Key_of_a_Chamberlain_at_Kingdom_of_P
russia_Kalinigrad_Blindage_museum.JPG
• Beziers Fonseranes by Dedounet [CC BY-SA 1.0]
https://commons.wikimedia.org/wiki/File:Beziers_Fonseranes.jpg
Image attribution](https://image.slidesharecdn.com/cloudnativecontinuousdelivery2-180519084139/85/Cloud-native-Continuous-Delivery-89-320.jpg)
Cloud native Continuous Delivery
- 1. Cloud native Continuous Delivery Christian Deger, [email protected], @cdeger Continuous Lifecycle London, 17.05.2018
- 2. Cloud native Continuous Delivery • Cloud Native Computing Foundation • Container packaged • Dynamically managed • Microservices oriented • My opinion • Serverless, FaaS • Managed services
- 3. Cloud native Continuous Delivery • Bring changes into production • Fast • Reliable • Repeatable • Traceable • In order to • Get fast feedback • Lower risk
- 7. Microservices Speed Fast local decisions Scale the organization
- 8. Microservices Speed Fast local decisionsAutonomous teams Scale the organization
- 9. Microservices Speed Fast local decisionsAutonomous teams Scale the organization Loosely coupled
- 10. Microservices Speed Fast local decisionsAutonomous teams Strong boundaries Scale the organization Loosely coupled
- 11. Microservices Speed Fast local decisionsAutonomous teams Strong boundaries Scale the organization Independent deployable Loosely coupled
- 12. Microservices Speed Fast local decisionsAutonomous teams Strong boundaries Technology diversity Scale the organization Independent deployable Loosely coupled
- 18. Science and Continuous Delivery Forsgren, Nicole and Humble, Jez, The Role of Continuous Delivery in IT and Organizational Performance (October 27, 2015). Forsgren, N., J. Humble (2016). "The Role of Continuous Delivery in IT and Organizational Performance." In the Proceedings of the Western Decision Sciences Institute (WDSI) 2016, Las Vegas, NV. . Available at SSRN: http://ssrn.com/abstract=2681909 or http://dx.doi.org/10.2139/ssrn.2681909
- 20. Simplified deployment pipeline Application code in one repository per service.
- 21. Simplified deployment pipeline Application code in one repository per service. CI Deployment package as artifact.
- 22. Simplified deployment pipeline Application code in one repository per service. CI Deployment package as artifact. CD Deliver package to production
- 23. Code hosting is commodity Application code in one repository per service. CI Deployment package as artifact. CD Deliver package to production
- 24. • GitHub, GitLab, Bitbucket, … • Increased developer productivity • Ecosystem: Apps and integrations • Security • Account Management, SSO, MFA • git-secrets • Backup using clone/fetch Code hosting: Managed over self-hosted
- 25. One tool? CI CI/CD tool with support for deployment pipelines CD • Simpler • Better overview
- 26. Two tools? CI CI tool CD CD tool Artifact as trigger/ handover • Best tool for the job • More complex
- 27. • Travis CI, CircleCI, GitLab CI, … • Deploy agent needs access to production • Use separate tools for CI and CD • AWS Code* • Definitely for OSS Managed deployment pipelines?
- 28. • CD infrastructure should be the first task in a new project • CD should not become a snowflake itself • For disaster recovery you will need your CD infrastructure • Aim for “CD as a service” Automate CD infrastructure
- 29. • Containerized • Isolated builds – bring your own agent • Elastic agents • Container as artifact • Pipeline as code • Declarative in service repository • Fast and simple bootstrapping of new pipelines • Avoid single, shared CI instance New CI/CD practices
- 30. • Everything that used to be good practices • No CI theatre • Embrace deployment pipelines • No smarts in the CI tool Old CI/CD practices – Recap
- 32. Pets?
- 33. Cattle, not pets
- 35. Cloud native deployment pipeline Application code and infrastructure specification in one repository per service.
- 36. Cloud native deployment pipeline Application code and infrastructure specification in one repository per service. CI Deployment package and infrastructure declaration as artifact.
- 37. Cloud native deployment pipeline Application code and infrastructure specification in one repository per service. CI Deployment package and infrastructure declaration as artifact. CD 1. Create or update service infrastructure.
- 38. Cloud native deployment pipeline Application code and infrastructure specification in one repository per service. CI Deployment package and infrastructure declaration as artifact. CD 1. Create or update service infrastructure. 2. New instances pull down package and start application.
- 40. • Follow microservices boundaries • At least one stack per microservice Decompose into Micro-Infrastructures
- 41. • Macro stack(s) • Outputs parameters exported • Keep it small, only things that don’t change often • No services Macro-Infrastructure • Network • Security • Bastion Host
- 42. • Services share macro stack • Service stacks import parameters • Service teams own service stack • All services are in service stacks Shared stack and service stacks
- 43. • Services have dependencies • CD infrastructure • Macro stack • Base images (AMI, container) • … • But avoid explicit pipeline dependencies • Try to reference pinned dependencies Isolate deployment pipelines
- 44. Deployment
- 45. You build it, you run it
- 46. How many environments? V2V3 V6 V5 V4 V7 V5 V8 Enginee r CI Dev Staging V1 V4 Prod
- 47. • Integrate in production • Consumer-driven contract testing • Reduce impact of failures • MTTR over MTBF • Monitoring • Canary releases • Rollbacks • Semantic monitoring No staging environment
- 48. • Separate code deployment from feature release • Trunk-based development • No long lived feature branches Feature toggles
- 49. Feature toggles – release and experiment • Product is in charge of releasing a feature • Canary releases • A/B testing
- 51. Function as a Service - FaaS
- 52. Done Function as a Service - FaaS
- 53. Lifecycle of immutable servers/containers Created V3
- 54. Lifecycle of immutable servers/containers Created V3 Healthcheck ok V3
- 55. Lifecycle of immutable servers/containers Created V3 Healthcheck ok V3 Traffic from load balancer V3
- 56. Lifecycle of immutable servers/containers V3 Created V3 Healthcheck ok V3 Traffic from load balancer V3 Connections drained
- 57. Lifecycle of immutable servers/containers V3 Created V3 Healthcheck ok V3 Traffic from load balancer V3 Terminated V3 Connections drained
- 58. Lifecycle of immutable servers/containers V3 Created V3 Healthcheck ok V3 Traffic from load balancer V3 Terminated V3 Connections drained • No need for configuration management tools: Chef, Puppet, Ansible • Patches/Security? Alert on base image age • Simpler with stateless services
- 59. Rolling update V3 V3 V3
- 60. Rolling update V3 V3 V3 V4
- 61. Rolling update V3 V3 V3 V4
- 62. Rolling update V3 V3 V3 V4
- 63. Rolling update V3 V3 V4V3
- 64. Rolling update V3 V3 V4V3
- 65. Rolling update V3 V3 V4
- 66. Rolling update V3 V3 V4 V4
- 67. Rolling update V3 V4 V4
- 68. Rolling update V3 V4 V4 V4
- 69. Rolling update V4 V4 V4
- 70. Rolling update V4 V4 V4 • Only few additional resources required during deployment • Takes some time
- 72. Blue/green V3 V3 V3 V4 V4 V4
- 73. Blue/green V3 V3 V3 V4 V4 V4
- 74. Blue/green V4 V4 V4V3 V3 V3
- 76. Blue/green V4 V4 V4V3V3V3 • Can keep drained instances for faster rollback
- 78. Blue/green V4 V4 V4 • Twice the resources required during deployment • Faster deployment
- 79. Canary analysis V3 V3 V3
- 80. Canary analysis V3 V3 V3 V4 • Make explicit, automated canary analysis • Error rate • Latency • Load • Alternative: Feature toggle based canaries
- 81. • Existing service in production Dark launches Service Client
- 82. • New service to be launched Dark launches Old New Client
- 83. • Fork real traffic to new service and discard response • Monitor new service under real load • Compare responses • Fork on server or client side Dark launches Old New Client
- 84. Wrapping it up
- 85. • Build isolation • Independent pipelines • Elasticity • Everything as code • Pipelines owned by teams Recommendations for deployment pipeline
- 86. • Time from commit to production – delivery lead time • Time to bootstrap a new service including the deployment pipeline • Deployment frequency • Mean time to recover – MTTR Metrics
- 87. • “You build it, you deploy it, you run it” • Embrace immutability • Infrastructure follows microservices architecture • Failures happen • Reduce impact • Fast detection • Fast recovery Important
- 88. Thank You!
- 89. • Regent's Rowing 8 by Jmf3333 [CC BY 3.0] https://en.wikipedia.org/wiki/File:Regents_rowing.JPG • Aquapark Aquacolors by Pantharei.2017 (Own work) [CC BY-SA 4.0] https://commons.wikimedia.org/wiki/File:Aquapark_Aquacolors.jpg • The Key of a Chamberlain by Niklitov [CC BY-SA 4.0] https://commons.wikimedia.org/wiki/File:The_Key_of_a_Chamberlain_at_Kingdom_of_P russia_Kalinigrad_Blindage_museum.JPG • Beziers Fonseranes by Dedounet [CC BY-SA 1.0] https://commons.wikimedia.org/wiki/File:Beziers_Fonseranes.jpg Image attribution
Editor's Notes
- #3: Embrace your cloud https://www.cncf.io/about/charter/ https://pivotal.io/de/cloud-native DevOps, Microservices, Containers, Continuous Delivery Attribution: CC0: https://pixabay.com/en/sky-cloud-blue-white-good-looking-254668/
- #4: Bring changes into production, or into the hands of users, safely and quickly in a sustainable way.
- #6: Speed -> Fast cycle time, Fast creation of new pipelines
- #7: Being fast with a small team is one thing, but we want to not slow down, when the organization is larger.
- #8: No committees, no waiting
- #9: Autonomous teams -> Pipeline owned by teams
- #12: Independent deployable -> Many pipelines
- #16: They became silos, separated by a virtual wall. Tickets, Waiting, Blaming
- #17: If you still have those silos: Get rid of them!
- #18: True DevOps culture
- #19: Jezz Humble: What We Learned from Three Years Sciencing the Crap out of Devops ACCELERATE: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations
- #23: Now lets walk through the pipeline steps and see what happens when we put on the cloud native hat.
- #25: Every engineer knows GitHub SSO: Depending on plan, additional tooling required. Backup: Scheduled clone/pull of all repos. Secure: Story of the committed GitHub key.
- #26: Orchestration in one tool Better overview, Simpler
- #27: CI: All the known tools CD: For example Spinnaker or Bosh, often platform specific
- #28: We didn’t want to handover the golden key to a managed CI tool.
- #29: If you cannot use a managed CD service…
- #30: Isolated builds: Bring your own build environment, multi tenant services, like Travis, already work like this Also need to solve the problem of dependency caches. For example local Nexus. Elastic agents: Cost efficient, quickly spin up agents on demand. No job queue. Container as artifact: DinD or DooD Pipeline as code: Service templates includes pipeline declaration Single CI: Autonomous teams, CI patterns and guidelines are shared, not the infrastructure. https://www.thoughtworks.com/radar/techniques/pipelines-as-code
- #31: CI theatre: Infrequent commit, long red, feature branches Embrace deployment pipelines: Beyond CI, eg. Jenkins, TC Smarts in the tool: Use the tool for coordination, smarts are in your automation, make like build scripts should run from shell https://www.thoughtworks.com/radar/techniques/ci-theatre
- #32: Now we are done with CI, let’s talk about infrastructure. https://www.thoughtworks.com/talks/infrastructure-design-patterns-xconf-eu-2017
- #33: No slowflakes Attribution: CC0, https://pxhere.com/en/photo/813531
- #34: Immutable servers Phoenix servers Attribution: CC0, https://pixabay.com/en/hungary-puszta-sheep-herd-graze-734026/
- #35: Ideally we are not even concerned with servers anymore: Serverless
- #36: Instance can also be containers
- #37: Instance can also be containers
- #38: Instance can also be containers
- #39: Instance can also be containers
- #40: Infrastructure as code also implies good engineering practices. Not one template/stack for the complete infrastructure Resist layered slicing
- #41: Stacks can be decomposed further. For example separate LB + Persistence stack from ASG for blue/green.
- #42: VPC, Subnets, Security Groups, Internet Gateway
- #43: Cross-stack references, not nested stacks. "All services" include ops service like logging
- #44: You are not triggering your builds, when a dependent OSS packages changes. Story of Base AMI
- #45: Attribution: CC0 https://pixabay.com/en/cry-fear-funny-psyche-face-human-2192148/
- #46: You build it, you deploy it, you run it
- #47: Which versions on staging? Prod differs anyway: Load, data, patterns
- #48: Still we want to release with confidence: Be bold, but not stupid. Mean time to detection
- #49: Who is using feature branches and does CI? You are not doing CI, when you are branching. Short lived branches < 1 day are ok. Branch by abstraction https://trunkbaseddevelopment.com/ https://launchdarkly.com/
- #50: Optional: Toggle forwarding to downstream services Toggle dependencies.
- #51: Many of those patterns are viable without going cloud native, but are feasible then. Attribution: CC0, not attribution required, https://pixabay.com/en/ibiza-rock-sea-water-821719/
- #53: Serverless frees you from this deployment details
- #54: Let’s step back to where we actually deal with servers or containers.
- #55: Verify health <> canary Attribution heartbeat CC0, no attribution required http://maxpixel.freegreatpicture.com/Heartbeat-Heart-Medical-Ecg-Electrocardiogram-2270728
- #56: Instead of load balancer also batch or queue processing possible Or added to cluster.
- #59: State should be pushed to client or down to managed state services form cloud provider. But this pattern also works with modern distributed data stores.
- #61: Create
- #62: Healthcheck
- #63: Traffic
- #64: Drain
- #65: Terminate
- #67: Repeat for all nodes
- #81: You bring your canary to the coal mine and observe its behavior. Feature toggle canary: Staged rollout. Attribution: Canary: Public domain, https://commons.wikimedia.org/wiki/File:Carduelis_chloris_m.jpg Coal Mine: CC0, no attribution required, https://pixabay.com/en/coal-black-mineral-underground-1626368/
- #84: Trade offs for client and server side forking: Server: Risk, timeout protection, additional load Client: Disclosure, bandwidth and load on client, less control
- #85: Attribution: CC0, no attribution required, https://www.pexels.com/photo/close-up-of-christmas-decorations-257855/
- #86: Build isolation: Decoupling, no coordination for a new Java SDK. Independent pipelines: Decoupling Elasticity: Do not waste resource or make teams wait for an agent Everything as code: Pipeline definition, deployment scripts, infrastructure Attribution: CC0, https://pixabay.com/en/thumbs-up-thumb-hand-positive-1006176/
- #87: Improve over time, what is included in a newly bootstrapped service: Logging, Monitoring, Alerting, Dashboards Attribution: CC0, http://maxpixel.freegreatpicture.com/Measurement-Stopwatch-Timer-Clock-Symbol-Icon-2624277
- #88: Autonomous, empowered teams. No silos Do not deal with DC constraints and mutate existing servers. Do not build an infrastructure monolith, decoupling
- #89: Attribution: Raised hands, CC0, no attribution required, https://pixabay.com/en/hands-hand-raised-hands-raised-220163/