Load Balancing for Containers and Cloud Native Architecture
4 likes1,333 views
This document discusses how load balancing needs have changed with the rise of containers and microservices. Traditional load balancers are not well-suited for dynamic container environments where services and topology frequently change. New approaches are needed like integrating load balancers with container orchestration platforms, using client-side load balancing, and leveraging load balancers for advanced patterns like zero-downtime deployments, circuit breaking, visibility and security segmentation. Load balancers are playing an increasingly important role in cloud native architectures.
Load Balancing for Containers and Cloud Native Architecture
- 1. NOT YOUR PARENTS’ LOADBALANCER Chiradeep Vittal ContainerCon 2016
- 2. About me ¨ Distinguished Engineer at Citrix ¨ Apache CloudStack PMC ¨ Work on Citrix Netscaler and containers
- 3. Load balancers are important (again) ¨ Containers and Microservices ¨ Teaching old load balancers new tricks ¨ Emerging patterns for Load Balancing ¨ Future directions
- 4. A brief history of Load balancing Internet W W High Availability Internet W W WW HA + Load Balancing Internet Webserver
- 5. A brief history of Load balancing Internet W WW App 1 App 2 W WW foo.com/app1 foo.com/app2 L7 routing Internet W WW SSL Traffic Unencrypted Traffic Internet W WW App Firewall
- 6. What LB-in-the-wire enables ¨ Resilience ¨ Encryption offload ¨ Application acceleration ¨ Defend L4à L7 ¨ L7 routing ¨ Application Performance Analytics
- 7. Load Balancing Form Factors Hardware Software VM Virtual As-a-service Containerized
- 9. What’s changed? ¨ DevOps + Automation ¨ Microservices / Containers Image by BMW Werk Leipzig - http://bmw-werk-leipzig.de, CC BY-SA 2.0 de
- 10. Containers or Microservices? ¨ Microservices == architecture ¨ Containers == implementation ¨ Containers win over VMs for Microservices
- 12. Load Balancing: Traditional vs. Cloud Native Static Applications, well defined topology Dynamic Microservices, changing topology W W W A A A A A W M M TrafficismostlyN-S M M M M M M M M M Traffic is mix of N-S and E-W
- 13. Cloud-Native Patterns of architecture and organization that deliver software with speed and reliability ¨ Auto Scale ¨ Continuous Delivery ¨ Baked-in Resilience ¨ Deep Monitoring ¨ Collaboration
- 14. Cloud Native + Containers is Network Intensive ¨ Implications on ¤ Performance ¤ Reliability ¤ Security ¤ Routing ¤ Naming and discovery ¤ Monitoring
- 15. Cloud Native Landscape - Microservices Microservices APIs Performance Resilience Security Visibility Continuous Delivery AutoScal e Circuit Breaker Load Balancing Throttling Discovery Audit Segmentation E2E Encryption Routing Chaos Monkey Distributed Debug Back-off Lifecycle Management Auth
- 16. Containers Container Networking Container Security Integrity Isolation AuthN/Z Container Orchestration Infrastructure Orchestration Network Orchestration Cloud Orchestration Multi-cloud Orchestration Private Cloud OrchestrationHypervisor Orchestration Network Orchestration IPAM / DNS Overlay / Underlay Microservices Lifecycle Management Container Performance Network Perf scheduler Container Storage Storage Orchestration Storage Orchestration Cloud Native Landscape - Infrastructure
- 17. Microservices Containers Contain er Networ king Contain er Security Integr ity Isolati on Auth N/Z Contain er Orchest ration Infrastru cture Orchest ration Networ k Orchest ration Cloud Orchest ration Multi- cloud Orchest ration Private Cloud Orchest ration Hypervi sor Orchest ration Networ k Orchest ration IPAM / DNS Overlay / Underla y Lifecycl e Manag ement Contain er Perform ance Netw ork Perf schedul er Contain er Storage Storage Orchest ration Storage Orchest ration Cloud Native Landscape Microservi ces APIs Perform ance Resilienc e Security Visibility Continuo us Delivery AutoS cale Circui t Break er Load Balan cing Throttl ing Disco very Audit Segment ation E2E Encrypti on Routing Chaos Monk ey Distribut ed Debug Back- off Lifecycle Manage ment Auth
- 18. Container Managers to the rescue? Docker Swarm Kubernetes (from Google) Mesosphere DCOS (based on Apache Mesos) AWS ECS
- 20. Cluster Managers – Sweet Spots ¨ Infrastructure independence ¨ Scheduling ¨ Discovery ¨ Scaling (partially) ¨ Security (a little bit) ¨ Load balancing (limited)
- 21. Load Balancing for container clusters – Ingress / Edge α5 α6 HostH4 Scale out Internet α1 β1 δ1 HostH1 β2 α2 δ3 HostH2 α3 α4 δ2HostH3 Public Endpoint https://alpha:443 LB
- 22. Load Balancing – intra-cluster α1 β1 HostH1 α2 δ3 HostH2 α4 δ2 HostH3 LB LB per endpoint
- 23. Load Balancing – intra-cluster α1 β1 HostH1 α2 δ3 HostH2 α4 δ2 HostH3 LBLB LB LB per container host • Service clients use an LB local to their host • E.g., Kube-proxy, Swarm, Mesos - MinuteMan
- 24. Ingress vs Intra-cluster differences ¨ Performance ¨ Count ¨ Frequency of reconfiguration ¨ L4 (intra-cluster) vs L7 (ingress) ¨ Form factor
- 25. Reconfiguration of Load Balancer α1 β1 HostH1 β2 α2 HostH2 α3 α4 HostH3 Ingress LB α5 α6 HostH4 ClusterManager LB Controller Cluster API Container Events Reconfigur ation Container State Container State Query
- 26. Nitrox – Configure Citrix NetScaler for Cluster Managers ¨ github.com/chiradeep/nitrox ¨ Apache license ¨ Support for integrating NetScaler with ¤ Kubernetes ¤ Swarm ¤ Consul ¤ Marathon
- 27. Kubernetes Ingress Controller ¨ https://github.com/chiradeep/kube-ingress-citrix- netscaler ¨ Ingress Load Balancing for Kubernetes Clusters
- 28. Cluster Managers: Native LB support ¨ Roll-your-own ¤ Docker Swarm, Kubernetes, Mesos ¨ AWS Application LB ¤ ECS ¨ L4 ¤ Docker Swarm, Kubernetes, Mesos ¨ L7 ¤ AWS Application LB (ECS) ¤ Linkerd Ingress LB Intra-cluster
- 29. Emerging LB patterns ¨ Client-side LB ¨ Sophisticated routing ¨ Resilience patterns ¨ Visibility / Insights
- 30. Client-side LB ¤ Embedded into calling application n e.g., Netflix Ribbon (with Eureka) ¤ Run as side-cars (alongside each application or one- per-host) Eureka M M M M M M
- 31. Client-side LB ¤ Run as side-cars, or one-per-host Service Discovery M M M M M M Clients Service LB LB
- 32. Client-side LB Examples ¨ Netflix Prana - Hystrix as side car (deprecated) ¨ Linkerd from Buoyant.io (based on Twitter Finagle) ¨ AirBnB Smartstack/Synapse (uses HAProxy) ¨ Uber Hyperbahn ¤ Like Finagle, switched from HAProxy ¨ Kube Proxy ¤ Initially user-space, now based on IPTables ¨ Docker Swarm LB ¤ Based on IPVS ¨ Mesos LB ¤ Based on IPTables
- 33. Recap: LB-in-the-wire ¨ Resilience ¨ Encryption offload ¨ Application acceleration ¨ Defend L4à L7 ¨ L7 routing ¨ Application Performance Analytics
- 34. A general-purpose L7 proxy? M M M M M M M M M M M
- 35. A general-purpose L7 proxy? M M Client Service A M Service B LB LB Host
- 36. A general-purpose L7 proxy? M M Client Service A L7 ProxyL7 Proxy M Service B L7 Proxy Service Discovery CD Pipeline Policy Engine
- 37. Zero-Downtime Deploys ¨ Canary deploys ¨ Traffic can be selected based on source (country), headers M M M M Version N M M M M M Version N+1 10%90% CD Pipeline
- 38. Zero-Downtime Deploys ¨ Blue-green deploys M M M M Version N M M M M M Version N+1 100% CD Pipeline
- 39. Advanced Resilience – Circuit Breaker ¨ Circuit Breaker is a pattern that prevents cascading failures due to unresponsive downstream services ¨ A load balancer can be configured to ignore a slow backend Credit: http://martinfowler.com/bliki/CircuitBreaker.html
- 40. Visibility ¨ Compliance ¨ Topology ¨ Analytics M M M M M M M M M M M Analytics
- 42. Future directions ¨ Multi-cloud resilience ¨ Security ¨ Protocol Insights
- 43. Multi-cloud ¨ Choose the best location for the microservice ¨ Automatic discovery and resilience ¨ See Kubernetes Federation for example M M M M Cloud 1 M M M M M Cloud 2 Further reading: http://kubernetes.io/docs/user-guide/federation/federated-services/
- 44. Security ¨ Segmentation ¨ Authorization ¨ End-to-End encryption M M Client A Service A L7 ProxyL7 Proxy M Service B L7 Proxy Policy who what why when ? How
- 45. Segmentation M M Client A Service A L7 ProxyL7 Proxy M Service B L7 Proxy ✓
- 46. Encryption M M Client A Service A L7 ProxyL7 Proxy M Service B L7 Proxy Client Certificate Further reading: https://blog.cloudflare.com/how-to-build-your-own-public-key-infrastructure/
- 47. Cryptographic Segmentation M M Client A Service A L7 ProxyL7 Proxy M Service B L7 Proxy Client Certificate Further reading: https://spiffe.io
- 48. Authorization M M Client A Service A L7 ProxyL7 Proxy M Service B L7 Proxy Authorization: Bearer: xxxxx.yyyyy.zzzzz Authorization: Bearer: aaaaa.bbbb.cccccc Token Insertion Token Validation Further reading: https://github.com/coreos/jwtproxy
- 49. Protocol Insights ¨ Microservice communication patterns are changing ¤ New protocols/formats: gRPC, Thrift, TChannel ¤ Asynchronous / Event-driven ¤ Queues and real-time streams
- 50. Wrap-up ¨ Load balancing is different for containers / microservices ¨ Integration with container cluster managers is needed ¨ Consider using the same LB technology for ingress and intra-cluster ¨ Emerging patterns solidify the importance of the load balancer ¨ LB in the wire brings ¤ Simplicity ¤ Resilience ¤ Future proofing