OpenStack on Kubernetes (BOS Summit / May 2017 update)
1 like2,110 views
This document discusses using Kubernetes as an underlay platform for OpenStack. Some key points: 1. Kubernetes is becoming more widely used and understood by operators compared to OpenStack. Using Kubernetes as an underlay could improve simplicity, stability, and upgrade processes for OpenStack. 2. There are still many technical challenges to address, such as networking, storage, tooling to manage OpenStack on Kubernetes, and ensuring containers meet Kubernetes' immutable infrastructure requirements. 3. Using Kubernetes as an underlay risks further confusing the messaging around OpenStack by implying Kubernetes is more stable or a replacement target. Clear communication will be important to avoid undermining OpenStack.
OpenStack on Kubernetes (BOS Summit / May 2017 update)
- 1. May 2017 Update: Will it blend? Joint OpenStack Kubernetes Environment A pragmatic operational assessment about how Kubernetes can become an underlay for OpenStack.
- 2. TL;DR: Yes and then Kubernetes wins as the platform. Video Demo: bit.ly/rebarhelm
- 3. Rob Hirschfeld (aka Zehicle online) In Community: OpenStack Board Member (4 years) Co-Chair of Kubernetes Cluster Ops SIG Founder of Digital Rebar & Crowbar Projects Professional: CEO of RackN - hybrid automation software Executive at Dell - scale data center ops Cloud Data Center Ops going back to 1999
- 4. Addressing Operators Needs Operational Success is Essential to Project Success Operators are not developers! Simple, Transparent and Stable are key concerns Becoming a super-user of the platform should not be required to run it Scale & Upgradability has both internal and external drivers Generally, Kubernetes has good operational fundamentals Even more, we need more community operational practices for OpenStack
- 5. We’re Talking Underlay, not Overlay We’re talking about installing Kubernetes first (aka underlay) and using it to manage the OpenStack control plane. Objectives for Kubernetes Underlay: ● Must Work with Kubernetes Primatives ● Not a Dedicated Kubernetes ● Limited Outside Management Physical Infrastructure Kubernetes Underlay OpenStack Kubernetes Overlay This Talk Simplest conception of the K8s OpenStack Sandwich
- 6. What is Kubernetes? Container Scheduler (no, it’s not really Orchestration) API driven to provide restart, placement, network routing and life-cycle For Applications designed for Kubernetes Key Design Elements: Immutable Infrastructure (stateless ops) 12 Factor Configuration Service Oriented
- 7. What is Kubernetes: A Three Tier Application Client 0 Ready 1 Prereq 2 Control 3 Nodes etcd (cluster) etcd (cluster) etcd (cluster) API (cluster) API (cluster) API (cluster) Kubelet KubeCtl Container Manager 5 Apps Network CNI Host Network Host Storage Host Init Pod Pod Pod Pod 4 Add-Ons Certificate Authority Scheduler (leader) Heapster Infrastructure APIs Routers, Storage, LBs... Proxy ... Controller (leader) DNS Watcher ...
- 8. Together 4ever: API server + Kubelet Client 0 Ready 1 Prereq 2 Control 3 Nodes etcd (cluster) etcd (cluster) etcd (cluster) API (cluster) API (cluster) API (cluster) Kubelet KubeCtl Container Manager 5 Apps Network CNI Host Network Host Storage Host Init Pod Pod Pod Pod 4 Add-Ons Certificate Authority Scheduler (leader) Heapster Infrastructure APIs Routers, Storage, LBs... Proxy ... Controller (leader) DNS Watcher ...
- 10. Why do we want Kubernetes as Underlay? Community Perception Accuracy 1 OpenStack Operations is still not “solved” True (no change) 2 We already do most new deploys in containers True (was partially) 3 Kubernetes is awesome at containers True (was partially) 4 Kubernetes is simple, stable and secure (for operators) Partially (was false) 5 Kubernetes means easy Upgrades and High Availability Partially (was false)
- 11. There are REAL Potential Benefits ● Leverage Docker packaging efforts and reduce Python & O/S dependencies ● Upgrades would benefit from Kubernetes built-in processes ● Use of the Kubernetes job scheduler for maintenance ● “Free” fault tolerance of key components ● Easier install if Kubernetes already running on-site ● More constrained options for configuration and operation BUT REALLY, IT’S ABOUT LOWER FRICTION AND COMMUNITY SIZE... I expect more people will understand Kubernetes operations than OpenStack operations because Kubernetes is 1) simpler and 2) cloud and physical.
- 12. Kubernetes Underlay is coming, So let’s get pragmatic about it. Leadership Kudos to SAP, ATT Comummity Dev, & Port Direct
- 13. Issues: Marketing Message is Confusing Marketing around Kubernetes under OpenStack is a “hot mess” ● People hear “Kubernetes is stable, OpenStack is not” ● Further confuses “OpenStack one platform message” ● Encourages Kubernetes as target instead of OpenStack Confusion with the Plain Old Container Install (“POCI”) message ● Canonical (Ubuntu Cloud Install), ● Rackspace (OpenStack Ansible) ● Cisco (Kolla) ● Triple O
- 14. Key Principle: Containerization vs Kubernetes Containers can be treated as a) lightweight vms or 2) packaged daemon sets. ● Canonical builds their containers like persistent vms and configures with Juju ● Kolla & OSA treats containers as packaging and configures with Ansible Kubernetes accepts neither approach – they expect containers to be immutable and 12 factor configured ● Kubernetes manages the full container life-cycle ● Containers need to be able to handle being added, removed ● Services need to be able to handle IP address changes (or use DNS names)
- 15. This work is progressing quickly! Using Kubernetes v1.5+ Primatives ● Using Kubernetes Helm Charts ● Services are tagged to nodes ● Agents become Daemon sets ● Databases using Stateful sets ● Multiple container sources Hard work remains…. ● Networking, Configuraton & Storage ● OpenStack Projects must handle immutable container requirements Physical Infrastructure Kubernetes + Helm Underlay OpenStack Mgmt OpenStack Nodes Other Apps
- 16. Kubernetes Workers More Detail: Kubernetes Underlay of OpenStack Physical Infrastructure Kubernetes Controllers OpenStack Mgmt OpenStack Nodes Data base If you to really want to build this, give me a call - RackN has all the components Msg Bus Software Defined NetworkingCeph Distributed Storage Other Workloads Helm
- 17. Technical Challenges Remain This discussion keep kicking the operations & install problems down the field Kubernetes is much newer than OpenStack, so even less understood Yet more complexity and some very basic questions: ● Now we have a both a Kubernetes and OpenStack upgrade problem ● We still need tooling to manage OpenStack in Kubernetes ● We still need someone to package the containers (+ multi-platform like ARM) ● Relies on Docker to keep systems running ● Storage and Networking are still being worked out
- 18. In summary, OpenStack operability is not solved via the underlay platform alone. Technical Leadership motivation required for OpenStack adopting Kubernetes architecture requirements. Serious messaging confusion in effort has to be resolved. However, this collaboration is required for OpenStack Because Kubernetes will have a larger footprint in Operations
- 19. By 2018, this approach will be THE install method Rob Hirschfeld, @zehicle RackN & Digital Rebar