Kubernetes Security
Download as PPTX, PDF2 likes1,550 views
The document presents a comprehensive overview of Kubernetes security challenges and best practices, addressing complexities in container management, deployment, and operations in cloud-native environments. It emphasizes the importance of reducing the attack surface through various techniques, including securing cluster components, managing access controls, and utilizing Kubernetes' built-in security features. Additionally, it highlights open-source tools available for enhancing security, such as kube-bench and clair, while encouraging periodic audits and adherence to best practices.
Kubernetes Security
- 1. Kubernetes Security Karthik Gaekwad INNOTECH Austin 2018
- 2. • I’m Karthik Gaekwad • NOT a DBA • https://cloudnative.oracle.com/ • Cloud Native evangelist at OCI • Used to be a developer on the OKE Team. Hello @iteration1
- 3. Hello • Been in Industry 15 years. • In general, I like building stuff with friends. • A maintainer for Gauntlt- Open source security scanner. • Love Teaching and building community. • Run Devopsdays Austin, Container Days, Cloud Austin. • Chair All Day Devops Cloud Native track. • LinkedIn Learning Author for Learning Kubernetes (and more).
- 4. Need an OCI Trial Account? http://bitly.com/ocicloud
- 5. My questions for you..
- 6. The Cloud Native Journey 6 Phase I Developer Focus Phase II DevOps Focus Phase III Business Focus (end-to-end) Container Adoption Application Deployment Intelligent Operations SpeedEfficiencyAgility Docker Kubernetes Core to Edge Developer adoption Dev/Test apps Simple orchestration Individual developers DevOps deployment Production apps Advanced orchestration Teams & lines of business End-to-end integration Digital business apps Serverless, DevSecOps, & ML Cloud native enterprises Focus Applications Automation Community
- 7. Latest CNCF Survey: August 2018 How Does Your Company Use Containers and Where?
- 8. Kubernetes Dominates Container Management Your company/organization manages containers with:
- 9. Good News, Bad News… Good: On average, CNCF project usage is up over 200% since the Dec 2017! But...
- 10. Complexity, Culture, Training, & Security Issues Remain
- 11. • Managing, maintaining, upgrading Kubernetes Control Plane • API Server, etcd, scheduler etc…. • Managing, maintaining, upgrading Kubernetes Data Plane • In place upgrades, deploy parallel cluster etc…. • Figuring out container networking & storage • Overlays, persistent storage etc… - it should just work • Managing Teams • How do I manage & control team access to my clusters? • Security, security, security Kubernetes & Cloud Native Challenges 11 Source: Oracle Customer Survey 2018
- 12. How Are Teams Addressing Complexity, Training Issues? App Management Upgrades & Patching Platform Backup & Recovery High Availability Scaling App Deployment Power, HVAC Rack and Stack Server Provisioning Software Installation Customer Managed Fully-Managed App Management Upgrades & Patching Platform Backup & Recovery High Availability Scaling App Deployment Power, HVAC Rack and Stack Server Provisioning Software Installation Faster Time to Deploy Lower Risk Accelerate Innovation Benefits YOU
- 13. Which brings us to security…
- 14. Where no news, is good news!
- 15. Unsecured K8s dashboards • Unsecured Kubernetes Dashboard with account creds. • Used this to mine cryptocurrency. • 2017: Aviva • 2018: Tesla, Weight Watchers • https://redlock.io/blog/cryptoja cking-tesla
- 16. Kubelet credentials hack • Shopify: Server Side request Forgery • Get kubelet certs/private key • Root access to any container in part of infrastructure. • https://hackerone.com/reports/ 341876
- 21. WAT?
- 22. How did we get here?
- 23. “Kubernetes is too complicated”
- 24. “Kubernetes is too complicated” “We hope it’ll get easier”
- 25. What is your strate
- 26. Let’s look at: •Attack Surface • More importantly, how to limit damage •Security related features in K8s • The more you know, the better you build •Opensource Tooling to help • Because we all need help
- 27. Attack Surface
- 28. Attack Surface Goal: Reduce the attack surface •Analysis for: •Host •Container (Images and running) •Kubernetes Cluster
- 29. Attack Surface • Work on reducing the attack surface: • Analysis for: • Host • Container (Images and running) • Kubernetes Cluster
- 30. Attack Surface: Host • These are the machines you’re running Kubernetes on. • Age old principles of Linux still apply: • Enable SELinux • AppArmor • Seccomp • Goal: Minimize privilege to applications running on the host
- 31. Attack Surface: Container Images • Know your base image when building containers • Smaller the better • Don’t rely on :latest tag • Check for vulnerabilities periodically
- 32. Attack Surface: Running Containers • Don’t run as root • Limit host mounts
- 33. Attack Surface: Kubernetes Cluster • TLS • Audit Logs • Network Policies • Pod Security Policies • Secrets
- 34. Kubernetes Cluster: TLS TLS ALL THE THINGS
- 35. Attack Surface: Host • These are the machines you’re running Kubernetes on. • Age old principles of Linux still apply: • Enable SELinux • AppArmor • Seccomp • Hardened Images • Goal: Minimize privilege to applications running on the host • Good news: Already a wealth of information on this subject! • http://lmgtfy.com/?q=how+to+reduce+attack+surface+linux
- 36. Attack Surface: Container Images GOAL: Know your base image when building containers
- 37. Attack Surface: Container Images GOAL: Know your base image when building containers **BTW, this is just a ruby helloworld app
- 38. Attack Surface: Container Images GOAL: Know your base image when building containers **BTW, this is just a ruby helloworld app
- 39. Attack Surface: Container Images GOAL: Know your base image when building containers Full disclosure: I’m karthequian; I created this as a ruby 101 container for learning purposes only
- 40. Attack Surface: Container Images GOAL: Know your base image when building containers • When in doubt, stick to an official images! • Or start from a sane base image (example: alpine linux)
- 41. Attack Surface: Container Images GOAL: Smaller the image, the better • Less things for an attacker to exploit. • Quicker to push, quicker to pull.
- 42. Attack Surface: Container Images GOAL: Don’t rely on :latest tag • :latest image yesterday might not be :latest image tomorrow • Instead, you’d want to know what specific version you’re operating with. • Side benefit: If there is a new vulnerability announced for OS version x.y.z, you know immediately whether you’re running that version!
- 43. Attack Surface: Container Images GOAL: Check for vulnerabilities periodically • Plenty of ways to do this in registries. We’ll cover more in the tooling section
- 44. Attack Surface: Running Containers GOAL: Don’t run as root • Containers running as root might be completely unnecessary for the actual application. • If compromised, attacker can do a lot more things.. • Pod security policies can help (we’ll see how later).
- 45. Attack Surface: Running Containers GOAL: Limit host mounts • Be wary of images that require broad access to paths on the host • Limit your host mount to a smaller subset of directories • Reduces blast radius on compromise
- 47. Kubernetes Cluster- TLS TLS ALL THE THINGS
- 48. Kubernetes Cluster- TLS • TLS Checklist: 1. Nodes and Master 2. User and Master 3. Everything etcd 4. Kubelet to API Server
- 50. Kubernetes Cluster- TLS • TLS Checklist: 1. User and Master 2. Nodes and Master 3. Everything etcd 4. Kubelet to API Server
- 51. We’re a little better off now. But what else to do?
- 52. K8s Features How can the platform help me make secure choices?
- 53. K8s Features • Authentication • Authorization • Audit Logging • Network Policies • Pod security policies • Kubernetes Secrets
- 54. Authentication and Authorization • Do you know how you are authenticating with Kubernetes? • Many ways to Authenticate • Client Certs • Static token file • Service Account tokens • OpenID • Webhook Mode • And more (https://kubernetes.io/docs/reference/access-authn-authz/authentication/)
- 55. Whatever you do, DO NOT YOLO! Goal: Pick a strategy that fits your use case
- 56. You can pick an authz strategy.. If you DO NOT YOLO…
- 58. Authentication and Authorization • Pro tip: Nobody uses ABAC anymore. Don’t be that guy…. • RBAC is the defacto standard • Based on roles and role bindings • Good set of defaults: https://github.com/uruddarraju/kubernetes-rbac-policies • Can use multiple authorizers together, but can get confusing. • 1st authorizer to authorize passes authz
- 59. Kubernetes Cluster- Audit Logs • Wat? • “Kubernetes auditing provides a security-relevant chronological set of records documenting the sequence of activities that have affected system by individual users, administrators or other components of the system.” • Answers: What/when/who/where information on security events. • Your job: Periodically watch Kubernetes Audit logs • https://kubernetes.io/docs/tasks/debug-application-cluster/audit/
- 61. Kubernetes Cluster- Network Policies • Consider adding a network policy to the cluster… • Default Policy: All pods can talk to all other pods. • Consider limiting this with a Network Policy • https://kubernetes.io/docs/concepts/services-networking/network-policies/
- 62. Kubernetes Cluster- Pod Security Policies • Consider adding Pod Security policies • PodSecurityPolicy: A Defined set of conditions a pod must run with. • Think of this as authorization for pods.
- 63. Kubernetes Cluster: Pod Security Policies https://kubernetes.io/docs/concepts/policy/pod-security-policy/#what-is-a-pod-security-policy Capability for an admin to control specific actions
- 64. Kubernetes Secrets • GOAL: Use Kubernetes secrets to store sensitive data instead of config maps. • Also look at: secrets encryption provider. • Controls how etcd encrypts API data • --experimental-encryption-provider-config • https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/
- 66. Keep tabs on the CNCF Security landscape https://landscape.cncf.io/landscape=security-compliance
- 67. CNCF Projects • “The Update Framework” • Is a framework or a methodology. • Used for secure software updates. • Based on ideas surrounding trust and integrity. • Is a project. • Based on TUF. • A solution to secure software updates and distribution. • Used in Docker Trusted Registry.
- 68. Clair • Open source project for the static analysis of vulnerabilities in containers. • Find vulnerable images in your repo. • Built into quay.io, but you can add to your own repo. • https://github.com/coreos/clair
- 70. Kube-bench • Checks whether a Kubernetes cluster is deployed according to security best practices. • Run this after creating your K8s cluster. • https://github.com/aquasecurity/kube-bench • Defined by the CIS Benchmarks Docs: https://www.cisecurity.org/cis- benchmarks/ • Run it against your Kubernetes Master, or Kubernetes node.
- 72. Kubesec • Helps you quantify risk for Kubernetes resources. • Run against your K8s applications (deployments/pods/daemonsets etc) • https://kubesec.io/ from controlplane • Can be used standalone, or as a kubectl plugin (https://github.com/stefanprodan/kubectl-kubesec)
- 73. Kubesec example
- 74. Kubeaudit • Opensourced from Shopify. • Auditing your applications in your K8s cluster. • https://github.com/Shopify/kubeaudit • Little more targeted than Kubesec.
- 77. “So much time and so little to do.”
- 78. Couple more resources to look at: • 11 ways not to get hacked: https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked • K8s security (from Image Hygiene to Network Policy): https://speakerdeck.com/mhausenblas/kubernetes-security-from- image-hygiene-to-network-policies
Editor's Notes
- #12: However, customers face challenges along the way. As we have spoken to customers, many have agreed with the challenges presented on this slide.
- #13: Faster Time to Deploy No need to provision and maintain Operating System and Platforms (Linux, Kubernetes, Docker Registry, Continuous Integration Systems) Lower Risk Oracle is committed to SLAs on Performance and Manageability, in addition to Availability Accelerate Innovation Develop new Container Native apps quickly, and port existing apps faster
- #24: Photo by Byron Sterk on Unsplash
- #25: Photo by Byron Sterk on Unsplash
- #26: Photo by rawpixel on Unsplash
- #28: Photo by rawpixel on Unsplash
- #47: Photo by rawpixel on Unsplash
- #50: Diagram from https://docs.google.com/presentation/d/1Gp-2blk5WExI_QR59EUZdwfO2BWLJqa626mK2ej-huo/edit#slide=id.g1e639c415b_0_56. Thanks @Lucas Käldström
- #56: Photo by Mikhail Vasilyev on Unsplash
- #57: Photo by Mikhail Vasilyev on Unsplash
- #66: Photo by Barn Images on Unsplash