Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on Enterprise Cloud Cost Management
0 likes171 views
Opencost is an open-source tool designed for monitoring Kubernetes and cloud costs, aiding enterprises in efficient cloud cost management. It tracks various metrics such as CPU, memory, and storage to help teams understand their expenditure and resource usage. The project is part of the Cloud Native Computing Foundation and offers a range of integrations and plugins to enhance its functionality.
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on Enterprise Cloud Cost Management
- 1. #ossummit Open Source Cloud Costs OpenCost's Impact on Enterprise Cloud Cost Management Don O'Neill Matt Ray @handle
- 2. Who are we? Don O’Neill Program Lead - Standardize & Enforce, MuleSoft Governance Co-leader of Seattle CoffeeOps - a DevOps/SRE meetup Matt Ray OpenCost Community Manager Open Source veteran, Software Defined Talk podcast co-host
- 3. The complexity of operating Kubernetes efficiently is real 3 Technical COMPLEXITY Metal VMs K8s Higher-level abstractions, more shared resources, and increasingly dynamic Behavioral COMPLEXITY Decentralized releases means any engineer can increase spend quickly Abstraction Any engineer Any org Any time $ $ $ $ $ $
- 4. What is OpenCost? Open Source Kubernetes and Cloud Costs Monitoring Specification and Implementation Cloud Native Computing Foundation Sandbox Project FinOps Certified Solution • https://opencost.io • https://github.com/opencost • https://www.cncf.io/projects/opencost/
- 5. OpenCost Specification https://www.opencost.io/docs/specification Kubernetes workloads are often transient and vary in the resources they consume. How do we measure who is responsible for what and how much? • Management fees • Expenses from nodes • Persistent volumes • Attached disks • Load balancers • Network ingress/egress
- 6. Kubernetes Costs Specification What is Measured • CPU • Memory • GPU • Storage Volume • Load Balancer Aggregations • Cluster • Node • Container • Annotation • Controller Kind • Controller Name • DaemonSet • Deployment • Job • Label • Namespace • Pod • Service • StatefulSet
- 7. What is Monitored Kubernetes • Amazon Web Services • Google Cloud Platform • Microsoft Azure • Oracle Cloud Infrastructure • On-premises Cloud Costs • Billing data • Cost and usage reports Carbon Costs (1.110) • Carbon footprint of workloads OpenCost Plugins (1.110) • New Plugin architecture • Datadog first example
- 8. OpenCost an end user’s perspective
- 9. Kubernetes - the great mystery!
- 10. Let's dig into some terminology AWS CUR - Cost and Usage Report EKS - Elastic Kubernetes Service
- 11. EKS costs in the CUR Your bill is $10,000 dollars for EKS
- 12. Enter OpenCost How do we help teams understand their costs, efficiency and usage?
- 13. OpenCost architecture (at MuleSoft)
- 14. OpenCost architecture (cont.) ● deployed in its own namespace “oversight” ● deployed in all clusters, in all flavors of Kubernetes ● writes data daily as a .csv file to an S3 bucket ● uses a seven day rolling data window ● data lake consumes the .csv file, refreshing the reports
- 15. OpenCost lessons learned ● Be sure you have a Prometheus endpoint available ● Bringing your own Prometheus? tune its memory and cpu requests/limits! ● Pre existing data lake? Parquet exporter ● You can use your own rate sheet
- 16. Running OpenCost https://www.opencost.io/docs/install Prometheus • Datastore for metrics Helm Chart • Configurable settings (Prometheus, namespaces, etc.) Docker • Run without Kubernetes, Cloud Costs and other data sources Prometheus OpenCost Exporter • Prometheus Community Helm chart
- 17. OpenCost UI
- 18. Accessing OpenCost Data ● API ● Backstage ● CSV export ● Grafana Dashboards ● kubectl cost ● OpenCost Parquet Exporter ● Prometheus directly ● Prometheus OpenCost Exporter ● Web UI
- 19. OpenCost on GitHub https://github.com/opencost ● https://github.com/opencost/opencost ● https://github.com/opencost/opencost-grafana-dashboards ● https://github.com/opencost/opencost-helm-chart/ ● https://github.com/opencost/opencost-parquet-exporter ● https://github.com/opencost/opencost-plugins ● https://github.com/opencost/opencost-website/
- 20. Get Involved with OpenCost! https://opencost.io Slack • https://slack.cncf.io/ #opencost OpenCost Community Meetings • https://bit.ly/opencost-calendar • https://bit.ly/opencost-community-meeting-cncf Social Media • https://www.linkedin.com/showcase/opencost/ (LinkedIn) • https://bsky.app (Bluesky) • https://hachyderm.io/@opencost (Mastodon) • https://twitter.com/open_cost (Twitter)