Why Modern Systems Require a New Approach to Observability
1 like180 views
The document discusses the necessity for a new approach to observability in modern application development, emphasizing the role of DevOps, microservices, and operational analytics. It highlights the importance of comprehensive telemetry and intelligent data filtering to enhance software performance and user experience. Recommendations are provided for selecting observability tools that support complex systems and improve real-time analysis capabilities.
Why Modern Systems Require a New Approach to Observability
- 1. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Why Modern Systems Require a New Approach to Observability Steve Hendrick Research Director Enterprise Management Associates Daniel “Spoons” Spoonhower CTO and Cofounder Lightstep
- 2. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Watch the On-Demand Webinar Slide 2 • Why Modern Systems Require a New Approach to Observability On-Demand Webinar: https://info.enterprisemanagement.com/why- modern-systems-require-a-new-approach-to-observability-webinar-ws • Check out upcoming webinars from EMA here: http://www.enterprisemanagement.com/freeResearch
- 3. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Featured Speakers Steve Hendrick, Research Director, EMA Steve has over 20 years of experience as an industry analyst and 10 years of experience across a variety of application development roles. He specializes in application development and deployment topics, including DevOps, lifecycle management, application platforms, decision analytics, middleware, and the broad spectrum of tools and technologies that support professional developers and IT managers. Daniel “Spoons” Spoonhower, CTO and Cofounder, Lightstep Spoons is a co-founder at Lightstep, where he builds performance management tools for deep software systems. Previously, Spoons spent almost six years at Google where he worked on developer tools as part of both Google’s internal infrastructure and Cloud Platform teams. He has published papers on the performance of parallel programs, garbage collection, and real-time programming. He has a PhD in programming languages from Carnegie Mellon University but still hasn’t found one he loves. Slide 3 © 2020 Enterprise Management Associates, Inc.
- 4. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING How Application Development is Changing Key Tenets of Modern Application Development Slide 4 © 2020 Enterprise Management Associates, Inc. InnovationCloud Native Computing Continuous ImprovementAbstraction DecompositionSoC Microservices OSS APIs Modern Application Development Manageability VelocityPolicy Observability Governance Data Other Deep System Orchestration Data Variety Volume AI & ML
- 5. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTINGSlide 5 © 2020 Enterprise Management Associates, Inc. The Changing Focus of Application Development Every company becomes a software company 21% 40% 39% 37% 27% 35% We develop custom applications to support our internal informational needs only We develop a moderate number of custom apps to support internal info needs as well as transacting with customers/partners We develop many custom applications that impact nearly every activity in our enterprise What response best describes your enterprises’ approach to app dev? 2019 Q3 2018 Q2 DevOps 2021 Survey, September 2019, N=400 -43% +48% +11% Change Over 5 Quarters
- 6. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Top Three DevOps Objectives 3% 18% 13% 25% 14% 16% 13% 7% 10% 14% 18% 22% 29% Other Improve the performance of applications Improve IT productivity and reduce cost Improve our agility in addressing changing market needs Improve the customer experience Improve the speed of software delivery Improve the quality and reliability of applications What are the top three objectives that your enterprise wants to accomplish through DevOps? 2019 Q3 2018 Q2 DevOps 2021 Survey, September 2019, N=400 DevOps 2020 Survey, June 2018, N=420 Slide 6 © 2020 Enterprise Management Associates, Inc.
- 7. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Reasons for Using Operational Analytic Tools DevOps 2021 Survey, September 2019, N=377 49% 47% 46% 45% 45% 35% 35% 35% 34% 33% 33% 31% 30% 2% It helps us avoid hardware and software failures It helps us improve performance It helps us improve quality It helps us eliminate delays It helps us improve staff productivity It helps us reduce cost It helps us know where our value chain can be improved It helps how we design/develop/deploy/operate products It provides visibility into the operation of our infra hardware It provides visibility into the ops of our software processes It is a necessary step in moving us to the use of AI and ML It enables us to better support customer needs It provides a single unified view of our machine data Don’t know or not sure Percentage valid cases of current or planned OA tool users Why does your enterprise use or plan to use operational analytic tools? Slide 7 © 2020 Enterprise Management Associates, Inc.
- 8. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Projected DevOps Growth by Market Segment 0% 10% 20% 30% 40% 50% 60% AnnualGrowth DevOps Growth by DevOps Market Segment Growth 2021 Growth 2020 DevOps 2021 Survey, September 2019, N=400 Slide 8 © 2020 Enterprise Management Associates, Inc.
- 9. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING How Application Development is Changing • How has IT responded? • DevOps practices • Microservices • More managed services • Accelerate delivery by decoupling... • Services • Deployments • Teams Slide 9 © 2020 Enterprise Management Associates, Inc.
- 10. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING The Implications of App Dev Changes • Understanding microservice behavior is complex … and you’ve handed that responsibility to every DevOps team • Many independently changing aspects • Dependencies • Ephemeral services • Asymmetric scalability • Challenges in leveraging telemetry? Slide 10 © 2020 Enterprise Management Associates, Inc.
- 11. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Lightstep and Observability • What does Lightstep mean when you say observability? • It’s the intersection of comprehensive telemetry, analytics, and policy to solve problems and create a better user experience • Concretely, it enables teams to detect and resolve regressions quickly, regardless of system scale or complexity • How should observability be approached to drive value? • Unify telemetry (logs, metrics, traces) • Collect all the data but filter intelligently • Leverage the OpenTelemetry* API • Telemetry should be addressed during development • Integrate into existing workflows to accelerate/automate responses Slide 11 © 2020 Enterprise Management Associates, Inc. *OpenTelemetry is a CNCF Sandbox Project
- 12. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Lightstep and Observability • The mechanics of how Lightstep collects telemetry data • The important of open source and APIs • Efficient techniques for data collection: network, compute, storage • SLAs, SLOs, and SLIs • Building context and automated feedback • The mechanics of how Lightstep leverages telemetry data • Operational issues that Lightstep addresses • ML capabilities • What constitutes an anomaly? • RCA, policy, and issue resolution • Lightstep benefits • Accelerate resolution while controlling costs Slide 12 © 2020 Enterprise Management Associates, Inc.
- 13. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Recommendations for Driving Observability • Adopt tools that can support the complexity of deep systems • Tools that take unified and nonhierarchical approaches to data collection • Tools that put Golden Signals and end user performance in context • Tools selected must support real-time analysis (support shift right) • Select tools that shift left and drive developer value • Observability tools must be easy to implement • And must provide incremental value • Seek out tools that continually adjust SLOs in supervised and unsupervised ways Slide 13 © 2020 Enterprise Management Associates, Inc.
- 14. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING What is Your Observability Readiness? • How important is microservice development going forward? • Do your current operational analytic tools support deep system data collection and analytics? • How complete is your observability solution? • Are you leveraging ML capabilities for ongoing analytic optimization? • How cost effective is your current observability solution? Slide 14 © 2020 Enterprise Management Associates, Inc.
- 15. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Observability Conclusions Key Tenets of Operational Analytics in Modern Application Development Slide 15 © 2020 Enterprise Management Associates, Inc. InnovationCloud Native Computing Continuous ImprovementAbstraction DecompositionSoC Microservices OSS APIs Modern Application Development Manageability VelocityPolicy Observability Governance Data Other Deep System Orchestration Data Variety Volume AI & ML
- 16. IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Q&A – Log Your Questions in the Q&A Panel Try Lightstep’s interactive sandbox where you can debug an iOS error or resolve a performance regression in less than 10 minutes. Go to: https://lightstep.com/play/ © 2020 Enterprise Management Associates, Inc.Slide 16