SlideShare a Scribd company logo
Observability, Distributed Tracing,
and Open Source
The Missing Primer
2
https://laprensasa.com/culture/art-music/mozart-festival-texas-returns-uiw/
3
4
5
• DanielKhan
daniel.khan@dynatrace.com
@dkhan
• Dir. TechnologyStrategy @Dynatrace
• Everything Open Source Monitoring &
standards& our contributionsto it
• Chairof W3C Trace Context
About me
6
Why I am doing this talk
Distributed
Tracing
Observability
W3C Trace
Context
OpenCensus
OpenTracing
OpenTelemetry
Metrics
Span
Trace
7
Application
In the Beginning there was the Monolith
Presentation
Business Logic
Data Access
Database
Services
Presentation
API Gateway
Auth Inventory CartAccount
Offers Shipping CheckoutStatus
Wire
8
Developmentin a Microservices World
Cart
Dev
Preproduction
Cart Auth InventoryAccount
Offers Shopping CheckoutStatus
Push
Cart
• Latency
• Response Time
• Error Rate
• Number of queries
KPI’s
9
Metrics
Source: https://techblog.commercetools.com/adding-consistency-and-automation-to-grafana-e99eb374fe40
… containtime correlated datapoints
• Counter
Monotonously increasing values
Think: Odometer
• Gauge
Increasing and decreasing values
Think: Tachometer
• Histogram
Groups values into buckets
Think: Knock events 0-50mph, 51-100mph, …
10
Collecting and Charting Metrics
11
Error
242
Success
1302
Cart Service
12
Complecity has movedto the NetworkLayer
Client API GW Service
Service
Service
Service
Service
ServiceCart
Which requests lead to an error in our cart service?
Trace
a42b a42b
a42b
a42b
a42b
a42b
a42b
a42b
a42b = Trace Context
13
A Trace is a Tree of Spans
Trace
Span
Span
Span
Click GW API
Spans represent a single operationand containmetadatalike the HTTP method, or a databasequery, or an error code
JDBC
Span: callDB()
Span: JDBC call
14
Trace Context Propagation
Cartsa42b a42b
Extract
Inject
In process propagation
Auto Instrumentation
• Zipkin
• Sleuth
• OpenTelemetry
• Commercial
• …
15
Trace Context Header Formats
• Proprietary
• B3 Header (Zipkin)
• W3C Trace Context
What’sthe header name and what does it contain?
16
W3C Trace Context
Service A API GW Service B
Trace
Service C
OpenTelemetry AWS Zipkin OpenTelemetry
Goal: All monitoring systems and middlewaresagree on one format for trace context propagation
Span
Span
Span
Span
17
W3C Trace Context Format
traceparent: 00-0af7651916cd43dd8448eb211c80319c-00f067aa0ba902b7-01 tracestate:
rojo=00f067aa0ba902b7,congo=t61rcWkgMzE
Version TraceID ParentID Flags
18
Data collection
So far we just instrumentedthe code to propagatecontext but no data has been collected
Trace
Span
Span
Span
Agent Agent Agent
Click GW API
TraceContext TraceContext
MonitoringSystem Storage
19
Data Collection & PresentationSystems
Solution Agents Instrumentation Storage Presentation
Zipkin / Sleuth + + + +
Jaeger - - + +
OpenTelemetry + + - -
Commercial + + + +
20
Zipkin
21
Jaeger
22
Commercial
23
Entity Model Based Service Flow
24
Detecting Errors
25
Solving our Cart Problem
Client API GW Service
Service
Service
Service
Service
ServiceCart
Trace
Client
Service
Currency
Cart
API GW
GET: Currency=EURO
26
What we did to Solve the Problem
1. We used metrics to learn about a problem
2. We used distributedtracing to pass along a unique ID per trace
• For that,we used auto instrumentationto extract and inject the trace ID
3. We used a monitoring system and its agentsto collected traces and we could filter transactionsthatproduced an
error
4. We looked into the metadata of such a transactionto identify how it differs from succeeding ones
27
You’ve mentionedOpenTelemetry …
+ =
In early 2019 OpenCensusand OpenTracingmerged into OpenTelemetry
Metrics, Traces, Logs
28
APIs SDKs Exporters Collector
29
30
31
32
33
OpenTelemetry – Developer usecases
• Cloud nativemicroservices architectures are hard
to trace and debug during development
• In developmentOpenTelemetry can be used to
either
• manuallycreate spans to trace certain
execution paths
• use provided auto-instrumentation tomake
a system observable
• As backend and UI, Jaeger is the most popular
tool. It’s open source and solely displaystraces
34
OpenTelemetry – in Production
• Provides just a fraction of what modern tools provide
• Traces
• Metrics
• Logs
• Topology
• Behavior
• Code level visibility
• Metadata
• Manual instrumentation codeneeds to be kept up-to-date
• A backend needs to be maintained
• No support model if instrumentation breaksproductioncode
• No enterprise features (access control, throttling, scaling, …)
35
Why do Vendors Care then?
36
OpenTelemetry Company Contribution Stats
Google
Microsoft
Dynatrace
37
38
What happens when we add support for a new framework?
• Today, our engineers reverse engineer frameworks to add
instrumentationsupport to them
• Every time an update is released, the instrumentationcode is
being tested.
• In case of issues, it goes back to the developmentteam who
needs to fix it and deploy an update.
• The whole process is automated and transparent to the customer ☺
• This is costly and time consuming
39
In-process tracing
Click GW API
MonitoringSystem
Trace
Span
Span
Span
HZQ
Span: doHZQ()()
Span: HZQ call
OTEL HZQ
Wrapper
40
“We want every platform and library to
be pre-instrumented with
OpenTelemetry and we’re committed to
making this as easy as possible.”
Sergey Kanzhelev (Google)
41
What is Observability and how does it differ fromMonitoring?
1. In control theory, observability is a measure of how well internal states of a system can be inferred from
knowledgeof its external outputs.
Source: Wikipedia
2. In software development, observabilityisachieved by adding code (instrumentation)that emits telemetry
data.
3. Monitoringis the act of displayingand analyzing this telemetry data.
4. Monitoringalone can tell you that there is a problem.
E.g. ”We see that some users experience a 50% higher response time on check-out”
5. Observablityhelps finding the root cause (the why) by providingdatathat can be correlatedand analysed
freely even if this problem is completelynew to you (unknown unknowns)
E.g. “The response time of the checkout increases exponentially with the number of items in the basket,
because of a misplaced for loop that executes the same database query times the number of items for every
item in the basket”
42
Putting it all Together
• Metrics can help you to learn that there is a problem
• Distributedtracing becomes increasingly importantto understandmulti-tier execution paths and root causes
of problems
• Developersnow rely on metrics and traces to understandhow their service functionsin their microservice
architectures
• Pure Open Source solutionsare viable for pre-prod environments
• Standardization isthe only way to tackle today’scomplexity and Open Source is the key driver
• Vendorsare prepared to tap intodata collected by Open Source standard toolsto add enterprise features on
top to support web-scale workloads
43
dynatrace.com
@dkhan
daniel.khan@dynatrace.com
Thank you!

More Related Content

What's hot (20)

Meetup OpenTelemetry Intro
Meetup OpenTelemetry IntroMeetup OpenTelemetry Intro
Meetup OpenTelemetry Intro
DimitrisFinas1
 
OpenTelemetry Introduction
OpenTelemetry Introduction OpenTelemetry Introduction
OpenTelemetry Introduction
DimitrisFinas1
 
Kubernetes PPT.pptx
Kubernetes PPT.pptxKubernetes PPT.pptx
Kubernetes PPT.pptx
ssuser0cc9131
 
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
Kubernetes Concepts And Architecture Powerpoint Presentation SlidesKubernetes Concepts And Architecture Powerpoint Presentation Slides
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
SlideTeam
 
Improve monitoring and observability for kubernetes with oss tools
Improve monitoring and observability for kubernetes with oss toolsImprove monitoring and observability for kubernetes with oss tools
Improve monitoring and observability for kubernetes with oss tools
Nilesh Gule
 
Exploring the power of OpenTelemetry on Kubernetes
Exploring the power of OpenTelemetry on KubernetesExploring the power of OpenTelemetry on Kubernetes
Exploring the power of OpenTelemetry on Kubernetes
Red Hat Developers
 
Migration d'une Architecture Microservice vers une Architecture Event-Driven ...
Migration d'une Architecture Microservice vers une Architecture Event-Driven ...Migration d'une Architecture Microservice vers une Architecture Event-Driven ...
Migration d'une Architecture Microservice vers une Architecture Event-Driven ...
Daniel Rene FOUOMENE PEWO
 
Intro to open source observability with grafana, prometheus, loki, and tempo(...
Intro to open source observability with grafana, prometheus, loki, and tempo(...Intro to open source observability with grafana, prometheus, loki, and tempo(...
Intro to open source observability with grafana, prometheus, loki, and tempo(...
LibbySchulze
 
Distributed Tracing
Distributed TracingDistributed Tracing
Distributed Tracing
distributedtracing
 
Introduction to helm
Introduction to helmIntroduction to helm
Introduction to helm
Jeeva Chelladhurai
 
Prometheus and Grafana
Prometheus and GrafanaPrometheus and Grafana
Prometheus and Grafana
Lhouceine OUHAMZA
 
Introduction to Distributed Tracing
Introduction to Distributed TracingIntroduction to Distributed Tracing
Introduction to Distributed Tracing
petabridge
 
Docker Kubernetes Istio
Docker Kubernetes IstioDocker Kubernetes Istio
Docker Kubernetes Istio
Araf Karsh Hamid
 
DevSecOps 101
DevSecOps 101DevSecOps 101
DevSecOps 101
Narudom Roongsiriwong, CISSP
 
Adopting OpenTelemetry
Adopting OpenTelemetryAdopting OpenTelemetry
Adopting OpenTelemetry
Vincent Behar
 
Application Performance Monitoring with OpenTelemetry
Application Performance Monitoring with OpenTelemetryApplication Performance Monitoring with OpenTelemetry
Application Performance Monitoring with OpenTelemetry
Jan MikeĹĄ
 
The Patterns of Distributed Logging and Containers
The Patterns of Distributed Logging and ContainersThe Patterns of Distributed Logging and Containers
The Patterns of Distributed Logging and Containers
SATOSHI TAGOMORI
 
Getting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and GrafanaGetting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and Grafana
Syah Dwi Prihatmoko
 
DevOps Monitoring and Alerting
DevOps Monitoring and AlertingDevOps Monitoring and Alerting
DevOps Monitoring and Alerting
Khairul Zebua
 
An Introduction to Kubernetes
An Introduction to KubernetesAn Introduction to Kubernetes
An Introduction to Kubernetes
Imesh Gunaratne
 
Meetup OpenTelemetry Intro
Meetup OpenTelemetry IntroMeetup OpenTelemetry Intro
Meetup OpenTelemetry Intro
DimitrisFinas1
 
OpenTelemetry Introduction
OpenTelemetry Introduction OpenTelemetry Introduction
OpenTelemetry Introduction
DimitrisFinas1
 
Kubernetes PPT.pptx
Kubernetes PPT.pptxKubernetes PPT.pptx
Kubernetes PPT.pptx
ssuser0cc9131
 
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
Kubernetes Concepts And Architecture Powerpoint Presentation SlidesKubernetes Concepts And Architecture Powerpoint Presentation Slides
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
SlideTeam
 
Improve monitoring and observability for kubernetes with oss tools
Improve monitoring and observability for kubernetes with oss toolsImprove monitoring and observability for kubernetes with oss tools
Improve monitoring and observability for kubernetes with oss tools
Nilesh Gule
 
Exploring the power of OpenTelemetry on Kubernetes
Exploring the power of OpenTelemetry on KubernetesExploring the power of OpenTelemetry on Kubernetes
Exploring the power of OpenTelemetry on Kubernetes
Red Hat Developers
 
Migration d'une Architecture Microservice vers une Architecture Event-Driven ...
Migration d'une Architecture Microservice vers une Architecture Event-Driven ...Migration d'une Architecture Microservice vers une Architecture Event-Driven ...
Migration d'une Architecture Microservice vers une Architecture Event-Driven ...
Daniel Rene FOUOMENE PEWO
 
Intro to open source observability with grafana, prometheus, loki, and tempo(...
Intro to open source observability with grafana, prometheus, loki, and tempo(...Intro to open source observability with grafana, prometheus, loki, and tempo(...
Intro to open source observability with grafana, prometheus, loki, and tempo(...
LibbySchulze
 
Prometheus and Grafana
Prometheus and GrafanaPrometheus and Grafana
Prometheus and Grafana
Lhouceine OUHAMZA
 
Introduction to Distributed Tracing
Introduction to Distributed TracingIntroduction to Distributed Tracing
Introduction to Distributed Tracing
petabridge
 
Docker Kubernetes Istio
Docker Kubernetes IstioDocker Kubernetes Istio
Docker Kubernetes Istio
Araf Karsh Hamid
 
Adopting OpenTelemetry
Adopting OpenTelemetryAdopting OpenTelemetry
Adopting OpenTelemetry
Vincent Behar
 
Application Performance Monitoring with OpenTelemetry
Application Performance Monitoring with OpenTelemetryApplication Performance Monitoring with OpenTelemetry
Application Performance Monitoring with OpenTelemetry
Jan MikeĹĄ
 
The Patterns of Distributed Logging and Containers
The Patterns of Distributed Logging and ContainersThe Patterns of Distributed Logging and Containers
The Patterns of Distributed Logging and Containers
SATOSHI TAGOMORI
 
Getting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and GrafanaGetting Started Monitoring with Prometheus and Grafana
Getting Started Monitoring with Prometheus and Grafana
Syah Dwi Prihatmoko
 
DevOps Monitoring and Alerting
DevOps Monitoring and AlertingDevOps Monitoring and Alerting
DevOps Monitoring and Alerting
Khairul Zebua
 
An Introduction to Kubernetes
An Introduction to KubernetesAn Introduction to Kubernetes
An Introduction to Kubernetes
Imesh Gunaratne
 

Similar to Observability, Distributed Tracing, and Open Source: The Missing Primer (20)

"Distributed Tracing: New DevOps Foundation" by Jayesh Ahire
"Distributed Tracing: New DevOps Foundation" by Jayesh Ahire  "Distributed Tracing: New DevOps Foundation" by Jayesh Ahire
"Distributed Tracing: New DevOps Foundation" by Jayesh Ahire
CodeOps Technologies LLP
 
Distributed Tracing: New DevOps Foundation
Distributed Tracing: New DevOps FoundationDistributed Tracing: New DevOps Foundation
Distributed Tracing: New DevOps Foundation
CodeOps Technologies LLP
 
Observability for Application Developers (1)-1.pptx
Observability for Application Developers (1)-1.pptxObservability for Application Developers (1)-1.pptx
Observability for Application Developers (1)-1.pptx
OpsTree solutions
 
Manage Microservices Chaos and Complexity with Observability
Manage Microservices Chaos and Complexity with ObservabilityManage Microservices Chaos and Complexity with Observability
Manage Microservices Chaos and Complexity with Observability
NGINX, Inc.
 
Agile Gurugram 2023 | Observability for Modern Applications. How does it help...
Agile Gurugram 2023 | Observability for Modern Applications. How does it help...Agile Gurugram 2023 | Observability for Modern Applications. How does it help...
Agile Gurugram 2023 | Observability for Modern Applications. How does it help...
AgileNetwork
 
DockerCon SF 2019 - Observability Workshop
DockerCon SF 2019 - Observability WorkshopDockerCon SF 2019 - Observability Workshop
DockerCon SF 2019 - Observability Workshop
Kevin Crawley
 
What Is OpenTelemetry? A Complete Introduction
What Is OpenTelemetry? A Complete IntroductionWhat Is OpenTelemetry? A Complete Introduction
What Is OpenTelemetry? A Complete Introduction
Ciente
 
stackconf 2023 | Practical introduction to OpenTelemetry tracing by Nicolas F...
stackconf 2023 | Practical introduction to OpenTelemetry tracing by Nicolas F...stackconf 2023 | Practical introduction to OpenTelemetry tracing by Nicolas F...
stackconf 2023 | Practical introduction to OpenTelemetry tracing by Nicolas F...
NETWAYS
 
Beginner's Guide to Observability@Devoxx PL 2024
Beginner's  Guide to Observability@Devoxx PL 2024Beginner's  Guide to Observability@Devoxx PL 2024
Beginner's Guide to Observability@Devoxx PL 2024
michniczscribd
 
Everything You wanted to Know About Distributed Tracing
Everything You wanted to Know About Distributed TracingEverything You wanted to Know About Distributed Tracing
Everything You wanted to Know About Distributed Tracing
Amuhinda Hungai
 
stackconf 2022: Open Source for Better Observability
stackconf 2022: Open Source for Better Observabilitystackconf 2022: Open Source for Better Observability
stackconf 2022: Open Source for Better Observability
NETWAYS
 
ThroughTheLookingGlass_EffectiveObservability.pptx
ThroughTheLookingGlass_EffectiveObservability.pptxThroughTheLookingGlass_EffectiveObservability.pptx
ThroughTheLookingGlass_EffectiveObservability.pptx
Grace Jansen
 
Introduction to Open Telemetry as Observability Library
Introduction to Open  Telemetry as Observability LibraryIntroduction to Open  Telemetry as Observability Library
Introduction to Open Telemetry as Observability Library
Tonny Adhi Sabastian
 
Adopting Open Telemetry as Distributed Tracer on your Microservices at Kubern...
Adopting Open Telemetry as Distributed Tracer on your Microservices at Kubern...Adopting Open Telemetry as Distributed Tracer on your Microservices at Kubern...
Adopting Open Telemetry as Distributed Tracer on your Microservices at Kubern...
Tonny Adhi Sabastian
 
"Introducing Distributed Tracing in a Large Software System", Kostiantyn Sha...
"Introducing Distributed Tracing in a Large Software System",  Kostiantyn Sha..."Introducing Distributed Tracing in a Large Software System",  Kostiantyn Sha...
"Introducing Distributed Tracing in a Large Software System", Kostiantyn Sha...
Fwdays
 
WJAX 2019 - Taking Distributed Tracing to the next level
WJAX 2019 - Taking Distributed Tracing to the next levelWJAX 2019 - Taking Distributed Tracing to the next level
WJAX 2019 - Taking Distributed Tracing to the next level
Frank Pfleger
 
2307 - DevBCN - Otel 101_compressed.pdf
2307 - DevBCN - Otel 101_compressed.pdf2307 - DevBCN - Otel 101_compressed.pdf
2307 - DevBCN - Otel 101_compressed.pdf
DimitrisFinas1
 
ADDO Open Source Observability Tools
ADDO Open Source Observability Tools ADDO Open Source Observability Tools
ADDO Open Source Observability Tools
Mickey Boxell
 
Java il spanning services 2019
Java il   spanning services 2019Java il   spanning services 2019
Java il spanning services 2019
Yair Galler
 
Migrating from OpenTracing to OpenTelemetry - Kubernetes Community Days Munic...
Migrating from OpenTracing to OpenTelemetry - Kubernetes Community Days Munic...Migrating from OpenTracing to OpenTelemetry - Kubernetes Community Days Munic...
Migrating from OpenTracing to OpenTelemetry - Kubernetes Community Days Munic...
SonjaChevre
 
"Distributed Tracing: New DevOps Foundation" by Jayesh Ahire
"Distributed Tracing: New DevOps Foundation" by Jayesh Ahire  "Distributed Tracing: New DevOps Foundation" by Jayesh Ahire
"Distributed Tracing: New DevOps Foundation" by Jayesh Ahire
CodeOps Technologies LLP
 
Distributed Tracing: New DevOps Foundation
Distributed Tracing: New DevOps FoundationDistributed Tracing: New DevOps Foundation
Distributed Tracing: New DevOps Foundation
CodeOps Technologies LLP
 
Observability for Application Developers (1)-1.pptx
Observability for Application Developers (1)-1.pptxObservability for Application Developers (1)-1.pptx
Observability for Application Developers (1)-1.pptx
OpsTree solutions
 
Manage Microservices Chaos and Complexity with Observability
Manage Microservices Chaos and Complexity with ObservabilityManage Microservices Chaos and Complexity with Observability
Manage Microservices Chaos and Complexity with Observability
NGINX, Inc.
 
Agile Gurugram 2023 | Observability for Modern Applications. How does it help...
Agile Gurugram 2023 | Observability for Modern Applications. How does it help...Agile Gurugram 2023 | Observability for Modern Applications. How does it help...
Agile Gurugram 2023 | Observability for Modern Applications. How does it help...
AgileNetwork
 
DockerCon SF 2019 - Observability Workshop
DockerCon SF 2019 - Observability WorkshopDockerCon SF 2019 - Observability Workshop
DockerCon SF 2019 - Observability Workshop
Kevin Crawley
 
What Is OpenTelemetry? A Complete Introduction
What Is OpenTelemetry? A Complete IntroductionWhat Is OpenTelemetry? A Complete Introduction
What Is OpenTelemetry? A Complete Introduction
Ciente
 
stackconf 2023 | Practical introduction to OpenTelemetry tracing by Nicolas F...
stackconf 2023 | Practical introduction to OpenTelemetry tracing by Nicolas F...stackconf 2023 | Practical introduction to OpenTelemetry tracing by Nicolas F...
stackconf 2023 | Practical introduction to OpenTelemetry tracing by Nicolas F...
NETWAYS
 
Beginner's Guide to Observability@Devoxx PL 2024
Beginner's  Guide to Observability@Devoxx PL 2024Beginner's  Guide to Observability@Devoxx PL 2024
Beginner's Guide to Observability@Devoxx PL 2024
michniczscribd
 
Everything You wanted to Know About Distributed Tracing
Everything You wanted to Know About Distributed TracingEverything You wanted to Know About Distributed Tracing
Everything You wanted to Know About Distributed Tracing
Amuhinda Hungai
 
stackconf 2022: Open Source for Better Observability
stackconf 2022: Open Source for Better Observabilitystackconf 2022: Open Source for Better Observability
stackconf 2022: Open Source for Better Observability
NETWAYS
 
ThroughTheLookingGlass_EffectiveObservability.pptx
ThroughTheLookingGlass_EffectiveObservability.pptxThroughTheLookingGlass_EffectiveObservability.pptx
ThroughTheLookingGlass_EffectiveObservability.pptx
Grace Jansen
 
Introduction to Open Telemetry as Observability Library
Introduction to Open  Telemetry as Observability LibraryIntroduction to Open  Telemetry as Observability Library
Introduction to Open Telemetry as Observability Library
Tonny Adhi Sabastian
 
Adopting Open Telemetry as Distributed Tracer on your Microservices at Kubern...
Adopting Open Telemetry as Distributed Tracer on your Microservices at Kubern...Adopting Open Telemetry as Distributed Tracer on your Microservices at Kubern...
Adopting Open Telemetry as Distributed Tracer on your Microservices at Kubern...
Tonny Adhi Sabastian
 
"Introducing Distributed Tracing in a Large Software System", Kostiantyn Sha...
"Introducing Distributed Tracing in a Large Software System",  Kostiantyn Sha..."Introducing Distributed Tracing in a Large Software System",  Kostiantyn Sha...
"Introducing Distributed Tracing in a Large Software System", Kostiantyn Sha...
Fwdays
 
WJAX 2019 - Taking Distributed Tracing to the next level
WJAX 2019 - Taking Distributed Tracing to the next levelWJAX 2019 - Taking Distributed Tracing to the next level
WJAX 2019 - Taking Distributed Tracing to the next level
Frank Pfleger
 
2307 - DevBCN - Otel 101_compressed.pdf
2307 - DevBCN - Otel 101_compressed.pdf2307 - DevBCN - Otel 101_compressed.pdf
2307 - DevBCN - Otel 101_compressed.pdf
DimitrisFinas1
 
ADDO Open Source Observability Tools
ADDO Open Source Observability Tools ADDO Open Source Observability Tools
ADDO Open Source Observability Tools
Mickey Boxell
 
Java il spanning services 2019
Java il   spanning services 2019Java il   spanning services 2019
Java il spanning services 2019
Yair Galler
 
Migrating from OpenTracing to OpenTelemetry - Kubernetes Community Days Munic...
Migrating from OpenTracing to OpenTelemetry - Kubernetes Community Days Munic...Migrating from OpenTracing to OpenTelemetry - Kubernetes Community Days Munic...
Migrating from OpenTracing to OpenTelemetry - Kubernetes Community Days Munic...
SonjaChevre
 
Ad

More from VMware Tanzu (20)

Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14
VMware Tanzu
 
What AI Means For Your Product Strategy And What To Do About It
What AI Means For Your Product Strategy And What To Do About ItWhat AI Means For Your Product Strategy And What To Do About It
What AI Means For Your Product Strategy And What To Do About It
VMware Tanzu
 
Make the Right Thing the Obvious Thing at Cardinal Health 2023
Make the Right Thing the Obvious Thing at Cardinal Health 2023Make the Right Thing the Obvious Thing at Cardinal Health 2023
Make the Right Thing the Obvious Thing at Cardinal Health 2023
VMware Tanzu
 
Enhancing DevEx and Simplifying Operations at Scale
Enhancing DevEx and Simplifying Operations at ScaleEnhancing DevEx and Simplifying Operations at Scale
Enhancing DevEx and Simplifying Operations at Scale
VMware Tanzu
 
Spring Update | July 2023
Spring Update | July 2023Spring Update | July 2023
Spring Update | July 2023
VMware Tanzu
 
Platforms, Platform Engineering, & Platform as a Product
Platforms, Platform Engineering, & Platform as a ProductPlatforms, Platform Engineering, & Platform as a Product
Platforms, Platform Engineering, & Platform as a Product
VMware Tanzu
 
Building Cloud Ready Apps
Building Cloud Ready AppsBuilding Cloud Ready Apps
Building Cloud Ready Apps
VMware Tanzu
 
Spring Boot 3 And Beyond
Spring Boot 3 And BeyondSpring Boot 3 And Beyond
Spring Boot 3 And Beyond
VMware Tanzu
 
Spring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdf
Spring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdfSpring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdf
Spring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdf
VMware Tanzu
 
Simplify and Scale Enterprise Apps in the Cloud | Boston 2023
Simplify and Scale Enterprise Apps in the Cloud | Boston 2023Simplify and Scale Enterprise Apps in the Cloud | Boston 2023
Simplify and Scale Enterprise Apps in the Cloud | Boston 2023
VMware Tanzu
 
Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023
Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023
Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023
VMware Tanzu
 
tanzu_developer_connect.pptx
tanzu_developer_connect.pptxtanzu_developer_connect.pptx
tanzu_developer_connect.pptx
VMware Tanzu
 
Tanzu Virtual Developer Connect Workshop - French
Tanzu Virtual Developer Connect Workshop - FrenchTanzu Virtual Developer Connect Workshop - French
Tanzu Virtual Developer Connect Workshop - French
VMware Tanzu
 
Tanzu Developer Connect Workshop - English
Tanzu Developer Connect Workshop - EnglishTanzu Developer Connect Workshop - English
Tanzu Developer Connect Workshop - English
VMware Tanzu
 
Virtual Developer Connect Workshop - English
Virtual Developer Connect Workshop - EnglishVirtual Developer Connect Workshop - English
Virtual Developer Connect Workshop - English
VMware Tanzu
 
Tanzu Developer Connect - French
Tanzu Developer Connect - FrenchTanzu Developer Connect - French
Tanzu Developer Connect - French
VMware Tanzu
 
Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023
Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023
Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023
VMware Tanzu
 
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring BootSpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
VMware Tanzu
 
SpringOne Tour: The Influential Software Engineer
SpringOne Tour: The Influential Software EngineerSpringOne Tour: The Influential Software Engineer
SpringOne Tour: The Influential Software Engineer
VMware Tanzu
 
SpringOne Tour: Domain-Driven Design: Theory vs Practice
SpringOne Tour: Domain-Driven Design: Theory vs PracticeSpringOne Tour: Domain-Driven Design: Theory vs Practice
SpringOne Tour: Domain-Driven Design: Theory vs Practice
VMware Tanzu
 
Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14
VMware Tanzu
 
What AI Means For Your Product Strategy And What To Do About It
What AI Means For Your Product Strategy And What To Do About ItWhat AI Means For Your Product Strategy And What To Do About It
What AI Means For Your Product Strategy And What To Do About It
VMware Tanzu
 
Make the Right Thing the Obvious Thing at Cardinal Health 2023
Make the Right Thing the Obvious Thing at Cardinal Health 2023Make the Right Thing the Obvious Thing at Cardinal Health 2023
Make the Right Thing the Obvious Thing at Cardinal Health 2023
VMware Tanzu
 
Enhancing DevEx and Simplifying Operations at Scale
Enhancing DevEx and Simplifying Operations at ScaleEnhancing DevEx and Simplifying Operations at Scale
Enhancing DevEx and Simplifying Operations at Scale
VMware Tanzu
 
Spring Update | July 2023
Spring Update | July 2023Spring Update | July 2023
Spring Update | July 2023
VMware Tanzu
 
Platforms, Platform Engineering, & Platform as a Product
Platforms, Platform Engineering, & Platform as a ProductPlatforms, Platform Engineering, & Platform as a Product
Platforms, Platform Engineering, & Platform as a Product
VMware Tanzu
 
Building Cloud Ready Apps
Building Cloud Ready AppsBuilding Cloud Ready Apps
Building Cloud Ready Apps
VMware Tanzu
 
Spring Boot 3 And Beyond
Spring Boot 3 And BeyondSpring Boot 3 And Beyond
Spring Boot 3 And Beyond
VMware Tanzu
 
Spring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdf
Spring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdfSpring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdf
Spring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdf
VMware Tanzu
 
Simplify and Scale Enterprise Apps in the Cloud | Boston 2023
Simplify and Scale Enterprise Apps in the Cloud | Boston 2023Simplify and Scale Enterprise Apps in the Cloud | Boston 2023
Simplify and Scale Enterprise Apps in the Cloud | Boston 2023
VMware Tanzu
 
Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023
Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023
Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023
VMware Tanzu
 
tanzu_developer_connect.pptx
tanzu_developer_connect.pptxtanzu_developer_connect.pptx
tanzu_developer_connect.pptx
VMware Tanzu
 
Tanzu Virtual Developer Connect Workshop - French
Tanzu Virtual Developer Connect Workshop - FrenchTanzu Virtual Developer Connect Workshop - French
Tanzu Virtual Developer Connect Workshop - French
VMware Tanzu
 
Tanzu Developer Connect Workshop - English
Tanzu Developer Connect Workshop - EnglishTanzu Developer Connect Workshop - English
Tanzu Developer Connect Workshop - English
VMware Tanzu
 
Virtual Developer Connect Workshop - English
Virtual Developer Connect Workshop - EnglishVirtual Developer Connect Workshop - English
Virtual Developer Connect Workshop - English
VMware Tanzu
 
Tanzu Developer Connect - French
Tanzu Developer Connect - FrenchTanzu Developer Connect - French
Tanzu Developer Connect - French
VMware Tanzu
 
Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023
Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023
Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023
VMware Tanzu
 
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring BootSpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
VMware Tanzu
 
SpringOne Tour: The Influential Software Engineer
SpringOne Tour: The Influential Software EngineerSpringOne Tour: The Influential Software Engineer
SpringOne Tour: The Influential Software Engineer
VMware Tanzu
 
SpringOne Tour: Domain-Driven Design: Theory vs Practice
SpringOne Tour: Domain-Driven Design: Theory vs PracticeSpringOne Tour: Domain-Driven Design: Theory vs Practice
SpringOne Tour: Domain-Driven Design: Theory vs Practice
VMware Tanzu
 
Ad

Recently uploaded (20)

iOS Developer Resume 2025 | Pramod Kumar
iOS Developer Resume 2025 | Pramod KumariOS Developer Resume 2025 | Pramod Kumar
iOS Developer Resume 2025 | Pramod Kumar
Pramod Kumar
 
From Chaos to Clarity - Designing (AI-Ready) APIs with APIOps Cycles
From Chaos to Clarity - Designing (AI-Ready) APIs with APIOps CyclesFrom Chaos to Clarity - Designing (AI-Ready) APIs with APIOps Cycles
From Chaos to Clarity - Designing (AI-Ready) APIs with APIOps Cycles
Marjukka Niinioja
 
Leveraging Foundation Models to Infer Intents
Leveraging Foundation Models to Infer IntentsLeveraging Foundation Models to Infer Intents
Leveraging Foundation Models to Infer Intents
Keheliya Gallaba
 
How to Generate Financial Statements in QuickBooks Like a Pro (1).pdf
How to Generate Financial Statements in QuickBooks Like a Pro (1).pdfHow to Generate Financial Statements in QuickBooks Like a Pro (1).pdf
How to Generate Financial Statements in QuickBooks Like a Pro (1).pdf
QuickBooks Training
 
Agentic Techniques in Retrieval-Augmented Generation with Azure AI Search
Agentic Techniques in Retrieval-Augmented Generation with Azure AI SearchAgentic Techniques in Retrieval-Augmented Generation with Azure AI Search
Agentic Techniques in Retrieval-Augmented Generation with Azure AI Search
Maxim Salnikov
 
Marketo & Dynamics can be Most Excellent to Each Other – The Sequel
Marketo & Dynamics can be Most Excellent to Each Other – The SequelMarketo & Dynamics can be Most Excellent to Each Other – The Sequel
Marketo & Dynamics can be Most Excellent to Each Other – The Sequel
BradBedford3
 
Design by Contract - Building Robust Software with Contract-First Development
Design by Contract - Building Robust Software with Contract-First DevelopmentDesign by Contract - Building Robust Software with Contract-First Development
Design by Contract - Building Robust Software with Contract-First Development
Par-Tec S.p.A.
 
Online Queue Management System for Public Service Offices [Focused on Municip...
Online Queue Management System for Public Service Offices [Focused on Municip...Online Queue Management System for Public Service Offices [Focused on Municip...
Online Queue Management System for Public Service Offices [Focused on Municip...
Rishab Acharya
 
14 Years of Developing nCine - An Open Source 2D Game Framework
14 Years of Developing nCine - An Open Source 2D Game Framework14 Years of Developing nCine - An Open Source 2D Game Framework
14 Years of Developing nCine - An Open Source 2D Game Framework
Angelo Theodorou
 
Eliminate the complexities of Event-Driven Architecture with Domain-Driven De...
Eliminate the complexities of Event-Driven Architecture with Domain-Driven De...Eliminate the complexities of Event-Driven Architecture with Domain-Driven De...
Eliminate the complexities of Event-Driven Architecture with Domain-Driven De...
SheenBrisals
 
Topic 26 Security Testing Considerations.pptx
Topic 26 Security Testing Considerations.pptxTopic 26 Security Testing Considerations.pptx
Topic 26 Security Testing Considerations.pptx
marutnand8
 
Providing Better Biodiversity Through Better Data
Providing Better Biodiversity Through Better DataProviding Better Biodiversity Through Better Data
Providing Better Biodiversity Through Better Data
Safe Software
 
COBOL Programming with VSCode - IBM Certificate
COBOL Programming with VSCode - IBM CertificateCOBOL Programming with VSCode - IBM Certificate
COBOL Programming with VSCode - IBM Certificate
VICTOR MAESTRE RAMIREZ
 
How Insurance Policy Management Software Streamlines Operations
How Insurance Policy Management Software Streamlines OperationsHow Insurance Policy Management Software Streamlines Operations
How Insurance Policy Management Software Streamlines Operations
Insurance Tech Services
 
Top 11 Fleet Management Software Providers in 2025 (2).pdf
Top 11 Fleet Management Software Providers in 2025 (2).pdfTop 11 Fleet Management Software Providers in 2025 (2).pdf
Top 11 Fleet Management Software Providers in 2025 (2).pdf
Trackobit
 
Automating Map Production With FME and Python
Automating Map Production With FME and PythonAutomating Map Production With FME and Python
Automating Map Production With FME and Python
Safe Software
 
Integration Ignited Redefining Event-Driven Architecture at Wix - EventCentric
Integration Ignited Redefining Event-Driven Architecture at Wix - EventCentricIntegration Ignited Redefining Event-Driven Architecture at Wix - EventCentric
Integration Ignited Redefining Event-Driven Architecture at Wix - EventCentric
Natan Silnitsky
 
Best Inbound Call Tracking Software for Small Businesses
Best Inbound Call Tracking Software for Small BusinessesBest Inbound Call Tracking Software for Small Businesses
Best Inbound Call Tracking Software for Small Businesses
TheTelephony
 
AI and Deep Learning with NVIDIA Technologies
AI and Deep Learning with NVIDIA TechnologiesAI and Deep Learning with NVIDIA Technologies
AI and Deep Learning with NVIDIA Technologies
SandeepKS52
 
How AI Can Improve Media Quality Testing Across Platforms (1).pptx
How AI Can Improve Media Quality Testing Across Platforms (1).pptxHow AI Can Improve Media Quality Testing Across Platforms (1).pptx
How AI Can Improve Media Quality Testing Across Platforms (1).pptx
kalichargn70th171
 
iOS Developer Resume 2025 | Pramod Kumar
iOS Developer Resume 2025 | Pramod KumariOS Developer Resume 2025 | Pramod Kumar
iOS Developer Resume 2025 | Pramod Kumar
Pramod Kumar
 
From Chaos to Clarity - Designing (AI-Ready) APIs with APIOps Cycles
From Chaos to Clarity - Designing (AI-Ready) APIs with APIOps CyclesFrom Chaos to Clarity - Designing (AI-Ready) APIs with APIOps Cycles
From Chaos to Clarity - Designing (AI-Ready) APIs with APIOps Cycles
Marjukka Niinioja
 
Leveraging Foundation Models to Infer Intents
Leveraging Foundation Models to Infer IntentsLeveraging Foundation Models to Infer Intents
Leveraging Foundation Models to Infer Intents
Keheliya Gallaba
 
How to Generate Financial Statements in QuickBooks Like a Pro (1).pdf
How to Generate Financial Statements in QuickBooks Like a Pro (1).pdfHow to Generate Financial Statements in QuickBooks Like a Pro (1).pdf
How to Generate Financial Statements in QuickBooks Like a Pro (1).pdf
QuickBooks Training
 
Agentic Techniques in Retrieval-Augmented Generation with Azure AI Search
Agentic Techniques in Retrieval-Augmented Generation with Azure AI SearchAgentic Techniques in Retrieval-Augmented Generation with Azure AI Search
Agentic Techniques in Retrieval-Augmented Generation with Azure AI Search
Maxim Salnikov
 
Marketo & Dynamics can be Most Excellent to Each Other – The Sequel
Marketo & Dynamics can be Most Excellent to Each Other – The SequelMarketo & Dynamics can be Most Excellent to Each Other – The Sequel
Marketo & Dynamics can be Most Excellent to Each Other – The Sequel
BradBedford3
 
Design by Contract - Building Robust Software with Contract-First Development
Design by Contract - Building Robust Software with Contract-First DevelopmentDesign by Contract - Building Robust Software with Contract-First Development
Design by Contract - Building Robust Software with Contract-First Development
Par-Tec S.p.A.
 
Online Queue Management System for Public Service Offices [Focused on Municip...
Online Queue Management System for Public Service Offices [Focused on Municip...Online Queue Management System for Public Service Offices [Focused on Municip...
Online Queue Management System for Public Service Offices [Focused on Municip...
Rishab Acharya
 
14 Years of Developing nCine - An Open Source 2D Game Framework
14 Years of Developing nCine - An Open Source 2D Game Framework14 Years of Developing nCine - An Open Source 2D Game Framework
14 Years of Developing nCine - An Open Source 2D Game Framework
Angelo Theodorou
 
Eliminate the complexities of Event-Driven Architecture with Domain-Driven De...
Eliminate the complexities of Event-Driven Architecture with Domain-Driven De...Eliminate the complexities of Event-Driven Architecture with Domain-Driven De...
Eliminate the complexities of Event-Driven Architecture with Domain-Driven De...
SheenBrisals
 
Topic 26 Security Testing Considerations.pptx
Topic 26 Security Testing Considerations.pptxTopic 26 Security Testing Considerations.pptx
Topic 26 Security Testing Considerations.pptx
marutnand8
 
Providing Better Biodiversity Through Better Data
Providing Better Biodiversity Through Better DataProviding Better Biodiversity Through Better Data
Providing Better Biodiversity Through Better Data
Safe Software
 
COBOL Programming with VSCode - IBM Certificate
COBOL Programming with VSCode - IBM CertificateCOBOL Programming with VSCode - IBM Certificate
COBOL Programming with VSCode - IBM Certificate
VICTOR MAESTRE RAMIREZ
 
How Insurance Policy Management Software Streamlines Operations
How Insurance Policy Management Software Streamlines OperationsHow Insurance Policy Management Software Streamlines Operations
How Insurance Policy Management Software Streamlines Operations
Insurance Tech Services
 
Top 11 Fleet Management Software Providers in 2025 (2).pdf
Top 11 Fleet Management Software Providers in 2025 (2).pdfTop 11 Fleet Management Software Providers in 2025 (2).pdf
Top 11 Fleet Management Software Providers in 2025 (2).pdf
Trackobit
 
Automating Map Production With FME and Python
Automating Map Production With FME and PythonAutomating Map Production With FME and Python
Automating Map Production With FME and Python
Safe Software
 
Integration Ignited Redefining Event-Driven Architecture at Wix - EventCentric
Integration Ignited Redefining Event-Driven Architecture at Wix - EventCentricIntegration Ignited Redefining Event-Driven Architecture at Wix - EventCentric
Integration Ignited Redefining Event-Driven Architecture at Wix - EventCentric
Natan Silnitsky
 
Best Inbound Call Tracking Software for Small Businesses
Best Inbound Call Tracking Software for Small BusinessesBest Inbound Call Tracking Software for Small Businesses
Best Inbound Call Tracking Software for Small Businesses
TheTelephony
 
AI and Deep Learning with NVIDIA Technologies
AI and Deep Learning with NVIDIA TechnologiesAI and Deep Learning with NVIDIA Technologies
AI and Deep Learning with NVIDIA Technologies
SandeepKS52
 
How AI Can Improve Media Quality Testing Across Platforms (1).pptx
How AI Can Improve Media Quality Testing Across Platforms (1).pptxHow AI Can Improve Media Quality Testing Across Platforms (1).pptx
How AI Can Improve Media Quality Testing Across Platforms (1).pptx
kalichargn70th171
 

Observability, Distributed Tracing, and Open Source: The Missing Primer

  • 1. Observability, Distributed Tracing, and Open Source The Missing Primer
  • 3. 3
  • 4. 4
  • 5. 5 • DanielKhan [email protected] @dkhan • Dir. TechnologyStrategy @Dynatrace • Everything Open Source Monitoring & standards& our contributionsto it • Chairof W3C Trace Context About me
  • 6. 6 Why I am doing this talk Distributed Tracing Observability W3C Trace Context OpenCensus OpenTracing OpenTelemetry Metrics Span Trace
  • 7. 7 Application In the Beginning there was the Monolith Presentation Business Logic Data Access Database Services Presentation API Gateway Auth Inventory CartAccount Offers Shipping CheckoutStatus Wire
  • 8. 8 Developmentin a Microservices World Cart Dev Preproduction Cart Auth InventoryAccount Offers Shopping CheckoutStatus Push Cart • Latency • Response Time • Error Rate • Number of queries KPI’s
  • 9. 9 Metrics Source: https://techblog.commercetools.com/adding-consistency-and-automation-to-grafana-e99eb374fe40 … containtime correlated datapoints • Counter Monotonously increasing values Think: Odometer • Gauge Increasing and decreasing values Think: Tachometer • Histogram Groups values into buckets Think: Knock events 0-50mph, 51-100mph, …
  • 12. 12 Complecity has movedto the NetworkLayer Client API GW Service Service Service Service Service ServiceCart Which requests lead to an error in our cart service? Trace a42b a42b a42b a42b a42b a42b a42b a42b a42b = Trace Context
  • 13. 13 A Trace is a Tree of Spans Trace Span Span Span Click GW API Spans represent a single operationand containmetadatalike the HTTP method, or a databasequery, or an error code JDBC Span: callDB() Span: JDBC call
  • 14. 14 Trace Context Propagation Cartsa42b a42b Extract Inject In process propagation Auto Instrumentation • Zipkin • Sleuth • OpenTelemetry • Commercial • …
  • 15. 15 Trace Context Header Formats • Proprietary • B3 Header (Zipkin) • W3C Trace Context What’sthe header name and what does it contain?
  • 16. 16 W3C Trace Context Service A API GW Service B Trace Service C OpenTelemetry AWS Zipkin OpenTelemetry Goal: All monitoring systems and middlewaresagree on one format for trace context propagation Span Span Span Span
  • 17. 17 W3C Trace Context Format traceparent: 00-0af7651916cd43dd8448eb211c80319c-00f067aa0ba902b7-01 tracestate: rojo=00f067aa0ba902b7,congo=t61rcWkgMzE Version TraceID ParentID Flags
  • 18. 18 Data collection So far we just instrumentedthe code to propagatecontext but no data has been collected Trace Span Span Span Agent Agent Agent Click GW API TraceContext TraceContext MonitoringSystem Storage
  • 19. 19 Data Collection & PresentationSystems Solution Agents Instrumentation Storage Presentation Zipkin / Sleuth + + + + Jaeger - - + + OpenTelemetry + + - - Commercial + + + +
  • 23. 23 Entity Model Based Service Flow
  • 25. 25 Solving our Cart Problem Client API GW Service Service Service Service Service ServiceCart Trace Client Service Currency Cart API GW GET: Currency=EURO
  • 26. 26 What we did to Solve the Problem 1. We used metrics to learn about a problem 2. We used distributedtracing to pass along a unique ID per trace • For that,we used auto instrumentationto extract and inject the trace ID 3. We used a monitoring system and its agentsto collected traces and we could filter transactionsthatproduced an error 4. We looked into the metadata of such a transactionto identify how it differs from succeeding ones
  • 27. 27 You’ve mentionedOpenTelemetry … + = In early 2019 OpenCensusand OpenTracingmerged into OpenTelemetry Metrics, Traces, Logs
  • 29. 29
  • 30. 30
  • 31. 31
  • 32. 32
  • 33. 33 OpenTelemetry – Developer usecases • Cloud nativemicroservices architectures are hard to trace and debug during development • In developmentOpenTelemetry can be used to either • manuallycreate spans to trace certain execution paths • use provided auto-instrumentation tomake a system observable • As backend and UI, Jaeger is the most popular tool. It’s open source and solely displaystraces
  • 34. 34 OpenTelemetry – in Production • Provides just a fraction of what modern tools provide • Traces • Metrics • Logs • Topology • Behavior • Code level visibility • Metadata • Manual instrumentation codeneeds to be kept up-to-date • A backend needs to be maintained • No support model if instrumentation breaksproductioncode • No enterprise features (access control, throttling, scaling, …)
  • 35. 35 Why do Vendors Care then?
  • 36. 36 OpenTelemetry Company Contribution Stats Google Microsoft Dynatrace
  • 37. 37
  • 38. 38 What happens when we add support for a new framework? • Today, our engineers reverse engineer frameworks to add instrumentationsupport to them • Every time an update is released, the instrumentationcode is being tested. • In case of issues, it goes back to the developmentteam who needs to fix it and deploy an update. • The whole process is automated and transparent to the customer ☺ • This is costly and time consuming
  • 39. 39 In-process tracing Click GW API MonitoringSystem Trace Span Span Span HZQ Span: doHZQ()() Span: HZQ call OTEL HZQ Wrapper
  • 40. 40 “We want every platform and library to be pre-instrumented with OpenTelemetry and we’re committed to making this as easy as possible.” Sergey Kanzhelev (Google)
  • 41. 41 What is Observability and how does it differ fromMonitoring? 1. In control theory, observability is a measure of how well internal states of a system can be inferred from knowledgeof its external outputs. Source: Wikipedia 2. In software development, observabilityisachieved by adding code (instrumentation)that emits telemetry data. 3. Monitoringis the act of displayingand analyzing this telemetry data. 4. Monitoringalone can tell you that there is a problem. E.g. ”We see that some users experience a 50% higher response time on check-out” 5. Observablityhelps finding the root cause (the why) by providingdatathat can be correlatedand analysed freely even if this problem is completelynew to you (unknown unknowns) E.g. “The response time of the checkout increases exponentially with the number of items in the basket, because of a misplaced for loop that executes the same database query times the number of items for every item in the basket”
  • 42. 42 Putting it all Together • Metrics can help you to learn that there is a problem • Distributedtracing becomes increasingly importantto understandmulti-tier execution paths and root causes of problems • Developersnow rely on metrics and traces to understandhow their service functionsin their microservice architectures • Pure Open Source solutionsare viable for pre-prod environments • Standardization isthe only way to tackle today’scomplexity and Open Source is the key driver • Vendorsare prepared to tap intodata collected by Open Source standard toolsto add enterprise features on top to support web-scale workloads