Distributed Tracing in Practice. Instrumenting, Analyzing, and Debugging Microservices (ebook)

Distributed Tracing in Practice. Instrumenting, Analyzing, and Debugging Microservices (ebook) - Opis

Most applications today are distributed in some fashion. Monitoring the health and performance of these distributed architectures requires a new approach. Enter distributed tracing, a method of profiling and monitoring applications-especially those that use microservice architectures. There...s just one problem: distributed tracing can be hard. But it doesn...t have to be.With this practical guide, you...ll learn what distributed tracing is and how to use it to understand the performance and operation of your software. Key players at Lightstep walk you through instrumenting your code for tracing, collecting the data that your instrumentation produces, and turning it into useful, operational insights. If you want to start implementing distributed tracing, this book tells you what you need to know.You...ll learn:The pieces of a distributed tracing deployment: Instrumentation, data collection, and delivering valueBest practices for instrumentation (the methods for generating trace data from your service)How to deal with or avoid overhead, costs, and samplingHow to work with spans (the building blocks of request-based distributed traces) and choose span characteristics that lead to valuable tracesWhere distributed tracing is headed in the future Spis treści:
Foreword
Introduction: What Is Distributed Tracing?
Distributed Architectures and You
Deep Systems
The Difficulties of Understanding Distributed Architectures
How Does Distributed Tracing Help?
Distributed Tracing and You
Conventions Used in This Book
Using Code Examples
OReilly (...) więcej Online Learning
How to Contact Us
Acknowledgments
1. The Problem with Distributed Tracing
The Pieces of a Distributed Tracing Deployment
Distributed Tracing, Microservices, Serverless, Oh My!
The Benefits of Tracing
Setting the Table
2. An Ontology of Instrumentation
White Box Versus Black Box
Application Versus System
Agents Versus Libraries
Propagating Context
Interprocess Propagation
Intraprocess Propagation
The Shape of Distributed Tracing
Tracing-Friendly Microservices and Serverless
Tracing in a Monolith
Tracing in Web and Mobile Clients
3. Open Source Instrumentation: Interfaces, Libraries, and Frameworks
The Importance of Abstract Instrumentation
OpenTelemetry
OpenTracing and OpenCensus
OpenTracing
OpenCensus
Other Notable Formats and Projects
X-Ray
Zipkin
Interoperability and Migration Strategies
Why Use Open Source Instrumentation?
Interoperability
Portability
Ecosystem and Implicit Visibility
4. Best Practices for Instrumentation
Tracing by Example
Installing the Sample Application
Adding Basic Distributed Tracing
Custom Instrumentation
Where to StartNodes and Edges
Framework Instrumentation
Service Mesh Instrumentation
Creating Your Service Graph
Whats in a Span?
Effective Naming
Effective Tagging
Effective Logging
Understanding Performance Considerations
Trace-Driven Development
Developing with Traces
Testing with Traces
Creating an Instrumentation Plan
Making the Case for Instrumentation
Instrumentation Quality Checklist
Knowing When to Stop Instrumenting
Smart and Sustainable Instrumentation Growth
5. Deploying Tracing
Organizational Adoption
Start Close to Your Users
Start Centrally: Load Balancers and Gateways
Leverage Infrastructure: RPC Frameworks and Service Meshes
Make Adoption Repeatable
Tracer Architecture
In-Process Libraries
Sidecars and Agents
Collectors
Centralized Storage and Analysis
Incremental Deployment
Data Provenance, Security, and Federation
Frontend Service Telemetry
Server-Side Telemetry for Managed Services
6. Overhead, Costs, and Sampling
Application Overhead
Latency
Throughput
Infrastructure Costs
Network
Storage
Sampling
Minimum Requirements
Strategies
Up-front sampling
Response-based sampling
Centralized sampling decisions
Selecting Traces
Off-the-Shelf ETL Solutions
7. A New Observability Scorecard
The Three Pillars Defined
Metrics
Types of metrics: Counters and gauges
Metrics tools
Logging
Logging conventions
Distributed Tracing
Fatal Flaws of the Three Pillars
Design Goals
Accounting for every transaction
Immunity from cardinality issues
Cost growing proportionally with business value
Assessing the Three Pillars
Fatal flaws in metrics
Fatal flaw in logging
Fatal flaws in distributed tracing
Three Pipes (Not Pillars)
Observability Goals and Activities
Two Goals in Observability
Two Fundamental Activities in Observability
A New Scorecard
Statistical fidelity
Cardinality limits
Volume limits
Time limits
Provide context
Prioritize by impact
Automate correlation
The Path Ahead
8. Improving Baseline Performance
Measuring Performance
Percentiles
Histograms
Defining the Critical Path
Approaches to Improving Performance
Individual Traces
Biased Sampling and Trace Comparison
Trace Search
Multimodal Analysis
Aggregate Analysis
Correlation Analysis
9. Restoring Baseline Performance
Defining the Problem
Human Factors
(Avoiding) Finger-Pointing
Suppressing the Messenger
Incident Hand-off
Good Postmortems
Approaches to Restoring Performance
Integration with Alerting Workflows
Individual Traces
Biased Sampling
Real-Time Response
Knowing Whats Normal
Aggregate and Correlation Root Cause Analysis
10. Are We There Yet? The Past and Present
Distributed Tracing: A History of Pragmatism
Request-Based Systems
Response Time Matters
Request-Oriented Information
Notable Work
Pinpoint
Magpie
X-Trace
Dapper
Where to Next?
11. Beyond Individual Requests
The Value of Traces in Aggregate
Example 1: Is Network Congestion Affecting My Application?
Example 2: What Services Are Required to Serve an API Endpoint?
Organizing the Data
A Strawperson Solution
What About the Trade-offs?
Sampling for Aggregate Analysis
The Processing Pipeline
Incorporating Heterogeneous Data
Custom Functions
Joining with Other Data Sources
Recap and Case Study
The Value of Traces in Aggregate
Organizing the Data
Sampling for Aggregate Analysis
The Processing Pipeline
Incorporating Heterogeneous Data
12. Beyond Spans
Why Spans Have Prevailed
Visibility
Pragmatism
Portability
Compatibility
Flexibility
Why Spans Arent Enough
Graphs, Not Trees
Inter-Request Dependencies
Decoupled Dependencies
Distributed Dataflow
Machine Learning
Low-Level Performance Metrics
New Abstractions
Seeing Causality
13. Beyond Distributed Tracing
Limitations of Distributed Tracing
Challenge 1: Anticipating Problems
Challenge 2: Completeness Versus Costs
Challenge 3: Open-Ended Use Cases
Other Tools Like Distributed Tracing
Census
A Motivating Example
A Distributed Tracing Solution?
Tag Propagation and Local Metric Aggregation
Comparison to Distributed Tracing
Pivot Tracing
Dynamic Instrumentation
Recurring Problems
How Does It Work?
Dynamic Context
Comparison to Distributed Tracing
Pythia
Performance Regressions
Design
Overheads
Comparison to Distributed Tracing
14. The Future of Context Propagation
Cross-Cutting Tools
Use Cases
Distributed Tracing
Cross-Component Metrics
Cross-Component Resource Management
Managing Data Quality Trade-offs
Failure Testing of Microservices
Enforcing Cross-System Consistency
Request Duplication
Record Lineage in Stream Processing Systems
Auditing Security Policies
Testing in Production
Common Themes
Should You Care?
The Tracing Plane
Is Baggage Enough?
Beyond Key-Value Pairs
Compiling BDL
BaggageContext
Merging
Overheads
A. The State of Distributed Tracing Circa 2020
Open Source Tracers and Trace Analysis
Commercial Tracers and Trace Analyzers
Language-Specific Tracing Features
Java and C#
Go, Rust, and C++
Python, JavaScript, and Other Dynamic Languages
B. Context Propagation in OpenTelemetry
Why a Separate Context Model?
The OpenTelemetry Context Model
W3C CorrelationContext and the Correlations API
Distributed and Local Context
Examples and Potential Applications
Bibliography
Index mniej