OpenTelemetry in 2023 – by Kevin Lin
OpenTelemetry (OTEL) began in 2019 when the 2 dominant open-source initiatives round tracing, OpenTracing and OpenCencus, merged to type OpenTelemetry. The scope of the mixed mission expanded past tracing to cowl all of observability. OTEL’s mission is to assist organizations ship high-quality, ubiquitous, and transportable telemetry.
4 years later, OTEL is making good on its promise. It has so far offered a steady customary for the three pillars of observability (metrics, logs, and traces), a collector that may obtain, course of, and export telemetry in any setting, and SDKs to instrument code in all main languages. It has additionally continued to increase its scope and launched extra requirements round semantic conventions and agent administration.
Immediately, OTEL is the second most energetic mission within the CNCF, behind solely Kubernetes in reputation. Its contributors are unfold throughout all main observability distributors and its protocol has near-universal adoption amongst observability suppliers.
On this publish, we’ll go over the place OTEL is right now, present a abstract of the assorted initiatives that fall underneath its area, and discuss the place it is going subsequent.
OTEL consists primarily of two varieties of initiatives: specs and implementations.
Specs are the inspiration of OTEL. They describe how telemetry must be captured, collected, processed, and exported. Specs are usually vendor-focused as they outline the frequent customary that distributors must implement to be suitable with OTEL.
Implementations are the particular shopper libraries and tooling which are used to deal with telemetry information. Implementations are usually end-user-focused as they include the components that end-users use to instrument their code.
A mission’s stability is usually marked on a signal-by-signal foundation. Every sign is a particular sort of telemetry information – in OTEL, this implies metrics, logs, and traces.
OTEL right now consists of the next sub-projects:
-
OpenTelemetry Specification (specification)
-
OpenTelemetry SDKs (implementation)
-
OpenTelemetry Protocol (specification)
-
OpenTelemetry Collector (implementation)
-
Open Agent Administration Protocol (specification)
-
OpenTelemetry Semantic Conventions (specification)
The Open Telemetry specification is the inspiration of OTEL – it supplies the APIs, SDKs, and information fashions that every one different OTEL requirements are derived from.
A quick timeline of its growth:
When it comes to maturity, the OTEL spec is now steady for all indicators. As a result of the logging sign was not deemed steady till earlier this 12 months, many OTEL SDKs nonetheless don’t assist logging presently.
The OTEL SDKs present client-side instrumentation based mostly on the OTEL spec. For every language-specific SDK, there’s a separate maturity degree for every sign.
Relying on the programming language, some SDKs additionally assist automated instrumentation. That is when the SDK robotically injects indicators, primarily traces, into your utility while not having to manually instrument code.
Auto instrumentation does not work for compiled languages like go
and rust
. That mentioned, you’ll be able to nonetheless get automated hint injection by going outdoors of the SDK and utilizing eBPF and/or service mesh based mostly tooling.
OTLP describes a typical wire protocol for delivering observability information. There are two blessed transports in OTLP: http with protocol buffers and gRPC.
This spec is taken into account steady and may be carried out on any service that receives, processes, or exports OTEL information. The spec is carried out by the OpenTelemetry collector and likewise by brokers of observability distributors like Grafana and Datadog.
The OTEL collector is a vendor-agnostic agent that collects, transforms, and sends observability information.
The collector consists of the next parts:
-
receivers: push/pull information from completely different sources
-
processors: rework/filter/enrich/derive information in flight
-
exporters: ship information to downstream locations
-
connectors: connectors are each a receiver and exporter and allow a number of pipelines to be stitched collectively
-
pipelines: a series of receiver, zero or extra processors, and exporters
-
extensions: parts that present extra capabilities outdoors of dealing with telemetry information (eg. fundamental auth, well being checks, and so forth)
Collectively, the assorted parts act as an observability pipeline, permitting you to gather telemetry information from any supply, course of it in flight, and ship it to any vacation spot.
The OTEL collector consists of two initiatives, otel-collector and otel-collector-contrib. otel-collector
comprises solely the core parts of the collector, mainly logic strictly associated to processing OLTP information.
otel-collector-contrib
is a kitchen sink of each integration you’ll be able to think about, with exporters and receivers for many observability suppliers current. On the time of this writing, this consists of 91 receivers, 48 exporters, and 24 processors.
Finish customers are suggested to make use of the OpenTelemetry Collector Builder to create a customized construct of otel-collector-contrib
with solely parts that they want.
Along with otel-collector-contrib
, distributors like AWS and Splunk present their very own customized distributions of OTEL.
OpAMP is a community protocol for the distant administration of brokers. This can be a latest addition to OTEL, launched in 2022, and supplies a vendor-agnostic customary for controlling fleets of brokers. These brokers may very well be situations of the otel-collector
or vendor-specific brokers that implement OpAMP.
With OpAMP, you’ll be able to allow options like dynamic configuration deployments, agent updates, and credential administration.
There’s at the moment a work-in-progress go implementation of the OpAMP spec.
NOTE: In the event you’re involved in managing OTEL brokers through OpAMP – that is what my startup is engaged on.
OpenTelemetry Semantic Conventions (1.21.0)
The OTEL Semantic Conventions outline a typical set of attributes for observability information. They cowl a variety of areas together with cloud assets, databases, exceptions, and programs.
The semantic conventions are utilized by the OTEL SDKs and robotically utilized for SDKs that assist auto instrumentation. Widespread semantics allows the flexibility to correlate between completely different indicators and is one thing I am particularly excited by, the rationale for which might (and doubtless will) be the topic of a complete weblog publish.
OTEL is an enormous mission. Whereas we lined the flagship initiatives, there are extra components of OTEL that deserve a notable point out:
-
OpenTelemetry Transformation Language (OTTL): a typical transformation language for telemetry that’s each performant and versatile – at the moment designed and carried out in
otel-collector-contrib
-
OTEL Demo: A micro-service-based purchasing web site that demonstrates a majority of options and language SDKs of OTEL
OTEL began as a merger between competing tracing specs and advanced to grow to be an trade customary for observability. The final 4 years laid the groundwork for constructing a typical basis throughout distributors and instruments. The subsequent 4 will reveal the fruits of that labor.