OpenTelemetry 落地困境复盘：维护者稀缺与稳定性博弈

For years now one of the most reliable complaints I hear when I try to drag a team off their vendor specific SDK and onto OpenTelemetry is some variation of: "why does it seem like this isn't done yet?"

Vendor SDKs for observability are, to put it charitably, idiot-proof. You install the thing, dashboards just load data, someone else worries about how all those pieces fit together, and you get on with your life. OpenTelemetry, by contrast, greets you at the door with a lot of "experimental" stamps and roughly six different ways to accomplish any given task.

In OpenTelemetry's defense this was never what they were going for as a project. I've always respect that they stuck to their guns by attempting to build a truly vendor agnostic system that really doesn't care what you do with the data. I have never gotten a sense of a vendor being strongly preferred with OTel, which is quite the feat considering how lucrative and contentious the observability ecosystem was. Also considering that the maintainers of this project are largely employed by exclusively those companies.

As the years wore on, I started to get nervous. Conversations in the semantic-conventions repo drag on and on and on. Different languages had dramatically different stories. Golang and Dotnet were first class citizens, but other languages lagged years behind the others.

I started asking a lot of probing questions before recommending OpenTelemetry to smaller teams who didn't have the time, budget, or emotional bandwidth for it. Auto-instrumentation was genuinely magical, but the cliff between "auto-instrument works" and "now I have to manually instrument something" was steep enough that you owed people a warning before you pushed them off it.

This narrative has been going on for awhile in the observability space, a vague sense of "something is wrong in Otel-land". But let's try to generate some actual data here. Is there an actual problem, or is this something where the perception by the community of slow progress is imaginary? Is the problem not enough maintainers, too big of a scope, or something in-between?

My guess when I started was "oh this is your classic open-source bit off more than they can chew". Not enough maintainers, not enough budget. Now there is some of that, but there's also something else going on.

The actual problem happening inside of OpenTelemetry is a three way crash. You have a binary stability gate which, when combined with a very small bench of actual maintainers means there is understandable worry about marking a feature not experimental then add on just a massive scope of languages and frameworks they are attempting to cover. This creates a perfect storm where there is an incentive to argue about potential problems a feature might create since once it is locked in and shipped as stable you can never change them.

How does OpenTelemetry Work

So OpenTelemetry currently is attempting to support a dizzying number of languages and frameworks.

OpenTelemetry is a giant project. It spans dozens of languages, hundreds of libraries, and countless backends. To keep things sane, the project splits work into two buckets:

• Core → Maintained directly by the OTel project. Small, stable, vendor-neutral, and tightly reviewed. This is the "spec-defining" surface.
• Contrib → Community- and vendor-contributed. Broader, faster-moving, and covers the long tail of integrations.

There exists the otel-collector, the thing that runs along the thing so that you can ship logs metrics and traces. That copies the same rough pattern. But for the languages when we're talking about core vs contrib this is what we're talking about.

Stuff that breaks goes in contrib, stuff that doesn't break goes into core.

Now the reason this causes a conflict. contrib is massive overkill for most projects. You don't want 300 exporters to add the one you typically need. On the language side, this isn't that big of a problem. pip install opentelemetry-instrumentation-flask gives you the stuff you need for flask. However on the collector side you end up having to do the OpenTelemetry Collector Builder to make your own collector (or just kinda ride the wave and hope it works out). While cool that this exists, it's a lot of scope to ask a team to take on.

Process of adding a new feature

So I believe I have captured the workflow of adding a new feature to OTel. You can check my homework here:

• OpenTelemetry Enhancement Proposal (OTEP) (https://github.com/open-telemetry/opentelemetry-specification/tree/main/oteps/)
• Once the OTEP is accepted, the text goes into the Specification directory in the same repo.
• After that it seems to go to Semantic conventions. This seems to be where we get down to the specific details and where most of the long discussions seem to live. At this point we're talking about more or less a permanent commitment to this design and where the lock-in process becomes very hard to change.
• Each of the SDKs implements the API surface that is defined in the specification. Now some of the SDKs have done 2.0 breaking changes, so it does seem like the earlier "please no 2.0 at all costs" sentiment has been abandoned (which I think is smart and good).
• Contrib / instrumentation. This is slightly more mushy. Looks like they should track latest API/SDK but each contrib package may version independently so its more flexible as a design.
• Collector + OTLP. The data has to actually go somewhere. OTLP (wire protocol) has its own stability lifecycle and specification (here). Collector components have their own stability in their READMEs and as far as I can tell that's kinda all over the place.

Things I'm not really clear on

• It's unclear how long the OTEP -> Specification process takes. I've looked through the Git history but there doesn't seem to be any predictable number or cycle.
• I don't fully understand what is the relationship between all these stability commitments. Does Collector + OTLP group work in lockstep? Can a language "fall out of scope" if you lag too far behind?

Attempting to test it

So because OpenTelemetry is a CNCF project, I figured it made the most sense to compare them to other CNCF projects. My basis for comparison is Envoy and Prometheus. I have used a hacky Python script I've used before for measuring the "health" of open-source projects, which is probably not the best. However I'll include a link to the raw data without the charts so folks can review it and (more than likely) find a problem in what I generated.

So we look at 24 months of activity for Envoy and what we see is a pretty healthy project. There's good distribution of authors, mergers, issue closers. phlax is obviously pretty important to the project but in general there's a good bench of people to step in if needed. I've attempted to filter out all the known bot traffic.

Let's compare that to one of the OpenTelemetry languages. The ones I have the most professional experience with are Golang and Python, but I hear from a lot of folks in the community that the Ruby and PHP ones struggle a lot. This is the PHP one for the same period.

So we see pretty clearly that there's way too much concentrated on 2 people. This is not a healthy open-source project and they clearly don't have enough people to cover the kind of scope OTel needs to cover. Same story with Ruby.

In comparison the "strongest" OpenTelemetry SDKs in my opinion, Golang and Dotnet (although Python is also no slouch) look more healthy.

Golang

So the first issue is maybe the least surprising. There's too much concentration among too few maintainers. Your authors shouldn't also be your mergers and your issue closers. Ideally these tasks should be distributed out more evenly.