DEV Community

Yoshi Yamaguchi for AWS

Posted on

What're inside OpenTelemetry Collector and ADOT?

AWS has some observability services including Amazon CloudWatch, Amazon X-Ray, Amazon OpenSearch Service, and Amazon Managed Service for Prometheus. AWS Distro for OpenTelemetry (ADOT) is an option for sending telemetry data from applications you implement yourself to these services.

In addition to a general description of the OpenTelemetry Collector, this article describes ADOT in detail.

TL;DR

ADOT is a distribution of OpenTelemetry Collector for AWS that comes with the components you need to use the AWS and its partner solutions. If you want to use OpenTelemetry Collector in a more elaborate way, custom build is recommended.

What is OpenTelemetry?

Image description

Though I know that many AWS users are already using OpenTelemetry, just in case let me briefly explain OpenTelemetry, a project under CNCF. OpenTelemetry is a project under CNCF that standardizes the instrumentation, collection and export of telemetry signals from the system.

What is OpenTelemetry Collector?

The OpenTelemetry Collector (hereafter "Collector") is an agent for instrumenting, collecting, and exporting telemetry that is being specified and a reference implementation is being developed by the OpenTelemetry project. The reference implementation runs as a single binary and is written in Go.

It consists of three main components: receiver, processor, and exporter, and the required components are specified and made into a single binary before distribution.

  • Receiver: Responsible for receiving telemetry
  • Processor: Responsible for processing and buffering of telemetry
  • Exporter: Responsible for transmitting telemetry to the backend

OpenTelemetry Collector components

The components included in the binary can be freely linked in the configuration file, but only in the direction of receiver, processor, and exporter 1. Receivers and exporters can only be linked one at a time within a pipeline, while processors can be linked in multiple times.

For example, the following configuration file shows receiving telemetry at OTLP, adjusting the flow rate of telemetry to be sent using the batch and memory_limiter processors, and then sending telemetry again at OTLP.

receivers:
  otlp:

processors:
  batch:
  memory_limiter:
    limit_mib: 1536
    spike_limit_mib: 512
    check_interval: 5s

exporters:
  otlp:

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp]
Enter fullscreen mode Exit fullscreen mode

The Collector's source code is available in GitHub repositories open-telemetry/opentelemetry-collector and open-telemetry/opentelemetry-collector-contrib in which the source code of each component are available. In addition, they distribute several pre-built binaries of the collectors as the official OpenTelemetry project versions, as well as the OCI container that ships with them.

As of February 2025, four different builds are available for the officially distributed collectors, depending on the type of components included 2.

  • otelcol-otlp: Binary that can only receive and send OTLP. The file size is very small but has almost no functionality.
  • otelcol: Binaries that support OSS that should be supported by OpenTelemetry project.
  • otelcol-k8s: Binary with additional components for Kubernetes in addition to otelcol.
  • otelcol-contrib: This is a binary that includes all components managed under the OpenTelemetry project. The file size is very large, although all components, including those for Observability SaaS, can be used.

What is ADOT (AWS Distro for OpenTelemetry)?

If you want to use AWS components, the only way to achieve this with officially distributed collectors is to use otelcol-contrib. However, this would be a waste of a large binary because it includes a large number of unnecessary components. Therefore, AWS provides a Collector distribution called ADOT.

ADOT pre-builds and distributes the components used in the official otelcol distribution 3, plus additional components that are useful for using AWS services and partner SaaS solutions. The binaries distributed here can be invoked by Amazon ECS, AWS Lambda, etc.

For basic use of OpenTelemetry on AWS, using ADOT would be the first option.

Collector custom build

On AWS, it is better to use ADOT Collector, but there will be components in ADOT that you will not use, and if you want to use OpenTelemetry in a more advanced way, you may not want to use these unnecessary components, or conversely, you may want to use components not included in ADOT. For example, the Span Metrics Connector component, which converts distributed trace spans into metrics, is not included in ADOT. For such advanced use cases, the OpenTelemetry Collector builder (ocb) can be used.

ocb is a convenient tool that allows you to build a Collector binary from the configuration in YAML file that lists the components you need in the form of Go package names. For example, to build a collector with only the following components, you need to create the following YAML file otelcol-foo.yaml

dist:
  name: otelcol-foo
  description: Custom OpenTelemery Collector for Project foo AWS account
  output_path: ./otelcol-foo

receivers:
  - gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.119.0

processors:
  - gomod: go.opentelemetry.io/collector/processor/batchprocessor v0.119.0
  - gomod: go.opentelemetry.io/collector/processor/memorylimiterprocessor v0.119.0

exporters:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/awsemfexporter v0.119.0
Enter fullscreen mode Exit fullscreen mode

Using this YAML file, a custom-built Collector is built by executing the ocb command as follows

./ocb --config otelcol-foo.yaml
Enter fullscreen mode Exit fullscreen mode

Conclusion

I hope this article has deepened your understanding of the contents of ADOT, which has evolved along with the development of OpenTelemetry. In fact, the contents of ADOT have changed a little with each version. However, ADOT is only a custom build, so there is nothing to fear if you know what it is. We hope you will find it useful in selecting the best collector for your project.


  1. In the advanced use cases, the components called "Connector" is used, which loop backs the telemetry from the exporter to the receive. Because this artile aims at explaining basic concept of Collector, I omit the explanation of that part. 

  2. Manifest files for each binary build can be found in the open-telemetry/opentelemetry-collector-releases repository. (otelcol-otlp, otelcol, otelcol-k8s, otelcol-contrib

  3. The list of components included in ADOT can be found in the repository README. The actual components can be found in the Go source code

Top comments (0)