feat(outputs.opentelemetry): Support http protocol #17997

minuk-dev · 2025-11-13T19:06:53Z

Summary

OTLP (opentelemetry) provides gRPC and HTTP but opentelemetry output plugin only supports gRPC.
- ref. https://opentelemetry.io/docs/specs/otlp/#otlphttp
So, add some logic to support http protocol with protobuf & json from outputs.opentelemetry

Question

I checked other plugins' directories, but is it a coding convention to keep everything within a single file? There's too much logic, so I want to split the file.

Local Test

You can test this feature using below config.

otel.yaml

# otel.yaml
receivers:
  otlp:
    protocols:
      http:
        # OTLP/HTTP standard port is 4318
        endpoint: "0.0.0.0:4318"

exporters:
  debug:

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: []
      exporters: [debug]

    metrics:
      receivers: [otlp]
      processors: []
      exporters: [debug]

    logs:
      receivers: [otlp]
      processors: []
      exporters: [debug]

docker run --rm -it -v "$(pwd)/otel.yaml":/etc/otel/config.yaml -p 4318:4318 --name otel-collector otel/opentelemetry-collector:latest --config /etc/otel/config.yaml

# telegraf.conf
# skip
[[outputs.opentelemetry]]
protocol = "http"
service_address = "http://localhost:4318/v1/metrics"

Checklist

No AI generated code was used in this PR
AI generated code used in this PR follows the InfluxData Policy on AI-Generated Code Contributions

Related issues

telegraf-tiger · 2025-11-13T19:07:00Z

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

srebhan

Thanks @minuk-dev for your contribution! I do have some initial comments below. Let's first get the code structure right and then work on the details...

plugins/outputs/opentelemetry/opentelemetry.go

srebhan · 2025-11-17T12:51:06Z

plugins/outputs/opentelemetry/opentelemetry.go

+	// EncodingType is the encoding type to use for sending data to the OpenTelemetry Collector.
+	// It is used only when Protocol is set to "http".
+	// Supported encoding types are "application/x-protobuf" & "application/json". Defaults to "application/x-protobuf".
+	EncodingType string `toml:"encoding_type"`


Does it make sense to support protobuf via HTTP? If so, please name the option format and use json or protobuf as values instead of the full HTTP encoding type to not mislead users to think they can specify an arbitrary encoding. We also usually use encoding for compression settings...

plugins/outputs/opentelemetry/opentelemetry.go

plugins/outputs/opentelemetry/opentelemetry_test.go

telegraf-tiger · 2025-11-26T16:16:50Z

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

⚠️ This pull request increases the Telegraf binary size by 1.31 % for linux amd64 (new size: 300.0 MB, nightly size 296.1 MB)

📦 Click here to get additional PR build artifacts

Artifact URLs

. DEB	. RPM	. TAR . GZ	. ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz

srebhan

Thanks @minuk-dev for the update! Here is round two of my comments... ;-)

srebhan · 2025-12-04T18:46:30Z

plugins/outputs/opentelemetry/client.go

+type gRPCClient struct {
+	grpcClientConn       *grpc.ClientConn
+	metricsServiceClient pmetricotlp.GRPCClient
+	callOptions          []grpc.CallOption
+}
+
+type httpClient struct {
+	httpClient   *http.Client
+	url          string
+	encodingType string
+	compress     string
+}


What I meant previously is to have a client_grpc.go and a client_http.go with the respective struct etc. Sorry for being not clear!

srebhan · 2025-12-04T18:47:37Z

plugins/outputs/opentelemetry/client.go

+}
+
+func (*httpClient) Close() error {
+	// No persistent connections to close for HTTP client


I think you need to CloseIdleConnections here.

srebhan · 2025-12-04T18:48:04Z

plugins/outputs/opentelemetry/client.go

+const (
+	maxHTTPResponseReadBytes = 64 * 1024 // 64 KB
+)


Don't declare this const but just use the value!

plugins/outputs/opentelemetry/opentelemetry_test.go

srebhan · 2025-12-04T18:52:13Z

plugins/outputs/opentelemetry/opentelemetry_test.go

+		if err != nil {
+			t.Error(err)
+		}


You need to return as t.Error will not stop the processing here! Same below...

Suggested change

if err != nil {

t.Error(err)

}

if err != nil {

t.Error(err)

w.WriteHeader(http.StatusInternalServerError)

return

}

srebhan · 2025-12-04T18:52:48Z

plugins/outputs/opentelemetry/opentelemetry_test.go

+		err = req.UnmarshalProto(body)
+		if err != nil {
+			t.Error(err)
+		}


Please limit the scope of the variables! Same below.

Suggested change

err = req.UnmarshalProto(body)

if err != nil {

t.Error(err)

}

if err := req.UnmarshalProto(body); err != nil {

t.Error(err)

w.WriteHeader(http.StatusInternalServerError)

return

}

srebhan · 2025-12-04T18:54:38Z

plugins/outputs/opentelemetry/opentelemetry_test.go

+	err = plugin.Connect()
+	require.NoError(t, err)


Please use

Suggested change

err = plugin.Connect()

require.NoError(t, err)

require.NoError(t, plugin.Connect())

as it is shorter and avoids an explicit err... Same for all other instances below.

srebhan · 2025-12-04T18:55:15Z

plugins/outputs/opentelemetry/opentelemetry_test.go

+		map[string]interface{}{
+			"gauge": 87.332,
+		},
+		time.Unix(0, 1622848686000000000))


Just a matter of taste but I like

Suggested change

time.Unix(0, 1622848686000000000))

time.Unix(0, 1622848686000000000),

)

to better see the closing bracket.

srebhan · 2025-12-04T18:57:31Z

plugins/outputs/opentelemetry/sample.conf

 # Send OpenTelemetry metrics over gRPC
 [[outputs.opentelemetry]]
+  ## Override the default (grpc) protocol
+  ## grpc, http
+  # protocol = "grpc"
  ## Override the default (localhost:4317) OpenTelemetry gRPC service
-  ## address:port
+  ## When the protocol is grpc, address:port
+  ## When the protocol is http, http(s)://address:port/path
  # service_address = "localhost:4317"
+  ## Override the default (application/x-protobuf) encodingType when Protocol is http
+  ## application/x-protobuf, application/json
+  # encoding_type = "application/x-protobuf"



Please update!

srebhan · 2025-12-04T19:02:09Z

plugins/outputs/opentelemetry/opentelemetry.go

 	}
+	if o.EncodingType == "" {
+		o.EncodingType = "application/x-protobuf"
+	}


Can we please check the encoding type!? What if some uses application/pdf? I guess we are only support JSON and protobuf for now, don't we?

Use a switch statement like

switch o.EncodingType { case "": o.EncodingType = "application/x-protobuf" case "application/json", "application/x-protobuf": // Do nothing, those are valid default: return fmt.Errorf("invalid encoding %q", o.EncodingType) }

to be honest I would even prefer

switch o.EncodingType { case "", "protobuf": o.EncodingType = "application/x-protobuf" case "json": o.EncodingType = "application/json" default: return fmt.Errorf("invalid encoding %q", o.EncodingType) }

minuk-dev added 8 commits November 14, 2025 02:57

refactor(output;opentelemetry): split grpcClient

6d49e80

feat(outputs.opentelemetry): add protocol to support http

322fd17

feat(outputs.opentelemetry): implement otlphttp

eb02025

docs(outputs.opentelemetry): update README.md

75d0fc7

feat(output;opentelemetry): set default value

f968650

fix(outputs.opentelemetry): fix nil error

20b4b6a

test(outputs.opentelemetry): add test

fe9c0cf

feat(output;opentelemetry): set default value

6fad8c9

telegraf-tiger bot added feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin plugin/output 1. Request for new output plugins 2. Issues/PRs that are related to out plugins labels Nov 13, 2025

minuk-dev added 3 commits November 14, 2025 04:10

docs: add comment about readResponseBody's original code

9cd3d42

docs: add sample.conf

7ff4a40

chore: fix lint

2e409df

minuk-dev force-pushed the iss17713 branch from c2363d0 to 2e409df Compare November 13, 2025 19:25

refactor: optimize switch as interface

3ad719e

minuk-dev marked this pull request as ready for review November 13, 2025 19:42

fix: add gzip content-encoding header

c01b496

srebhan reviewed Nov 17, 2025

View reviewed changes

srebhan self-assigned this Nov 17, 2025

srebhan changed the title ~~feat(outputs.opentelemetry): support http protocol with protobuf & json encoding~~ feat(outputs.opentelemetry): Support http protocol Nov 17, 2025

minuk-dev added 5 commits November 17, 2025 22:49

remove dummy interface assertion

d966492

remove comment

ffaf0f6

refactor close func

e1a5f8d

split client file

3b07809

refactor client init as connect

8fb5f23

minuk-dev marked this pull request as draft November 17, 2025 14:18

minuk-dev added 3 commits November 18, 2025 21:00

remove protocol

133444c

refactor: cleanup client config

166a8ff

remove declaring constant

5c11c30

minuk-dev marked this pull request as ready for review November 26, 2025 15:56

srebhan reviewed Dec 4, 2025

View reviewed changes

	err = plugin.Connect()
	require.NoError(t, err)
	require.NoError(t, plugin.Connect())

	time.Unix(0, 1622848686000000000))
	time.Unix(0, 1622848686000000000),
	)

feat(outputs.opentelemetry): Support http protocol #17997

Are you sure you want to change the base?

feat(outputs.opentelemetry): Support http protocol #17997

Conversation

minuk-dev commented Nov 13, 2025 • edited by srebhan Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Question

Local Test

Checklist

Related issues

Uh oh!

telegraf-tiger bot commented Nov 13, 2025

Uh oh!

srebhan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

telegraf-tiger bot commented Nov 26, 2025

Artifact URLs

Uh oh!

srebhan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

minuk-dev commented Nov 13, 2025 •

edited by srebhan

Loading