The otlp destination delivers to an OpenTelemetry collector over OTLP/HTTP: protobuf requests to /v1/logs, conventionally on port 4318. ClickStack's views — severity charts, services, the attribute facets you search on — are built on the columns OTLP fills, so events arrive already broken out rather than as one line of text.
It carries events from every kind of source: Windows Event Log, log files, file integrity, syslog and NetFlow.
ClickHouse, ClickStack and HyperDX are trademarks of ClickHouse, Inc. DPLens
is not affiliated with or endorsed by ClickHouse, Inc.
Which ClickStack
The same destination serves all three ways of running ClickStack.
| Deployment | What you point DPLens at | Status |
|---|---|---|
| Self-hosted — all-in-one, Docker Compose or Helm | The collector of the ClickStack you run. It listens on plain HTTP, port 4318, by default, and expects the ingestion key from HyperDX → Team Settings → API Keys. | Tested |
| Managed | An OpenTelemetry collector you run in front of Managed ClickStack. TLS and a key are whatever you configured on that collector. | Compatible; tested with self-hosted ClickStack |
| ClickStack Cloud | The HTTPS endpoint of your service, <service>.otel.<region>.aws.clickhouse.cloud:4318, with the ingestion key. | Compatible; tested with self-hosted ClickStack |
The destination speaks standard OTLP/HTTP, so another OpenTelemetry collector that accepts OTLP/HTTP logs can receive from it too.
Setting it up in the console
- On Destinations, choose + Add destination, then the ClickStack card — Confirmed hand-off · OTLP/HTTP · port 4318.
- Pick the ClickStack deployment: Self-hosted, Managed or ClickStack Cloud. This only pre-fills the form — TLS, an example address, and the key handle, which Managed leaves empty because a key is optional there — and never changes a value you have typed.
- Enter the Address as
host:port. A bare host name gets port 4318. You can also paste the endpoint URL, such ashttps://clickstack.example.internal:4318/v1/logs, and it is split into the address, TLS and the path for you. - Under Ingestion key, paste the key into Key. It is sealed into the agent's secret store when you save; afterwards the form shows only Key: configured ✓. Leave the Key handle at its default unless you keep several keys. To send no key at all, clear the handle.
- Turn on Encrypt with TLS (HTTPS) if your endpoint is HTTPS — ClickStack Cloud always is.
- Choose the Shape — see Native or collector-compatible.
- Optionally add Service names and resource attributes — see Service names and resource attributes.
- Save, map a pipeline to the destination, and apply.
If you type port 4317, the form warns you: that is the collector's OTLP/gRPC port, and this destination sends OTLP/HTTP.
Setting it up in configuration
destinations:
- type: otlp
name: clickstack
params:
address: "clickstack.example.internal:4318"
auth_handle: "secret://clickstack/ingestion-key"
shape: "native"
cache_max_bytes: "2147483648"
otlp:
service_names:
security-log: windows-security
app-log: payments-api
resource_attributes:
deployment.environment: production
Seal the key before you apply a configuration that names the handle — validation refuses a handle with nothing stored under it:
"C:\Program Files\DPLens\dplens.exe" --set-secret secret://clickstack/ingestion-key
The value is read from standard input, never from the command line. Every setting is listed in the configuration reference, and Configuration examples has a complete file.
The ingestion key
Three settings decide how the key is sent. Only one of them has anything to do with the key itself.
| Setting | What it is | Default |
|---|---|---|
auth_handle | The secret-store handle the key is sealed under. This is the only place the key goes. | Not set — no key is sent |
auth_header | The name of the HTTP header that carries the key. A name, never a value. | authorization |
auth_scheme | raw: the header value is the key itself. bearer: the header value is Bearer <key>. | raw |
Never paste the key, or Bearer and the key, into auth_header or any other configuration field. A configuration file is meant to be safe to copy, check in and push to a fleet; a key typed into it is none of those. ClickStack expects the key in the authorization header, so the defaults are right for it: seal the key under auth_handle and leave the other two alone. Self-hosted ClickStack accepts both the raw and the bearer form.
The key is never shown in the configuration, the logs, the console or the live stream, and if the collector echoes it back in an error message, it is removed before that message is displayed or logged.
To change the key on a running agent, use Replace beside the key on the destination in the console. The new key is sealed under the same handle, and the change is recorded in the audit log.
Plain HTTP or HTTPS
A self-hosted ClickStack collector listens on plain HTTP by default, so tls is off by default here — turning it on would fail against an unprepared collector. Over plain HTTP, though, your events and the ingestion key cross the network unencrypted. The console says so: a note in the form, and a key sent unencrypted tag on the destination card.
https:// endpoints are supported:
tls: "true"
ca_file: "C:\\ProgramData\\DPLens\\ca\\clickstack-ca.pem"
With TLS on, certificate validation is on, against the Windows trust store unless you give ca_file. The name verified is the host in address. Mutual TLS uses client_cert_handle and client_key_handle, as secret handles.
Use TLS and the ingestion key wherever the network path is not one you fully trust. Together they mean nothing in the path can read your events or the key, or pretend to be your collector.
Native or collector-compatible
shape decides how each event is laid out. It is set per destination.
| Shape | What you get | Choose it when |
|---|---|---|
native (default) | Every event field as a searchable attribute, a severity, the source it came from, and the agent that sent it. | You are starting fresh. Recommended. |
collector | The layout the standard OpenTelemetry Collector gives Windows event logs, syslog and log files. | You already have dashboards and saved searches built on that layout. |
The collector-compatible shape matches the collector's layout — the same keys, nesting and value types — not every value. For example, DPLens always sets the event time on a log-file line, and syslog severity text is the upper-case OpenTelemetry name. File-integrity and NetFlow events have no collector layout and are always sent native.
For the Windows Event Log layout to carry level, task, opcode and keyword names as the collector does, set Display names to Add on the source — see Display names.
What a native event carries
On the resource — the part HyperDX groups by:
| Attribute | Value |
|---|---|
host.name | Where the event came from — see The host name |
service.name | Blank, unless you name a service for the source |
dplens.source.name | The source's name in your configuration |
dplens.source.kind | The kind of source: Windows Event Log, file, and so on |
dplens.agent.id / dplens.agent.version | The agent that sent it |
dplens.run_id | A random number chosen each time the source starts |
On each record: dplens.source_seq, a sequence number within the source, and every field of the event. The record's timestamp is when the event happened; its observed timestamp is when DPLens read it.
The collector-compatible shape carries host.name, your service names and your resource attributes, and none of the dplens.* attributes.
The message
| Event | What ClickStack shows as the message |
|---|---|
| Anything with a message — log lines, syslog, Windows events with message rendering on | The message |
| A Windows event with message rendering off (the default) | A one-line summary: event ID, channel and provider, such as 4624 Security Microsoft-Windows-Security-Auditing |
| NetFlow and file-integrity events | The same JSON line the NDJSON destination would send. The fields are also sent as attributes, so these events cost about twice the bytes. |
| A log line or syslog event with no message | No message; the fields are still sent as attributes, and the event is counted sent without a message |
Severity
| Source | Severity |
|---|---|
| Windows Event Log | From the event's level: Critical → FATAL, Error → ERROR, Warning → WARN, Information → INFO, Verbose → DEBUG. In the native shape an Audit Failure is raised to at least WARN. |
| Syslog | From the message's severity, through the OpenTelemetry table (emergency → FATAL … debug → DEBUG) — once a parse-syslog stage has run in the pipeline. |
| Log files and everything else | None |
An event with no severity is sent without one and counted without a severity. ClickStack may then infer a level from the message text, and it shows severity text in lower case.
The host name
For a source on this machine, host.name is the machine's own name. For a syslog or NetFlow receiver it is the device that sent the event: for syslog, the host name in the message header when the pipeline has a parse-syslog stage, otherwise the sending address. Validation warns when a syslog source reaches this destination without one. These are the same rules as the source_id of an NDJSON record.
Events still in the disk cache from before a change to your list of sources are sent without their source details — no service name and no dplens.source attributes — and counted sent without source details after a change, because DPLens cannot be sure which source they now belong to.
Service names and resource attributes
The destination's otlp: block sets two things, in the console under Service names and resource attributes:
otlp:
service_names:
security-log: windows-security
resource_attributes:
deployment.environment: production
team: soc
service_namessetsservice.namefor the events of a source, named as in your configuration. This is the service HyperDX groups them under. Sources you do not list are sent with no service name. A name that matches no source draws a validation warning.resource_attributesadds key and value pairs to every event this destination sends. They are applied last, so a key here replaces the value DPLens would have computed — includinghost.name— and validation warns you when it does.
Both apply in either shape. The whole block is limited to 64 KiB of UTF-8 text. Everything in it travels with every request, so validation also warns when it takes half of max_request_bytes, and refuses it if no event could fit.
Confirmed hand-off
Events are checkpointed only after the collector accepts them, and a disk cache rides out outages. Records the collector rejects are dropped and counted on the destination card.
What the collector answers decides what happens:
| The collector answers | What DPLens does |
|---|---|
| Accepted (2xx) | Delivered. If the collector reports that it rejected some records of the request, those records are dropped and counted. |
| Busy or unavailable (408, 429, 502, 503, 504) | Retries with backoff, honouring the collector's Retry-After up to five minutes. |
| Your configuration is wrong (401, 403, 404, 415, a redirect, 500, 501 and similar) | Nothing is dropped. Events stay in the disk cache and the card shows a fault until you fix it. Redirects are not followed — point the destination at the collector itself. |
| This content is bad (400) or too large (413) | Splits the request to find the record the collector refuses, and drops only that record, and only once the collector has accepted the rest. If it cannot show that, nothing is dropped: events stay cached and the card shows a fault. |
| Any other refusal | Drops the request's records, as the OpenTelemetry specification directs, counts them and shows a fault. |
Two things to know:
- Accepted means accepted by the collector, not yet stored in ClickHouse. What happens between the two is the collector's own queue.
- The collector decides what is rejected. A collector — or anything in the network path pretending to be one — can therefore cause records to be dropped. Each one is counted on the destination card, and drops while the cache is draining are also written to the audit log. Use TLS and the ingestion key so that nothing in the path can impersonate your collector, and watch the drop counters.
An unreachable collector raises no fault: it shows as a cached backlog on the card, and as send failures, until the collector is back and the cache drains.
Duplicates
Rarely, an event is sent twice — after the collector accepted part of a request, after a reconnect, or after the service stopped between sending and recording the send. Such re-sends are bounded and counted (records re-sent). In the native shape, dplens.run_id + host.name + dplens.source.name + dplens.source_seq identifies an event, so duplicates can be removed at search time. An aggregated event carries the sequence number of the first event in its group.
Connections
DPLens keeps one connection open to each address and reuses it from request to request. A connection idle for 50 seconds is closed before the collector or a load balancer drops it (idle_timeout_secs, 1 to 300).
Connections have no maximum age, so a change of DNS record or load-balancer member is picked up at the next reconnect, not immediately.
Each connection carries one request at a time. On a link with a long round trip, larger requests carry more per round trip: max_request_events (default 1,024) and max_request_bytes (default 1 MiB).
Failover and pools
As on the other destinations, give either an ordered failover_addresses list, or an equal-peer pool in addresses with the lb_rotate_secs, lb_rotate_bytes and lb_readmit_secs settings — not both. See Sending to Splunk for how a pool behaves.
Reading the card
The card names the type and shape — OpenTelemetry (OTLP) · Native — and adds up to three lines:
| Line | What it shows |
|---|---|
| Requests | Requests sent, accepted (2xx), connections opened and requests on a reused connection; connection events once they happen |
| Problems | Refusals by status, retries, and the five ways a record can be dropped: records rejected by the collector, bad records dropped, too-large records dropped, records dropped on refusal, records over the request size dropped. Amber when any refusal or drop is non-zero. |
| Encode | How events were shaped: summary messages, JSON-line messages, sent without a message, without a severity, large numbers sent as text, sent without source details after a change |
A fault names what the collector said:
| Fault | Check |
|---|---|
unauthorized (401) | The ingestion key: the handle, auth_header and auth_scheme |
forbidden (403) | The key was refused for this endpoint |
not-found (404) | The path (default /v1/logs) and the port |
server-error (500) | The collector failed the request — this is also how a collector answers a request it cannot read |
config-error (…) | The endpoint refused the form of the request: the address, the path, any proxy in between |
bad-request (400) / too-large (413) | A refused record could not be isolated safely; nothing was dropped |
spec-drop (…) | Records were dropped on a refusal |
Display names
Windows events carry their level, task, opcode and keywords as numbers. A Windows Event Log source's Display names option adds the names Event Viewer shows — Add puts them beside the numbers, Replace swaps the numbers for them. See Sources for the details and limits.
Use Add with the collector-compatible shape, so its level, task, opcode and keywords carry names as the collector's do. Avoid Replace on a Windows installation in a language other than English: severity is then read from the level's name, so those events arrive without a severity.
Things to know about ClickStack
- Very large integers are rounded in attributes. ClickStack stores attribute numbers above 2^53 (about 9 × 10^15) with rounding. File-integrity timestamps in nanoseconds are that large: the exact value is in the message, the attribute is rounded.
- NetFlow and file-integrity messages are parsed. ClickStack reads the JSON message of these events into attributes. Where a key is in both, the value parsed from the message wins, and the record's own keys —
source_id,source_seq,time_event,time_collected— appear as attributes too. - Numbers too large for a signed 64-bit integer are sent as text, and counted large numbers sent as text.
- Requests are not compressed, there is no OTLP/gRPC, and the destination does not go through an HTTP proxy.
Checking it works
- Use Send test event from a source's row menu.
- Watch Live stream with this destination selected. Because OTLP is a binary format, the stream shows each record in OTLP's JSON form. The ingestion key never appears there.
- In HyperDX, search for the machine's
host.nameor your source'sdplens.source.name.
If the card shows a cached backlog and no fault, DPLens cannot reach the collector: check the address, the port and the firewall. If it shows a fault, the collector is answering — the fault says what it objected to.