Destinations

Sending to ClickStack

OpenTelemetry (OTLP/HTTP) delivery to ClickStack — self-hosted, Managed or ClickStack Cloud — with severity, service names and searchable attributes.

The otlp destination delivers to an OpenTelemetry collector over OTLP/HTTP: protobuf requests to /v1/logs, conventionally on port 4318. ClickStack's views — severity charts, services, the attribute facets you search on — are built on the columns OTLP fills, so events arrive already broken out rather than as one line of text.

It carries events from every kind of source: Windows Event Log, log files, file integrity, syslog and NetFlow.

ClickHouse, ClickStack and HyperDX are trademarks of ClickHouse, Inc. DPLens

is not affiliated with or endorsed by ClickHouse, Inc.

Which ClickStack

The same destination serves all three ways of running ClickStack.

DeploymentWhat you point DPLens atStatus
Self-hosted — all-in-one, Docker Compose or HelmThe collector of the ClickStack you run. It listens on plain HTTP, port 4318, by default, and expects the ingestion key from HyperDX → Team Settings → API Keys.Tested
ManagedAn OpenTelemetry collector you run in front of Managed ClickStack. TLS and a key are whatever you configured on that collector.Compatible; tested with self-hosted ClickStack
ClickStack CloudThe HTTPS endpoint of your service, <service>.otel.<region>.aws.clickhouse.cloud:4318, with the ingestion key.Compatible; tested with self-hosted ClickStack

The destination speaks standard OTLP/HTTP, so another OpenTelemetry collector that accepts OTLP/HTTP logs can receive from it too.

Setting it up in the console

  1. On Destinations, choose + Add destination, then the ClickStack card — Confirmed hand-off · OTLP/HTTP · port 4318.
  2. Pick the ClickStack deployment: Self-hosted, Managed or ClickStack Cloud. This only pre-fills the form — TLS, an example address, and the key handle, which Managed leaves empty because a key is optional there — and never changes a value you have typed.
  3. Enter the Address as host:port. A bare host name gets port 4318. You can also paste the endpoint URL, such as https://clickstack.example.internal:4318/v1/logs, and it is split into the address, TLS and the path for you.
  4. Under Ingestion key, paste the key into Key. It is sealed into the agent's secret store when you save; afterwards the form shows only Key: configured ✓. Leave the Key handle at its default unless you keep several keys. To send no key at all, clear the handle.
  5. Turn on Encrypt with TLS (HTTPS) if your endpoint is HTTPS — ClickStack Cloud always is.
  6. Choose the Shape — see Native or collector-compatible.
  7. Optionally add Service names and resource attributes — see Service names and resource attributes.
  8. Save, map a pipeline to the destination, and apply.

If you type port 4317, the form warns you: that is the collector's OTLP/gRPC port, and this destination sends OTLP/HTTP.

Setting it up in configuration

destinations:
  - type: otlp
    name: clickstack
    params:
      address: "clickstack.example.internal:4318"
      auth_handle: "secret://clickstack/ingestion-key"
      shape: "native"
      cache_max_bytes: "2147483648"
    otlp:
      service_names:
        security-log: windows-security
        app-log: payments-api
      resource_attributes:
        deployment.environment: production

Seal the key before you apply a configuration that names the handle — validation refuses a handle with nothing stored under it:

"C:\Program Files\DPLens\dplens.exe" --set-secret secret://clickstack/ingestion-key

The value is read from standard input, never from the command line. Every setting is listed in the configuration reference, and Configuration examples has a complete file.

The ingestion key

Three settings decide how the key is sent. Only one of them has anything to do with the key itself.

SettingWhat it isDefault
auth_handleThe secret-store handle the key is sealed under. This is the only place the key goes.Not set — no key is sent
auth_headerThe name of the HTTP header that carries the key. A name, never a value.authorization
auth_schemeraw: the header value is the key itself. bearer: the header value is Bearer <key>.raw

Never paste the key, or Bearer and the key, into auth_header or any other configuration field. A configuration file is meant to be safe to copy, check in and push to a fleet; a key typed into it is none of those. ClickStack expects the key in the authorization header, so the defaults are right for it: seal the key under auth_handle and leave the other two alone. Self-hosted ClickStack accepts both the raw and the bearer form.

The key is never shown in the configuration, the logs, the console or the live stream, and if the collector echoes it back in an error message, it is removed before that message is displayed or logged.

To change the key on a running agent, use Replace beside the key on the destination in the console. The new key is sealed under the same handle, and the change is recorded in the audit log.

Plain HTTP or HTTPS

A self-hosted ClickStack collector listens on plain HTTP by default, so tls is off by default here — turning it on would fail against an unprepared collector. Over plain HTTP, though, your events and the ingestion key cross the network unencrypted. The console says so: a note in the form, and a key sent unencrypted tag on the destination card.

https:// endpoints are supported:

      tls: "true"
      ca_file: "C:\\ProgramData\\DPLens\\ca\\clickstack-ca.pem"

With TLS on, certificate validation is on, against the Windows trust store unless you give ca_file. The name verified is the host in address. Mutual TLS uses client_cert_handle and client_key_handle, as secret handles.

Use TLS and the ingestion key wherever the network path is not one you fully trust. Together they mean nothing in the path can read your events or the key, or pretend to be your collector.

Native or collector-compatible

shape decides how each event is laid out. It is set per destination.

ShapeWhat you getChoose it when
native (default)Every event field as a searchable attribute, a severity, the source it came from, and the agent that sent it.You are starting fresh. Recommended.
collectorThe layout the standard OpenTelemetry Collector gives Windows event logs, syslog and log files.You already have dashboards and saved searches built on that layout.

The collector-compatible shape matches the collector's layout — the same keys, nesting and value types — not every value. For example, DPLens always sets the event time on a log-file line, and syslog severity text is the upper-case OpenTelemetry name. File-integrity and NetFlow events have no collector layout and are always sent native.

For the Windows Event Log layout to carry level, task, opcode and keyword names as the collector does, set Display names to Add on the source — see Display names.

What a native event carries

On the resource — the part HyperDX groups by:

AttributeValue
host.nameWhere the event came from — see The host name
service.nameBlank, unless you name a service for the source
dplens.source.nameThe source's name in your configuration
dplens.source.kindThe kind of source: Windows Event Log, file, and so on
dplens.agent.id / dplens.agent.versionThe agent that sent it
dplens.run_idA random number chosen each time the source starts

On each record: dplens.source_seq, a sequence number within the source, and every field of the event. The record's timestamp is when the event happened; its observed timestamp is when DPLens read it.

The collector-compatible shape carries host.name, your service names and your resource attributes, and none of the dplens.* attributes.

The message

EventWhat ClickStack shows as the message
Anything with a message — log lines, syslog, Windows events with message rendering onThe message
A Windows event with message rendering off (the default)A one-line summary: event ID, channel and provider, such as 4624 Security Microsoft-Windows-Security-Auditing
NetFlow and file-integrity eventsThe same JSON line the NDJSON destination would send. The fields are also sent as attributes, so these events cost about twice the bytes.
A log line or syslog event with no messageNo message; the fields are still sent as attributes, and the event is counted sent without a message

Severity

SourceSeverity
Windows Event LogFrom the event's level: Critical → FATAL, Error → ERROR, Warning → WARN, Information → INFO, Verbose → DEBUG. In the native shape an Audit Failure is raised to at least WARN.
SyslogFrom the message's severity, through the OpenTelemetry table (emergency → FATAL … debug → DEBUG) — once a parse-syslog stage has run in the pipeline.
Log files and everything elseNone

An event with no severity is sent without one and counted without a severity. ClickStack may then infer a level from the message text, and it shows severity text in lower case.

The host name

For a source on this machine, host.name is the machine's own name. For a syslog or NetFlow receiver it is the device that sent the event: for syslog, the host name in the message header when the pipeline has a parse-syslog stage, otherwise the sending address. Validation warns when a syslog source reaches this destination without one. These are the same rules as the source_id of an NDJSON record.

Events still in the disk cache from before a change to your list of sources are sent without their source details — no service name and no dplens.source attributes — and counted sent without source details after a change, because DPLens cannot be sure which source they now belong to.

Service names and resource attributes

The destination's otlp: block sets two things, in the console under Service names and resource attributes:

    otlp:
      service_names:
        security-log: windows-security
      resource_attributes:
        deployment.environment: production
        team: soc

Both apply in either shape. The whole block is limited to 64 KiB of UTF-8 text. Everything in it travels with every request, so validation also warns when it takes half of max_request_bytes, and refuses it if no event could fit.

Confirmed hand-off

Events are checkpointed only after the collector accepts them, and a disk cache rides out outages. Records the collector rejects are dropped and counted on the destination card.

What the collector answers decides what happens:

The collector answersWhat DPLens does
Accepted (2xx)Delivered. If the collector reports that it rejected some records of the request, those records are dropped and counted.
Busy or unavailable (408, 429, 502, 503, 504)Retries with backoff, honouring the collector's Retry-After up to five minutes.
Your configuration is wrong (401, 403, 404, 415, a redirect, 500, 501 and similar)Nothing is dropped. Events stay in the disk cache and the card shows a fault until you fix it. Redirects are not followed — point the destination at the collector itself.
This content is bad (400) or too large (413)Splits the request to find the record the collector refuses, and drops only that record, and only once the collector has accepted the rest. If it cannot show that, nothing is dropped: events stay cached and the card shows a fault.
Any other refusalDrops the request's records, as the OpenTelemetry specification directs, counts them and shows a fault.

Two things to know:

An unreachable collector raises no fault: it shows as a cached backlog on the card, and as send failures, until the collector is back and the cache drains.

Duplicates

Rarely, an event is sent twice — after the collector accepted part of a request, after a reconnect, or after the service stopped between sending and recording the send. Such re-sends are bounded and counted (records re-sent). In the native shape, dplens.run_id + host.name + dplens.source.name + dplens.source_seq identifies an event, so duplicates can be removed at search time. An aggregated event carries the sequence number of the first event in its group.

Connections

DPLens keeps one connection open to each address and reuses it from request to request. A connection idle for 50 seconds is closed before the collector or a load balancer drops it (idle_timeout_secs, 1 to 300).

Connections have no maximum age, so a change of DNS record or load-balancer member is picked up at the next reconnect, not immediately.

Each connection carries one request at a time. On a link with a long round trip, larger requests carry more per round trip: max_request_events (default 1,024) and max_request_bytes (default 1 MiB).

Failover and pools

As on the other destinations, give either an ordered failover_addresses list, or an equal-peer pool in addresses with the lb_rotate_secs, lb_rotate_bytes and lb_readmit_secs settings — not both. See Sending to Splunk for how a pool behaves.

Reading the card

The card names the type and shape — OpenTelemetry (OTLP) · Native — and adds up to three lines:

LineWhat it shows
RequestsRequests sent, accepted (2xx), connections opened and requests on a reused connection; connection events once they happen
ProblemsRefusals by status, retries, and the five ways a record can be dropped: records rejected by the collector, bad records dropped, too-large records dropped, records dropped on refusal, records over the request size dropped. Amber when any refusal or drop is non-zero.
EncodeHow events were shaped: summary messages, JSON-line messages, sent without a message, without a severity, large numbers sent as text, sent without source details after a change

A fault names what the collector said:

FaultCheck
unauthorized (401)The ingestion key: the handle, auth_header and auth_scheme
forbidden (403)The key was refused for this endpoint
not-found (404)The path (default /v1/logs) and the port
server-error (500)The collector failed the request — this is also how a collector answers a request it cannot read
config-error (…)The endpoint refused the form of the request: the address, the path, any proxy in between
bad-request (400) / too-large (413)A refused record could not be isolated safely; nothing was dropped
spec-drop (…)Records were dropped on a refusal

Display names

Windows events carry their level, task, opcode and keywords as numbers. A Windows Event Log source's Display names option adds the names Event Viewer shows — Add puts them beside the numbers, Replace swaps the numbers for them. See Sources for the details and limits.

Use Add with the collector-compatible shape, so its level, task, opcode and keywords carry names as the collector's do. Avoid Replace on a Windows installation in a language other than English: severity is then read from the level's name, so those events arrive without a severity.

Things to know about ClickStack

Checking it works

  1. Use Send test event from a source's row menu.
  2. Watch Live stream with this destination selected. Because OTLP is a binary format, the stream shows each record in OTLP's JSON form. The ingestion key never appears there.
  3. In HyperDX, search for the machine's host.name or your source's dplens.source.name.

If the card shows a cached backlog and no fault, DPLens cannot reach the collector: check the address, the port and the firewall. If it shows a fault, the collector is answering — the fault says what it objected to.