Distributed pipeline at the edge
Distributed security data pipeline: processing at the edge vs a central tier
Your logs can be filtered and masked in a central processing tier, or at the edge. DPLens puts a complete pipeline in every Windows agent, so the work runs on your own machines and there is no central processing server to run.
- Windows x64
- A complete pipeline in every agent
- No central processing server
- No ingestion pricing
In short
A distributed security data pipeline runs its processing stages at the edge, on your own machines, rather than sending logs to a central processing tier first. DPLens is a UK-built, self-hosted, distributed security data pipeline at the edge for SOCs and MSSPs, delivered as a log collection agent for Windows. Each agent filters, parses, aggregates, optimises, enriches, masks and rate-limits events, so data is processed, masked and reduced on your own machines before it leaves, and sent only where you choose, with no central processing server to build or run. DPLens Manager, included in every subscription, manages the agents centrally, with full support for multi-tenant environments. Pricing is per agent and per network source, with no per-GB ingestion cost.
The usual pattern
What is a central processing tier?
A central processing tier is a set of servers that sits between your hosts and your SIEM. Practitioners call it an aggregator, a gateway, a worker tier or a pipeline tier. These are servers you size, patch and keep available, and data crosses your network to reach them before it is filtered or masked. Teams build one for good reasons. Each of those reasons has a home in a distributed design.
Central policy
A tier gives you one place to set the rules. In a distributed design, that is central configuration from DPLens Manager, which manages the agents and leaves the processing to them.
Shared processing
A tier gives every source the same stages. In a distributed design, every agent has all seven stages, so the same processing runs on your own machines.
Routing to several destinations
A tier can send data to more than one place. In a distributed design, one pipeline in the agent can deliver the same processed events to several destinations at once.
Side by side
What changes when the pipeline runs in the agent?
The work does not disappear. It moves into the agent, so the centre only has to manage.
| Question | Central processing tier | Distributed pipeline (DPLens) |
|---|---|---|
| Where filtering and masking run | On dedicated servers, after data crosses your network | In the agent on your own machines, before anything is stored or sent |
| What crosses your network | Whatever the collector did not exclude | What you chose to send, with your masking applied |
| What you provision | Processing servers, sized, patched and kept available | One agent per Windows host, and no processing tier to run |
| How capacity grows | Add capacity to the tier | Each agent carries its own share of the work |
| Central policy | Set in the tier | Central configuration from DPLens Manager |
| Routing | From the tier | From each agent, to several destinations at once |
What you get
Four outcomes of processing at the edge
01
Infrastructure savings
No central processing server means no processing tier to size, host, patch, scale or make highly available, and it saves the resource and infrastructure that tier would have taken.
02
Data sovereignty
For data sovereignty, that means data is processed, masked and reduced on your own machines before it leaves, and sent only where you choose. You decide what leaves. Not legal or audit advice.
03
Less traffic on WAN and branch links
Events you drop never cross your network, so what travels between sites is only what you chose to send.
04
No ingestion pricing
Licensed per agent and per network source, so your data volume never sets the DPLens price. Reducing what reaches your SIEM is a separate saving, wherever your SIEM prices by volume. Estimate that saving.
A highly optimised agent
What does a pipeline in the agent do to the host?
An agent that carries the pipeline has to earn its place on the host. The DPLens agent is highly optimised, which is what makes processing on your own hosts practical.
Both figures are from our lab benchmark. Lab figures, not a guarantee. Request an evaluation licence and measure DPLens on your own servers, with your own pipeline rules.
Central management, included
Manage centrally. Process at the edge.
A pipeline in every agent still needs one place to manage it. DPLens Manager manages your agents and does not process your data, because the pipeline stays in every agent, and event data never reaches it. If Manager is unavailable, deployed agents keep collecting and delivering on their last configuration. It is included in every subscription, and gives you central configuration, fleet health monitoring, and deployment and upgrades, self-hosted and air-gap capable.
Whether you run one estate or many customers' estates, DPLens Manager has full support for multi-tenant environments. See DPLens Manager in a demo, or read more about DPLens Manager.
Sources around the host
Where do firewalls, switches and appliances fit?
The edge is your Windows hosts, where the DPLens agent runs. Firewalls, switches and appliances reach it as syslog and NetFlow sources received by an agent, which applies the same pipeline before delivery. Each receiver is one network source, never counted per sending device: one syslog receiver taking events from 500 firewalls is one network source.
Who it is for
What it means for SOCs and MSSPs
For SOCs
No central processing server to fund or keep available, seven stages on your own machines, and every drop counted against the rule that caused it.
DPLens for SOCs →For MSSPs
No processing infrastructure to stand up per customer, each customer's data processed on their own machines, and central management included.
DPLens for MSSPs →Keeping what you have
What if I already run a pipeline?
DPLens reduces and masks first, then delivers to what you run: an OpenTelemetry Collector over OTLP over HTTP, or NDJSON to a collector you run. See all integrations.
Secure by design
An agent you can defend in a security review.
FAQ
Questions buyers ask
What is a distributed security data pipeline?
A distributed security data pipeline runs its processing stages at the edge, instead of in a central tier. With DPLens, a complete pipeline runs in the agent on each Windows host.
What is the difference between edge processing and a central pipeline tier?
Edge processing filters and masks on or near the source. A central tier does it on dedicated servers after data has crossed the network, and those servers have to be sized and maintained.
Do I need a central processing server to run DPLens?
No. Processing happens in each agent. DPLens Manager, included in every subscription, handles central configuration, fleet health monitoring, and deployment and upgrades. Event data never reaches it.
What happens if DPLens Manager is unavailable?
Agents already deployed keep collecting and delivering events on their last configuration.
Does processing logs on the host slow the host down?
In our lab benchmark, DPLens delivered more than 30× the throughput of the popular free and paid Windows agents we tested, at about a tenth of the CPU per event. Lab figures, not a guarantee. Request an evaluation licence and measure DPLens on your own servers, with your own pipeline rules.
How does edge processing help with data sovereignty?
Data is processed, masked and reduced on your own machines before it leaves, and sent only where you choose. DPLens sends no telemetry to us, and you decide what leaves your network. Not legal or audit advice.
How do firewalls and other network devices fit in?
Firewalls and other network devices send syslog or NetFlow to a DPLens agent, which filters and masks before delivery. Each receiver is one network source, however many devices send to it.
What does DPLens run on?
DPLens runs on Windows Server 2016–2025 and Windows 10 and 11, x64.
Is DPLens a log forwarder or a security data pipeline?
Both. DPLens is a Windows log collection agent with a complete pipeline built in, delivering to Splunk, any syslog or JSON SIEM, or an OpenTelemetry Collector.
Related
Where to go next
Log masking and PII redaction
Sensitive values dealt with on your own machines, before anything is stored or sent.
Read more →Air-gapped log collection
A pipeline at the edge with no vendor cloud to depend on, and DPLens Manager is air-gap capable too.
Read more →Trust Centre
An agent on every host has to be trustworthy: signed, with an SBOM and a tamper-evident audit trail.
Read more →See it on your own logs
Try DPLens on one Windows server before you talk to anyone.
Book a demo, or request an evaluation licence and install it on a test host. The documentation covers everything from requirements to Group Policy rollout.