Security
The Edge Threat Pipeline: From Simulated Dashboard to Real Detections
We replaced a fake attack map with a real pipeline: IDS detections from the AS210622 edge, shipped over an authenticated fail-closed path, stored in SQLite, and rendered live — with a 'Blocked: 0' stat we refuse to inflate.
- Discipline
- Security
- Built
- Reading time
- 4 min read
- Security
- IDS
- Observability
- AS210622
Context
This site has a live threat map. For a while, it was a lie — a client-side script invented an attack every second on top of a static JSON file, with headline numbers to match. That is the industry default for these pages, and it is exactly the kind of thing we decided to stop doing.
Today the map renders real detections from intrusion detection running at the AS210622 edge. Every arc you see corresponds to an actual signature match against actual traffic. This case study is how the pipeline works, and the design decisions that a public security feed forces on you.
Constraints
The page is public
Everything on it is available to the people probing us, so the feed must never become free reconnaissance.Router uplinks are metered and modest
The shipping agent runs on the edge routers themselves; the pipeline had to be light in both bandwidth and dependencies.One deploy, no new hosts
The collector had to live inside the existing site container, not become another service to operate.
Approach
The pipeline is deliberately short:
edge routers ── detection agent: tail alerts → enrich (ASN/country/city)
→ batch + gzip → authenticated ship to the site
│
SQLite (node:sqlite)
│
GET /api/threats → live map (polls every 5s)
A small Go agent on each router tails the IDS's event log, enriches each alert with source ASN, country, and city, then ships batches gzipped — alert JSON is repetitive and compresses roughly 10x, which matters on a metered uplink. The write path is authenticated and fail-closed: if its credentials are unconfigured, it rejects everyone rather than accepting anonymous writes. Events land in SQLite via Node's built-in driver (chosen because the Docker build runs npm ci --ignore-scripts and cannot compile native modules), on a named volume so history survives redeploys. Attack types are resolved at write time and indexed, so the read path serves 24-hour aggregates from covering indexes instead of classifying on every request.
Three decisions define the public feed:
Destinations are sites, never IPs
The feed says "Pitesti (Port 22 — SSH)", not which customer host was probed. The event type deliberately has no destination-IP field, so the mistake is unrepresentable.The engine goes unnamed
Public copy says "detections from the AS210622 edge" — naming the IDS tells an attacker exactly what to evade, so signature text is scrubbed before display and the page never counts sensors."Blocked" reads zero, because it is zero
The previous copy claimed mitigation that wasn't happening. The stat becomes non-zero when detections drive real firewall blocks — not before.
What broke
The classifier filed reputation-list hits as exploits
A signature ending in "…Block Listed Source" substring-matched a barercepattern and doubled the exploit count. The fix — reputation rules match first, short acronyms anchored with\b— is now pinned by a Vitest suite of 38 cases using real signature names, verified to fail if either property is reintroduced. The classification rules are mirrored in the Go agent, so both sides change together or not at all.Inline blocking is quietly expensive
A filter that drops a share of legitimate traffic is an outage you inflict on yourself, so anything that blocks at the edge earns its place through evidence first. Working through that evidence is the same alert-fatigue problem that led us to build meerkat, our open-source console that rolls thousands of alerts into per-source decisions.Secrets kept trying to enter the build
The mail integration originally constructed its client at module scope, making its API key a build-time requirement — which means a Docker build arg, which means a secret readable in image history. It now initializes on first request; the production build needs no environment at all.
Where it stands
The pipeline is in production: real events, fail-closed ingest, tested classification, 24-hour retention with indexed aggregates, and data that survives container recreation. The honest gaps are documented too — blocking is not yet wired to the firewall, and a multi-replica deployment would need the process-local cache rethought.
If your security dashboard is showing you noise — or fiction — we can help you build one that tells the truth.
Want one like this?
We build and run these ourselves. Tell us what you need and we will quote it.