DDoS Mitigation at the Network Edge: How We Keep AS210622 Clean

Kicked TeamJanuary 5, 20266 min read

Last month we mitigated a 380 Gbps UDP reflection attack that lasted 47 minutes. Our customers didn't notice. No alerts, no degradation, no support tickets.

That's the goal: make DDoS attacks invisible to the people behind your network. Here's how we built mitigation into every layer of AS210622.

The Threat Landscape

Operating an ASN means you're a target. The attacks we see most frequently:

Attack Type Frequency Typical Volume Protocol
UDP reflection (DNS, NTP, memcached) Daily 10-400 Gbps UDP
SYN floods Weekly 1-50 Mpps TCP
HTTP floods (L7) Weekly 50k-500k RPS HTTP/S
Carpet bombing (spread across /24) Monthly 50-200 Gbps Mixed
GRE/IP-in-IP floods Occasional 10-100 Gbps GRE

The key insight: volumetric attacks and application-layer attacks require completely different mitigation strategies. A single solution doesn't cut it.

Layer 1: Transit Provider Filtering

The first line of defense is before traffic even reaches our routers. We work with our transit providers to:

  • Filter bogons — RFC 1918 addresses, unallocated space, and our own prefixes should never arrive from transit
  • Rate-limit ICMP — No legitimate use case for gigabits of ICMP
  • Community-based blackholing — If we tag a prefix with a blackhole community, the transit provider drops it upstream

This handles the "blunt instrument" scenarios. But for surgical mitigation, we need our own tools.

Layer 2: BGP Flowspec

BGP Flowspec lets us push traffic filtering rules to our edge routers via BGP. Think of it as a firewall that propagates across your entire network in seconds.

# Block UDP traffic from any source to our customer's IP on port 53
# (common DNS amplification target)
flow {
  match destination 185.x.x.10/32;
  match protocol udp;
  match destination-port 53;
  then discard;
}

The power of Flowspec is speed and scope. One BGP update, and every edge router in every PoP drops the traffic. No SSH-ing into routers, no manual ACL changes.

We've automated Flowspec rule generation based on traffic analysis:

  1. sFlow samples every packet at 1:1000 ratio
  2. FastNetMon analyzes flows in real-time
  3. When an attack is detected, FastNetMon pushes a Flowspec rule via ExaBGP
  4. Edge routers drop the attack traffic within 2-3 seconds

Layer 3: XDP/eBPF Filtering

For attacks that require more intelligence than "drop all UDP to port X," we use XDP (eXpress Data Path) programs running on our edge servers.

XDP processes packets at the NIC driver level, before the kernel networking stack. This means we can inspect and drop millions of packets per second with minimal CPU overhead.

// Simplified XDP program — drop packets matching attack signature
SEC("xdp")
int xdp_ddos_filter(struct xdp_md *ctx) {
    void *data = (void *)(long)ctx->data;
    void *data_end = (void *)(long)ctx->data_end;
    
    struct ethhdr *eth = data;
    if ((void *)(eth + 1) > data_end)
        return XDP_PASS;
    
    if (eth->h_proto != htons(ETH_P_IP))
        return XDP_PASS;
    
    struct iphdr *ip = (void *)(eth + 1);
    if ((void *)(ip + 1) > data_end)
        return XDP_PASS;
    
    // Drop UDP packets with specific payload signature
    if (ip->protocol == IPPROTO_UDP) {
        struct udphdr *udp = (void *)ip + (ip->ihl * 4);
        if ((void *)(udp + 1) > data_end)
            return XDP_PASS;
        
        // Check against attack signatures in BPF map
        if (is_attack_signature(udp, data_end))
            return XDP_DROP;
    }
    
    return XDP_PASS;
}

XDP is our secret weapon for:

  • SYN floods — SYN cookie validation at wire speed
  • Carpet bombing — Per-IP rate limiting across a /24 without blackholing legitimate traffic
  • Protocol anomalies — Dropping malformed packets before they waste CPU

Layer 4: Scrubbing Centers

For the largest volumetric attacks that exceed our port capacity, we divert traffic through scrubbing centers using BGP:

  1. Attack detected (FastNetMon)
  2. We announce the targeted prefix to our scrubbing provider with a longer path
  3. Traffic flows through the scrubber, which removes attack traffic
  4. Clean traffic is tunneled back to us via GRE
  5. When the attack stops, we withdraw the announcement and traffic flows directly again

We use this as a last resort because it adds latency (the GRE tunnel detour). Our goal is to handle 95% of attacks with Flowspec and XDP alone.

Layer 5: Application-Layer Protection

L7 attacks (HTTP floods) bypass all of the above because they use legitimate TCP connections. For these, we implement:

  • Rate limiting per IP, per session, per endpoint
  • Challenge pages — JavaScript challenges that bots can't solve
  • Behavioral analysis — Legitimate users don't request the same endpoint 1000 times per second with identical headers
  • WAF rules — Block known attack patterns (SQLi, XSS) that are often mixed into DDoS campaigns

Detection: Seeing Attacks Before They Hit

You can't mitigate what you can't see. Our detection pipeline:

sFlow (every packet sampled at 1:1000)
  → FastNetMon (real-time flow analysis)
    → Anomaly detection (baseline comparison)
      → Threshold alerts + auto-mitigation
        → Grafana dashboard (visibility)
          → PagerDuty (notification if manual intervention needed)

We maintain per-customer traffic baselines. A gaming server that normally receives 5 Gbps of UDP isn't alarming. But a web server that suddenly receives 5 Gbps of UDP definitely is.

The Numbers

Our mitigation infrastructure by the numbers:

  • < 3 seconds — Time from detection to automated mitigation
  • 400 Gbps — Maximum inline mitigation capacity (Flowspec + XDP)
  • 2+ Tbps — Scrubbing center capacity (for the big ones)
  • 99.7% — Attacks mitigated without customer impact
  • 0 — False positives causing legitimate traffic drops (in the last 6 months)

Lessons from the Trenches

  1. Automate everything — Manual mitigation is too slow for modern attacks
  2. Layer your defenses — No single tool handles all attack types
  3. Test regularly — We run controlled DDoS simulations monthly
  4. Over-provision — Our edge can handle 3x our normal peak. The headroom is your mitigation budget
  5. Communicate — Customers should see attack data on their dashboard, not find out from Twitter

Your Infrastructure, Protected

Every server on our network benefits from this mitigation stack. It's not an add-on or a premium tier — it's built into the infrastructure.

Want to see our network in action? Check the threat map for real-time attack visualization, or explore our network architecture.