Smart Scale-to-Zero: Buffering TCP SYNs While Waking Dormant Pods
A technical breakdown of how the MeshAtom node agent holds inbound TCP handshakes in-kernel, wakes the exact pod on demand, and completes the connection without client retries.
Idle Dev and QA namespaces burn money. Traditional scale-to-zero either drops connections during wake (forcing client retries) or keeps a warm proxy tax forever. MeshAtom takes a third path: the lightweight namespace-bound node agent stays awake, intercepts the first packet in eBPF, buffers the TCP SYN, wakes only the target workload, then replays the handshake as if the pod had been ready all along.
The problem with naive scale-to-zero
- Client SYN arrives while replicas = 0 → connection refused or timeout
- External autoscalers react after the failure, so the first request always pays the cold-start tax twice
- Sidecar-based meshes cannot help: there is no sidecar if there is no pod
Where MeshAtom sits in the path
The agent attaches eBPF programs at TC/XDP and the socket layer for the annotated namespace. When scale-to-zero is enabled, an eBPF map tracks which Service backends are dormant. Inbound packets for those backends never hit a dead endpoint — they enter a per-flow hold buffer owned by the agent.
High-level wake path
Client Node Agent (always up) Target Pod | | | |---- TCP SYN ------------>| | | [eBPF: match dormant backend] | | [buffer SYN + options in map] | | [emit wake event to userspace] | | |---- scale Deployment 0→1 --->| | |<--- Ready / endpoint up -----| | [replay buffered SYN to pod] | | |----------------------------->| |<------------- SYN-ACK ---------------------------------| |---- ACK ---------------> | ---------------------------->| | [normal kernel forwarding resumes] |
Buffering the TCP SYN in-kernel
- Classify: eBPF looks up the destination Service / pod identity in a dormant-backends map keyed by namespace + clusterIP:port.
- Hold: Instead of forwarding or dropping, the program clones essential SYN state (seq, window, TCP options, 5-tuple) into a ring buffer / LRU map with a short TTL (typically a few seconds).
- Signal: A userspace wake worker receives the event and issues a targeted scale-up for the owning Deployment / StatefulSet / custom workload controller — only that workload, only that namespace.
- Gate: While waking, subsequent SYNs for the same flow coalesce into the same buffer entry to avoid thundering herds.
- Replay: When the endpoint appears in the agent’s local endpoint map, the agent synthesizes the original SYN toward the pod (or splices the held connection) and lets the kernel complete the three-way handshake.
- Release: The hold entry is deleted; further packets take the fast path with no userspace involvement.
Why clients do not need special retries
From the client’s perspective the SYN simply takes longer to receive a SYN-ACK — within normal TCP retransmission windows for cold starts measured in hundreds of milliseconds to a couple of seconds. There is no ECONNREFUSED, no application-level “retry after 503”, and no always-on Envoy soaking CPU while the namespace sleeps.
Enabling it
Namespace opt-in
apiVersion: v1
kind: Namespace
metadata:
name: qa-checkout
annotations:
meshatom.io/network-mode: "cni-overlay"
meshatom.io/scale-to-zero: "enabled"
# Optional: max time to hold a SYN before failing closed
meshatom.io/scale-to-zero-hold-ms: "3000"