Circuit Breaker#

InfraGuard wraps each upstream C2 backend in an async circuit breaker that prevents cascading failures when a backend becomes unreachable. Instead of queuing requests against a dead host and burning through connection timeouts, the circuit opens after repeated failures and immediately drops or redirects requests until the backend recovers.

State Machine#

          success              failure count
  ┌──────────────────┐      exceeds threshold
  │                  │     ┌──────────────────┐
  ▼                  │     │                  ▼
CLOSED ──────────────┼─────┘               OPEN
  ▲                  │                       │
  │                  │         recovery       │
  │                  │         timeout        │
  │           ┌──────┘         expires        │
  │           │                  │            │
  │      success                 ▼            │
  └───────────────── HALF_OPEN ◄─────────────┘
              │           │
              │  failure   │
              │     │      │
              │     ▼      │
              │    OPEN    │
              └────────────┘

State	Behavior
CLOSED	Requests flow normally. Failures increment a counter.
OPEN	All requests are immediately rejected without contacting the backend.
HALF_OPEN	A single probe request is allowed through. Success transitions to CLOSED; failure returns to OPEN.

Configuration#

domains:
  cdn.example.com:
    upstream: "https://10.0.0.5:8443"
    circuit_breaker:
      failure_threshold: 5    # consecutive failures before opening (default: 5)
      recovery_timeout: 30    # seconds before transitioning OPEN -> HALF_OPEN (default: 30)

Tracked Failure Types#

The circuit breaker catches:

httpx.TimeoutException - upstream did not respond within the configured timeout
httpx.ConnectError - TCP connection refused or host unreachable

Other exceptions (HTTP 500s from the upstream, TLS errors, etc.) are not counted as circuit breaker failures since they indicate the backend is reachable but returning errors.

Behavior When Open#

When the circuit is open, InfraGuard falls through to the domain’s configured drop_action. This means:

If drop_action.type is redirect, the beacon gets a 302 to the decoy URL
If drop_action.type is reset, the connection is dropped with no response (TCP RST)
If drop_action.type is proxy, the request is forwarded to the decoy site

From the beacon’s perspective, an open circuit is indistinguishable from being filtered - this is intentional so that the implant’s retry logic handles both cases identically.

Metrics#

The circuit breaker state is exported as a Prometheus gauge:

infraguard_circuit_breaker_state{domain="cdn.example.com"} 0

Values: 0 = CLOSED, 1 = OPEN, 2 = HALF_OPEN.