Structured logging across microservices without a log aggregator

Cross-service logging has two problems. Structure: every service, framework, and language wants to log in its own format. Correlation: one request touches three services and leaves logs in three places, and connecting them requires an identifier that survives the trip.

The standard answer is a log aggregation stack — ELK, Loki, Datadog — which solves both problems and hands you a new distributed system to run, a parsing layer to maintain, and a bill that scales with how much your services like to talk. We solved both problems at the application layer instead: a shared JSON formatter, a context variable, and one HTTP header. No aggregator, no centralized log parsing, no sidecar containers.

The JSON formatter

Every service logs JSON:

{
  "timestamp": "2026-05-06T15:23:01.234Z",
  "level": "INFO",
  "logger": "app.api.flags",
  "message": "Flag evaluated",
  "request_id": "req-a7f2c891",
  "account_id": "acc-123e4567",
  "user_id": "usr-789abcde",
  "method": "GET",
  "path": "/api/v1/flags/checkout-v2",
  "status_code": 200,
  "duration_ms": 8
}

The formatter is written once, in smplkit-core, as a Python logging Formatter subclass: smplcore.request_logging.JSONFormatter. Every service points its root logger at it. No service invents its own format, which is most of what an aggregator’s parsing layer exists to clean up after.

CloudWatch Logs ingests JSON lines and CloudWatch Insights parses the fields automatically, so structured queries work out of the box.

The ContextVar pattern

Request context — request ID, account ID, user ID — rides through the call chain on Python’s contextvars.ContextVar, which behaves like thread-local storage that also works correctly under async/await:

from contextvars import ContextVar

_request_id: ContextVar[str | None] = ContextVar('request_id', default=None)
_account_id: ContextVar[str | None] = ContextVar('account_id', default=None)
_user_id: ContextVar[str | None] = ContextVar('user_id', default=None)

FastAPI middleware sets them at the start of each request:

@app.middleware("http")
async def request_logging_middleware(request: Request, call_next):
    request_id = request.headers.get("X-Request-Id") or generate_request_id()
    account_id = get_account_id_from_jwt(request)

    _request_id.set(request_id)
    _account_id.set(account_id)
    # ...

    response = await call_next(request)
    response.headers["X-Request-Id"] = request_id
    return response

The JSONFormatter reads the ContextVars when it formats each line. Every log statement in the request automatically carries the request ID, account ID, and user ID, without any callsite passing them explicitly.

The X-Request-Id header

Correlation crosses service boundaries in the X-Request-Id header. When the app service calls the flags service, it forwards the current request’s ID; the flags service’s middleware reads it and sets its own ContextVar. The same forwarding happens at every hop, so a request through three services leaves logs in three log groups, all carrying the same ID, and

filter request_id = "req-a7f2c891"

in CloudWatch Insights returns the whole story in one result set.

The header also goes back out the front door: every API response includes it. A customer who hits an error can paste the request ID into a support ticket, and that one string locates every log line the request produced, in every service it touched.

Stack traces

Exceptions log as structured fields, not as a wall of text stuffed into message:

{
  "level": "ERROR",
  "message": "Unhandled exception during request processing",
  "exception_type": "NotFoundError",
  "exception_message": "Flag 'checkout-v2' not found",
  "stack_trace": [
    "File \"app/api/flags.py\", line 47, in get_flag",
    "  raise NotFoundError(f\"Flag '{key}' not found\")",
    "NotFoundError: Flag 'checkout-v2' not found"
  ]
}

The payoff is that stack traces become queryable. Counting distinct exception_type values across all services is one Insights query; try that against free-text tracebacks and you’re writing regexes at 2 a.m. The formatter does the conversion whenever a record has an exception attached (logger.exception(...)).

CloudWatch is the log tool

The logs are already in CloudWatch, because that’s where ECS puts them. The only question was whether the default suffices once the lines are structured. So far it does:

fields @timestamp, level, message, request_id, account_id
| filter level = "ERROR"
| filter account_id = "acc-123e4567"
| sort @timestamp desc
| limit 20

Last 20 errors for one account, across every service.

The honest trade-off is speed. Insights queries take 5 to 30 seconds on large result sets. For incident forensics, fine. For monitoring, fine. For ad-hoc “what’s happening right now” poking, it’s sometimes slow enough to be annoying, and that’s the price of not running an aggregator. We’ll keep paying it for a while.

Where this stops scaling

Three things we already know we’ll want eventually. Sampling: CloudWatch charges per GB ingested, and logging 100% of successful requests is a habit for companies with either low traffic or a high tolerance for the bill — the plan is to sample successes and keep every error. A customer-facing log view, because the request IDs are in our logs but customers can’t query them; today the ID goes through a support ticket, which works but shouldn’t be the only way. And distributed tracing: a request ID tells you which services a slow request touched but not where the time went. When that question starts costing us real hours, OpenTelemetry exists.

None of it is urgent at our scale. The shared formatter and one header have been carrying the whole load.