The Gateway: Where Agents Plug In

Through Chapter 13, jackdaw can read its config (Chapter 11), call upstreams (Chapter 12), and pick a model inside a routing group (Chapter 13) — but it only talks to you at the terminal. Without a front door, every agent manages its own API keys, and switching models means editing configs one by one. This chapter installs the door with axum: a gateway that speaks exactly one dialect on the outside — the OpenAI-compatible chat protocol.

By the end of this chapter you should be able to answer:

• In chat_completions's parameter list, why must body: Bytes come last — and who hands those bytes over?
• A "stream": true request has already sent its first chunk to the agent when the upstream dies. Why does the gateway refuse to failover and instead emit one final SSE error event?
• Which two values do you fill in to point pi at jackdaw, and what does each one map to in the config?

The Skeleton: Two Routes, One Piece of State

/// Shared per-process state: one router, one resolved token.
pub struct AppState {
    pub router: Arc<Router>,
    /// The token agents must present. Resolved once at startup so a missing
    /// env var fails fast (exit code 2) instead of 401-ing every request.
    pub token: String,
}

/// Assemble the axum app. Public so integration tests can serve it on a
/// random port with a known token.
pub fn build_app(state: Arc<AppState>) -> axum::Router {
    axum::Router::new()
        .route("/v1/chat/completions", post(chat_completions))
        .route("/v1/models", get(list_models))
        .with_state(state)
}

How to read it: start at build_app. route binds "method + path" to a handler, and the whole gateway has exactly two of them. with_state attaches one Arc<AppState> to the routing tree, and every handler gets back the same one — Arc is a reference-counted smart pointer, so cloning it bumps a counter instead of copying data. The state has just two fields: the Router from Chapter 13, and a token resolved at startup. The comment says why it's read that early: a missing environment variable should blow up at startup with exit code 2 (Chapter 10), instead of 401-ing every request. The pub on build_app has a reason too, also in the comment: integration tests serve it on a random port (Chapter 15 covers that). Finally, serve binds 127.0.0.1 only, on the port from gateway.port (--port overrides it at launch) — 3426 by default.

A Request's Byte-Level Journey

Pin this interaction as the spine of the chapter; everything below chases its bytes:

$ jackdaw serve &
$ curl -s 127.0.0.1:3426/v1/chat/completions -i \
    -H "Authorization: Bearer $JACKDAW_TOKEN" \
    -d '{"model":"group/coder","messages":[{"role":"user","content":"hi"}]}'
HTTP/1.1 200 OK
x-jackdaw-model: openai/gpt-5.2
{"choices":[{"message":{"role":"assistant","content":"Hello!"}}],...}

The bytes leave curl: TCP connects to 127.0.0.1:3426, then the request line, the Authorization header, and the JSON body go on the wire. axum collects the request and hands the raw body bytes to the handler — no parsing on the way.

/// Order of extractors matters: the body (`Bytes`) must come last.
async fn chat_completions(
    State(state): State<Arc<AppState>>,
    headers: HeaderMap,
    body: Bytes,
) -> Response {
    if !authorized(&headers, &state.token) {
        tracing::warn!("rejected a request without a valid bearer token");
        return error_response(
            StatusCode::UNAUTHORIZED,
            "Missing or invalid Authorization header. Send `Authorization: Bearer <token>` \
             where <token> is the value of the gateway's auth_token_env variable.",
        );
    }

    let request = match decode_request(&body) {
        Ok(request) => request,
        Err(message) => return error_response(StatusCode::BAD_REQUEST, message),
    };

    let requested = request.model_field().to_string();
    if request.wants_stream() {
        stream_reply(&state, request, &requested).await
    } else {
        complete_reply(&state, request, &requested).await
    }
}

How to read it: the parameter list first. axum's convention: declare what you want, and axum pulls it out of the request — such parameters are extractors. The three here each take a share: the shared state, all request headers, the body bytes. The comment states the ordering rule: an extractor that consumes the body must go last, because the body can be taken only once — ownership rules apply at the HTTP layer too (Chapter 5). The function body is three gates. authorized checks the header; failure means 401. decode_request runs serde_json::from_slice on the bytes straight into a ChatRequest (the Value newtype from Chapter 12), adding exactly one requirement — "must be a JSON object" — or it's a 400. Then model_field().to_string() copies "group/coder" into requested; the timing matters, because this happens before request is moved into the router, saving the name for logs and error messages. Finally wants_stream() (the "stream" field in the body) picks the return path.

/// Bearer check: exact `Authorization: Bearer <token>` match.
fn authorized(headers: &HeaderMap, token: &str) -> bool {
    let Some(value) = headers
        .get(header::AUTHORIZATION)
        .and_then(|v| v.to_str().ok())
    else {
        return false;
    };
    value
        .strip_prefix("Bearer ")
        .is_some_and(|presented| presented == token)
}

How to read it: let Some(value) … else covers both bad ends — a missing header, or a value that can't be read as text (non-ASCII bytes, say) — and either way it's false right away. Otherwise strip the Bearer prefix and compare the remainder against the startup token in one plain, exact comparison. No hashing, no constant-time comparison; combined with the 127.0.0.1-only bind, this guards against other processes on your machine, not the public internet.

Past both gates, the bytes hand off to Chapter 13's router: resolve group/coder, pick a starting point per the order strategy, rewrite the model field in the body to openai/gpt-5.2 (everything else untouched), and send it through Chapter 12's provider. If the router reports an error, route_error_response does the translating: an unresolvable reference → 400; an upstream Fatal 4xx → status and body passed through unchanged, because switching models won't fix a bad request; the whole group failing → 502 with one line per member in details. On a first read, remember that three-way mapping and move on.

The Way Back: Whole JSON, SSE Passthrough, and x-jackdaw-model

There are two ways back. With stream not true, complete_reply wraps what the router returns — (used, response) — into (StatusCode::OK, Json(response.0)) and stamps it; the 200 OK and x-jackdaw-model: openai/gpt-5.2 lines in the curl output are its product. With stream true, the streaming (SSE) path runs instead, with one extra wrinkle:

async fn stream_reply(state: &AppState, request: ChatRequest, requested: &str) -> Response {
    match state.router.stream(request).await {
        Ok((used, stream)) => {
            // ...
            let passthrough = stream.map(|item| match item {
                // Both arms yield `Ok`: a converted failure is data now,
                // not an error, so the body stream itself never fails.
                Ok(chunk) => Ok::<_, std::convert::Infallible>(chunk),
                Err(error) => Ok(sse_error_event(&error)),
            });
            let response = (
                StatusCode::OK,
                [
                    (header::CONTENT_TYPE, "text/event-stream"),
                    (header::CACHE_CONTROL, "no-cache"),
                ],
                Body::from_stream(passthrough),
            )
                .into_response();
            tag_model(response, &used)
        }
        // ...
    }
}

How to read it: start at stream.map. stream is the upstream stream the router hands over (the chunk-by-chunk SSE reading from Chapter 12), and map matches each item: healthy chunks pass through untouched — that's what passthrough means, the gateway never parses individual data: events — while a failed chunk becomes one SSE event carrying error JSON, courtesy of sse_error_event. Both arms return Ok, and the comment explains the intent: a failure converted into data means the stream itself never fails. Infallible is the "never happens" error type; it writes that sentence into the type. Body::from_stream turns the stream into the response body, carrying the text/event-stream and no-cache headers.

Why not just let the Err escape, or switch models mid-stream? Because once the first byte leaves the process, failover is off the table — the agent would stitch two models' output into one franken-answer. Chapter 13 fixed that semantic; the gateway's side of the deal is that all failover happens before router.stream() returns. After that, one error event and an honest end of stream.

The transparency philosophy lands in one last function:

/// Attach the transparency header naming the model that actually answered.
fn tag_model(mut response: Response, used: &str) -> Response {
    // Model ids are config-controlled; a non-ASCII id would make an invalid
    // header, so the tag is skipped rather than dooming the whole response.
    if let Ok(value) = axum::http::HeaderValue::from_str(used) {
        response.headers_mut().insert("x-jackdaw-model", value);
    }
    response
}

How to read it: one line of real work — put used into x-jackdaw-model. The if let Ok reasoning sits in the comment: model ids come from your config, and a non-ASCII id would build an invalid HTTP header; drop the tag rather than doom the whole response. The gateway doesn't pretend to be one model — it reports who actually answered. Here, transparency is a verifiable response header.

GET /v1/models: Selling Groups as Models

An agent's config wants a model id, and the only ids jackdaw sells are groups:

/// Advertise every group as a model id, in OpenAI's list format. Group ids
/// are the stable names agents configure once; members may change freely.
async fn list_models(State(state): State<Arc<AppState>>) -> Response {
    let data: Vec<Value> = state
        .router
        .config()
        .groups
        .keys()
        .map(|id| {
            json!({
                "id": format!("group/{id}"),
                // ...
            })
        })
        .collect();
    Json(json!({ "object": "list", "data": data })).into_response()
}

How to read it: start at state.router.config().groups.keys() — the name of every [groups.X] table in your config becomes one group/X, dressed in OpenAI's list format. The .map(...).collect() in between is an iterator pipeline: map reshapes each id through a closure and collect gathers the results back into a Vec — the book teaches these two inline where they appear (the stream.map in stream_reply above is the same move). Why sell only groups? The comment says it: group ids are the stable names an agent configures once, while members, order, and strategy can change freely (Chapter 13) without the agent noticing. Note that this handler skips authorized — it publishes group names and touches no secrets.

Pointing Claude Code and pi at jackdaw

Now the third question from the opening. To any agent that speaks the OpenAI protocol, jackdaw is just a provider, and plugging it in takes two values:

• base URL: http://127.0.0.1:3426/v1 — the port is gateway.port from Chapter 11's config;
• API key: the value of $JACKDAW_TOKEN — that is, the value of the environment variable auth_token_env points at. It never travels to any upstream; its only job is getting past authorized.

For pi: its model and provider configuration lives in ~/.pi/ai.json, and it supports custom providers (baseUrl + apiKey) — fill in the same two lines, and as far as pi can tell, jackdaw is a provider. For Claude Code: the same two edits, base URL and API key; check its current docs for the exact field names — this chapter won't assert them.

TIP

Self-check after configuring: curl -s 127.0.0.1:3426/v1/models should list all your group/<id>s, then one chat request (the curl above) should come back 200 with an x-jackdaw-model header.

Compare with magpie: it writes your chosen model into each agent's config file for you. jackdaw deliberately doesn't — two manual edits buy you the certainty of seeing exactly what your agent reads. That's another entry on the "magpie has it, jackdaw doesn't" list, and it's a teaching point, not a shortfall.

Summary

• The gateway's job is deliberately narrow: check the token, check the JSON, hand off to the router, translate back into HTTP; all upstream semantics live in Chapter 13's router.
• Extractors pull from the request in order, and a body-consuming Bytes must go last; requested is copied out before request is moved away.
• SSE passthrough: errors become SSE events, so the stream itself never fails; no failover after the first byte.
• x-jackdaw-model names the model that actually answered, every time.
• Next chapter (Chapter 15) buys this gateway insurance: a local stub provider, integration tests, and CI.

Self-Check

1. Migration: you want the gateway to accept both Authorization: Bearer … and x-api-key: …. Which function changes? Does the meaning of its return value change?
2. Predict the output: the token is correct, but the body is the valid-JSON array [1,2,3]. What status code comes back, and which error message?
3. Explain: why does stream_reply convert upstream errors into Ok(sse_error_event(...)) instead of letting the Err escape?

Answers

1. authorized. Today it reads only the Authorization header; supporting a second header means also trying x-api-key and returning false only when neither header presents the right value. The return value keeps its meaning — "does this request pass" — only the decision logic changes; the 401 message in chat_completions should also mention both ways of authenticating, to stay actionable.
2. 400. It takes the second branch of decode_request: value.is_object() is false, and the message returned is exactly "request body must be a JSON object".
3. Once the response body has started, there is no chance to change the status code or send an ordinary error response; letting the Err escape would cut the stream mid-answer with no explanation. Converting it into one data: event delivers the failure inside the protocol, and the stream then ends normally — an honest failure.