What an Agent Is: The pi Worldview

In chapter 1 you watched a demo of the finished jackdaw: a request leaves curl, passes through a gateway, reaches a provider, and streams back. We promised to skip the details then. Now we take it apart. By the end you can answer:

• Given the JSON of a chat request, point to the system prompt, say what burns tokens, and explain why the entire history must fit into the context window;
• When someone says "an agent is just a loop," expand that into five steps and say what gets added to the JSON each time the loop turns;
• When you hear the names pi, magpie, and jackdaw, place each one at its layer in the chain.

Anatomy of one request

First, the through-line: while an agent works, what it POSTs to some address, over and over, is JSON shaped like this (an OpenAI-compatible chat endpoint; every JSON in this chapter keeps only the skeleton fields — real requests carry more):

{
  "model": "openrouter/qwen3-coder",
  "messages": [
    { "role": "system", "content": "You are a careful Rust code reviewer. Be concise." },
    { "role": "user", "content": "帮我看 src/router.rs 有没有问题" },
    { "role": "assistant", "content": "先告诉我文件里有哪些函数。" },
    { "role": "user", "content": "resolve 和 pick_next 两个。" }
  ],
  "stream": true
}

How to read it: start at model — it names which model this call wants. The provider/model slash naming comes from magpie; we pick it back up in the last section of this chapter. messages is the conversation in time order; each entry has a role (system / user / assistant) and a content. "stream": true is a switch — hold that thought until streaming.

The key insight: the model has no memory. Every call is stateless. The only way to make the model "remember" earlier turns is to resend the entire history, as is. This array is the messages — it is not a log, it is the model's entire input.

An agent is a loop

The short answer first: an agent is a program that calls a model inside a loop — send messages, run tools, feed the results back, repeat until the task is done.

What happens without the loop? A bare API call is one question, one answer: the model cannot read your files or run your commands. So you need a program to run its errands and manage the loop and its state — that runtime shell is called the harness.

The five steps:

1. Assemble messages (system + history + new instruction) and send them to the model.
2. The model returns either plain text or a tool call — one structured sentence: "run read_file for me, with src/router.rs."
3. The harness actually executes it.
4. The result goes back in as a new role: "tool" message.
5. Resend the now-longer messages, until the model stops asking for tools.

Steps 2 and 4 look like this:

{ "role": "assistant", "content": null,
  "tool_calls": [
    { "id": "call_01", "type": "function",
      "function": { "name": "read_file",
                    "arguments": "{\"path\": \"src/router.rs\"}" } } ] }

{ "role": "tool", "tool_call_id": "call_01",
  "content": "fn resolve(...) { /* ... */ }" }

How to read them: the assistant entry has content: null — this turn it says nothing, it only asks for a tool; the reply matches it via tool_call_id. Note what touches your disk: the harness, not the model. The model only outputs what it wants called.

That is exactly pi's worldview. pi calls itself "Pi Agent Harness" — a self-extensible family of coding agent tools, layered like this: pi-coding-agent (interactive CLI), pi-agent-core (the agent runtime: tool calling and state management), pi-ai (one unified multi-provider LLM API). The harness is the shell; the agent is the loop turning inside it. Worth knowing early: pi ships no built-in permission system and runs with your user permissions — transparency and auditability are part of its philosophy.

The model's raw material: tokens, context window, system prompt

Each turn of the loop makes messages longer. The bill comes in three words.

A token is the unit the model processes text in — roughly a word, or a piece of one. Billing and context are both counted in tokens.

The context window is the cap on how many tokens the model can see at once. System prompt, every turn of conversation, every tool result: the total must fit. When a long chat starts to "forget," it is not bad memory — whatever didn't fit was never sent.

The system prompt is the role: "system" entry: a hidden instruction that sets the model's role and behavior. Users usually never see it, yet it occupies the window every single turn — our through-line opens with one.

The return trip: streaming and providers

Time to make good on "stream": true. A full answer can run to hundreds of tokens; if the server waits for all of it before replying, the first character takes seconds to arrive. Streaming (SSE, Server-Sent Events) lets the server generate one chunk, push one chunk — no waiting for the full answer. The response is not one big JSON — it is a series of events:

data: {"choices":[{"delta":{"content":"fn"}}]}
data: {"choices":[{"delta":{"content":" resolve"}}]}
data: [DONE]

How to read it: each data: line is one event; delta holds the few characters added in this step; concatenate them all to get the answer, and [DONE] closes it (real events carry id and other fields, omitted here). We take the byte level apart in chapter 12.

One last term: provider. It means two things: the party that offers a model API (OpenAI, Anthropic, Google, ...), and, in code, the unified abstraction — each vendor's API has a different shape, and a provider flattens them into one interface the agent talks to. pi's pi-ai layer is exactly that.

One layer in between: the gateway topology and jackdaw

The short answer: insert a long-running local forwarding service between agent and provider — a gateway — and the chain becomes agent → gateway → provider.

magpie is such a local gateway plus model manager: it listens on 127.0.0.1:3425 by default and speaks the OpenAI, Anthropic, and Gemini protocols, translating between them (streaming and tool calls included). Its flagship feature is the routing group: group/<id> packs several models and accounts into one name, with a strategy and failover — when the upstream dies, the next one takes over. Chapter 13 opens this up.

How does this layer plug in? Two facts interlock: pi supports custom providers — give it a baseUrl and an apiKey, and that is all it needs; and magpie speaks the OpenAI-compatible protocol on the outside. So the "one provider" the agent sees is actually a gateway — the slash name in our through-line's model field is written for a gateway to read. magpie calls tools like Claude Code, Codex, and pi "Agents," and writes your chosen model into each agent's config file. The agent never notices.

jackdaw plays exactly that small role, deliberately cut down to a minimal core: OpenAI-compatible protocol in and out, hand-configured providers (no preset catalog), and group/<id> with only the order / rotate strategies plus failover. magpie also has smart / usage strategies, intent-based routing rules, and group nesting up to eight levels deep — jackdaw does none of it. This book needs only the core you can read; what got cut is a teaching point, not a defect. From chapter 10 on, we dissect it layer by layer.

Summary

Back to the opening questions: the system prompt is the role: "system" entry; the whole messages array burns tokens and must fit into the context window; an agent is the five-step loop, and each turn leaves the messages longer. Layer by layer: pi is the harness, magpie is the gateway in the middle, and jackdaw is the gateway you will read yourself. Next chapter sets the division of labor: AI writes, you read.

Self-Check

1. Transfer: map one of your long conversations (any chat product) onto messages — which entry is probably the system prompt? Can you see it? How does "forgetting" late in a long chat play out in this chapter's terms?
2. Predict: the model returned a tool call last turn. The program executes nothing, back-fills nothing, and resends the same messages. What happens next, and why?
3. Explain: why can pi treat magpie as an ordinary provider? Which two facts does it rest on?

Answers

1. The system prompt is usually the first entry, injected by the product; the interface generally hides it. Your inputs are role: "user", the model's replies role: "assistant". "Forgetting" happens because the context window caps tokens — early content that does not fit is simply never sent. The model didn't forget; it never saw it.
2. Most likely the model asks for the same tool again: its input still lacks the tool result, and the loop spins in place. That is why the "feed the result back" step exists.
3. First, pi supports custom providers — only a baseUrl and an apiKey. Second, magpie speaks the OpenAI-compatible protocol on the outside. To pi, a gateway is indistinguishable from an ordinary provider; translation, routing, and failover all happen inside the gateway.