A Field Guide to Modern AI // draft 01
The Author
in the Machine
How Claude and ChatGPT actually work — tokens, memory, tools, and retrieval — and what it means for the systems we build.
The map
Six ideas, one mental model
01 · Tokens
It doesn't read words. It reads tokens.
Text is chopped into little chunks — roughly ¾ of a word each — including spaces and punctuation. To the model, every chunk is just a number.
A tidy sentence — here every chunk happens to be a whole word (plus the full stop):
A rarer word and a number — sliced into fragments that aren't words at all:
01 · Token accounting
Two meters: what goes in, what comes out
Everything you send is input; everything it writes back is output. You pay — and hit limits — on both.
The catch: it re-reads the entire input every single turn. It has no memory between turns — only what's on the desk.
02 · The context window
Think of it as the model's desk
Everything it can see at once has to fit on one finite desk. Today that desk is large — ~200k tokens, about a 500-page book — but it is fixed.
02 · When the desk fills up
Why it "forgets" — and can't swallow the Handbook
Long conversations
The earliest messages slide off the desk to make room. It genuinely "forgets" how the chat began.
Big documents
The full FCA Handbook is millions of tokens. It will never fit on the desk all at once.
Cost & focus
More on the desk means slower, pricier answers — and more chances to lose the thread.
So we choose
The skill is deciding what earns a place on the desk for each question. Hold that thought.
03 · The author
The model is not the chatbot.
It's the author writing the chatbot's next line.
03 · The author
What's really happening: it completes a script
"Claude" and "ChatGPT" are characters in this script. The model's one job is to predict the next token of the Assistant line — over and over.
03 · Why it matters
How to think when you talk to it
It's steerable
Set the scene in the system prompt and you shape the character. Clear framing = a better author.
No memory between stories
Close the chat and it forgets. Only what's on the desk right now persists — nothing else.
It always continues
Even past what it knows. A fluent-but-wrong continuation is what we call a "hallucination".
No live world — by default
It only sees the words on the desk. It can't check today's facts… yet. (That's next.)
04 · Training
How the author learned to write
Before it ever met you, it read an enormous slice of text and tuned billions of internal dials to get better at one game: predict the next token.
the public web
tune the "weights"
the author's instincts
Weights = long-term memory (baked in, permanent). Context window = working memory (just for now).
04 · The cutoff
Trained once, then frozen in time
Training stops on a date — the knowledge cutoff. After it, the author knows nothing new: not today's news, not last week's FCA update, and not one word of your internal policies — it never saw them.
05 · Tools
Give the author a phone and a library
A tool is permission to act outside the text. The simplest is web search — and it sidesteps the frozen cutoff entirely.
look this up"
search()
updates
results
placed in context
grounded answer
05 · Retrieval — RAG
When the library is too big — or private
You can't paste the whole Handbook (slide 6). So: index your documents, fetch only the few relevant passages, drop them on the desk, then let the author write — grounded in them. That's Retrieval-Augmented Generation, the simplest useful agent.
COBS 9?"
indexed docs
top passages
relevant pages
answer + cites
05 · Our use case
This is the spine of the controls-gap tool
+ policy docs
retrieve
ideal controls
compare
that actually exist
& weak spots
Stage 1 — ideal state
Regulations → the controls that should exist: type, frequency, ownership, evidence.
Stage 2 — gap analysis
Ideal vs actual → what's missing, weak, or left to interpretation.
06 · Agents
Let the author act, look, and decide — in a loop
Give it several tools and let it choose: search a database, call an API, run a calculation, retrieve a doc. It works in a loop until the task is done.
need next?
an API
on the desk
enough to write
06 · The product
So what is "ChatGPT"? The theatre around the author
The chatbot you talk to is the author wrapped in a system: a prompt that sets the character, conversation memory, retrieval over documents, a toolbox, an orchestration loop, and a friendly UI.
Same author at the centre. The product is everything we build around the desk.
The takeaway