Friday: game awareness spec

Ground truth for the plan: how the setup should behave. Updated 2026-10-19. Status: design, not built. Replaces the architecture in research/07.

Pipeline

codeLunaSol

Code does all the routine seeing. Luna reads words and thinks; it sees an image only when code is unsure or Friday asks.

Ledger and World

Ledger: the recordEvery fact anyone reports, whether seen, read, heard or guessed, plus every chat and voice line, with its game time, source and confidence. Only ever added to.

World: the pictureOne entry per champion, objective and camp, plus the last few minutes of talk, rebuilt from the ledger on every new fact: latest values, duplicates merged, conflicts settled, old guesses dropped. Friday, triggers and the Analyst read only this.

Like a bank statement and its balance. Example, 9:38 to 10:00:

Ledger, by game time

9:38	minimap	Nocturne seen, their raptors	.97
9:41	voice	John: "Cho flashed"	line
9:41	analyst	Cho'Gath Flash used, from John	.7
9:43	friday	Cho'Gath Flash used, you told it	.9
9:44	chat	Cho'Gath Flash back 14:41	1
9:46	minimap	Nocturne seen, top river	.95
9:52	api	Nocturne level 6	1
9:58	analyst	Nocturne likely in our top jungle, until 10:18	.6

→

World at 10:00

Nocturnelevel 6, seen top river 14 s ago
trail: their raptors 9:38, top river 9:46
likely our top jungle, guess, until 10:18

Cho'GathFlash down until 14:41, 3 sources, sure

What Friday reads

Nocturne L6: top river, 14 s ago
  from their raptors 9:38
  likely our top jungle (guess)
Cho'Gath: no Flash until 14:41

Writers only add entries, so code readers, the Analyst and Friday can't overwrite each other. One set of rules builds the World.
Facts arrive late and out of order: John's 9:41 line becomes a fact when the Analyst runs. Ordering by game time puts it in place.
History comes free: trails, "unseen for" and "Flash used at" are read off the ledger.
Every World value knows the entries behind it, so a wrong call can be traced. Replays rebuild the World from a logged ledger, so rules can be tuned offline.

One minute of activity

How fresh the World is

From a minimap blip to Friday speaking. Targets, to measure once built.

0.5 s

Sol + voice ~3.5 s

Positions reach the World within half a second of showing on the minimap.
The Analyst's guesses about unseen enemies lag ~10 s. They're guesses, so that's fine.

Who writes what

Store rules

The ledger is append-only. Nothing is edited or deleted.
One writer process, the coach. Readers and agents return facts; the coach appends them.
Order and staleness use t, the game time of the evidence, not arrival.
Same fact from several sources merges: sources unioned, confidence up.
Conflicts follow the ladder. A fresh fact beats an expired one. Unresolved conflicts are shown to Friday.
Corrections reference what they fix (corrects). Dedupe by key.

Precedence, by evidence

api

screen

chat

voice

inferred

// ledger entry
{ seq, t, source, kind, subject, data, conf, evidence, key?, corrects?, expires? }
// subject  "enemy:Chogath" | "ally:Brand" | "obj:dragon" | "camp:their-raptors"

// World champion (all 10)
{ team, role, level, alive, respawnAt?, vitals?{hp,mana,t}, ult,
  spells{name: ready | {backAt,sources} | unknown},
  seen?{zone,x,y,t}, trail[{t,zone}], likely?{zone,conf,why,t} }

Minimap detector

Code, every frame (2/s). No model.

Native crop: 320 px, icons ~25 px. No upscaling.
Templates: the 10 champions' minimap circles for their skins (skin from the API, files from CommunityDragon), masked round, scaled to icon size.
Find red and blue rings; match each against its team's 5 champions. A 1-of-5 choice.
Pixel to named zone through a hand-drawn zone map: lanes, river, camps, bases. Camps and epic monsters: icon state at fixed spots.
Count check: blue rings found = allies alive (API). A shortfall means stacked icons.
Unsure (stacked icons, weak match): keep the identity seen there 0.5 s ago, else ask Look.
Pass bar, ~50 labelled frames: ≥95% of icons right, no invented enemies.

Friday

The only Sol agent. Proactive on triggers, reactive on questions.

Per-call part of the prompt, target tokens

World render 1.8k

talk 0.5k

events

Talk: last 2 min. Events: last 3 min. Last segment: the trigger.

World render: ages on everything, everyone's trails for the last 3 min.
Writes with record({kind, subject, data, say?}); the turn ends after the call. say only when addressed directly.
Images only through Look. If Sol is unavailable, triggers speak plain lines.

Analyst

Luna, text only. The only routine model call besides Friday.

Runs every ~10 s if the World changed, and within 3 s of a new chat or voice line.
In: World render, new lines, its last guesses. No images.
Out: facts from words ("Cho flashed", "going drake"), where unseen enemies probably are, the plan, worth a call.

Look

Luna with a small image. Rare.

The detector's unsure spot: a ~64 px crop with the candidate icons. Answer: which champions, how many.
Friday's question about the screen: one downscaled frame.

Triggers

Code, on every World change. The periodic check stays as a backstop.

enemy jungler on our side, near you, near our objective objective ≤60 s + lane priority + who's alive you in their jungle, ≥2 enemies missing fight startedfight ended kill window: lane enemy's Flash down, jungler near summoner or ult back soon analyst: worth a call

Model calls per 30-minute game

Code: capture, minimap detector, ally frames, chat OCR, speech-to-text, game API, World, triggers.
Luna: Analyst and Look. Medium effort, standard tier, never Fast.
Sol: Friday only, for both proactive lines and answers.
Images per game: ~30 small crops.
Left out: your own HUD. You know your cooldowns.
No rationing: run until the 5-hour window is empty, then triggers speak plain lines. Cost per call from token counts (codex.call).

Build order

1Record frames

2Ledger, World, record

3Minimap detector, labelled test

4Triggers, Friday on the World

5Analyst

6Ally frames, chat OCR

7Look

Open

Stacked icons: how often the detector is unsure, and whether "group of 3 near drake" is enough without Look.
Skins without their own minimap circle: check which icon they use on recordings.
Chat format for summoner timers posted from Tab: needs a screenshot.
Fast for Friday's direct answers: keep or drop?
When the 5-hour window is empty: spend credits or stop?

Game settings

chat timestamps on 2048×1152 fullscreen minimap scale 1.0, not flipped (calibrated in src/capture/minimap.json)

Source: research/09-awareness-agents.html in the repo. Publish with research/deploy-spec.sh.