Friday: game awareness spec

Ground truth for the plan: how the setup should behave. Updated 2026-10-19. Status: built (src/awareness), offline-tested on a recorded game; next is a live game. Replaces the architecture in research/07.

Pipeline

codeLunaSol

Code does all the routine seeing. Luna reads words and thinks; it sees an image only when code is unsure or Friday asks.

Ledger and World

Ledger: the recordEvery fact anyone reports, whether seen, read, heard or guessed, plus every chat and voice line, with its game time, source and confidence. Only ever added to.

World: the pictureOne entry per champion, objective and camp, plus the last few minutes of talk, rebuilt from the ledger on every new fact: latest values, duplicates merged, conflicts settled, old guesses dropped. Friday, triggers and the Analyst read only this.

Like a bank statement and its balance. Example, 9:38 to 10:00:

Ledger, by game time

9:38	minimap	Nocturne seen, their raptors	.97
9:41	voice	John: "Cho flashed"	line
9:41	analyst	Cho'Gath Flash used, from John	.7
9:43	friday	Cho'Gath Flash used, you told it	.9
9:44	chat	Cho'Gath Flash back 14:41	1
9:46	minimap	Nocturne seen, top river	.95
9:52	api	Nocturne level 6	1
9:58	analyst	Nocturne likely in our top jungle, until 10:18	.6

→

World at 10:00

Nocturnelevel 6, seen top river 14 s ago
trail: their raptors 9:38, top river 9:46
likely our top jungle, guess, until 10:18

Cho'GathFlash down until 14:41, 3 sources, sure

What Friday reads

Nocturne L6: top river, 14 s ago
  from their raptors 9:38
  likely our top jungle (guess)
Cho'Gath: no Flash until 14:41

Writers only add entries, so code readers, the Analyst and Friday can't overwrite each other. One set of rules builds the World.
Facts arrive late and out of order: John's 9:41 line becomes a fact when the Analyst runs. Ordering by game time puts it in place.
History comes free: trails, "unseen for" and "Flash used at" are read off the ledger.
Every World value knows the entries behind it, so a wrong call can be traced. Replays rebuild the World from a logged ledger, so rules can be tuned offline.

One minute of activity

How fresh the World is

From a minimap blip to Friday speaking. Targets, to measure once built.

0.5 s

Sol + voice ~3.5 s

Positions reach the World within half a second of showing on the minimap.
The Analyst's guesses about unseen enemies lag ~10 s. They're guesses, so that's fine.

Who writes what

Store rules

The ledger is append-only. Nothing is edited or deleted.
One writer process, the coach. Readers and agents return facts; the coach appends them.
Order and staleness use t, the game time of the evidence, not arrival.
Same fact from several sources merges: sources unioned, confidence up.
Conflicts follow the ladder. A fresh fact beats an expired one. Unresolved conflicts are shown to Friday.
Corrections reference what they fix (corrects). Dedupe by key.

Precedence, by evidence

api

screen

chat

voice

inferred

// ledger entry
{ seq, t, source, kind, subject, data, conf, evidence, key?, corrects?, expires? }
// subject  "enemy:Chogath" | "ally:Brand" | "obj:dragon" | "camp:their-raptors"

// World champion (all 10)
{ team, role, level, alive, respawnAt?, vitals?{hp,mana,t}, ult,
  spells{name: ready | {backAt,sources} | unknown},
  seen?{zone,x,y,t}, trail[{t,zone}], likely?{zone,conf,why,t} }

Minimap detector

Code, every frame (2/s). No model.

Native crop: 320 px; icons 26 px, a 2 px ring over ~24 px of portrait. No upscaling.
Templates: each champion's iconCircle from its skin's data (CommunityDragon skinN.bin.json). Many skins reuse the base circle.
Match the 10 portraits: correlation over the inner disc, coarse at half size, refined in colour. Each champion placed once. ~40 ms a frame.
Against false hits: the team's ring colour (blue allies, red enemies) unless the portrait match is strong; the patch must not look like the empty map (median of a game's frames); a weak match must beat the other champions clearly.
Tracking: each champion is searched near its spot 0.5 s ago with a lower bar (allies only; enemies vanish into fog, so they keep the ring rule). One that vanishes under another icon stays there as stacked (up to 8 s; Yuumi follows her ally).
Capture: League's own window, so another window over the corner can't hide the minimap. Frames that still aren't the minimap are dropped by comparing with a reference map.
Places: map units from the pixel, then lane, river, jungle quarter or base, and the camp or pit within ~15 px, said from our side ("their raptors", "dragon pit").
Pass bar, ~50 labelled frames: ≥95% of icons right, no invented enemies. Met on the first game: 52 frames reviewed by eye, 268 visible icons, 97.4% right, 0 invented enemies, 2 wrong ally placements. The labels started from the detector's output, so a second game should confirm.
Same game against the API (allies, 333 frames): found 91%, found or stacked 95%.
Next: camp and epic-monster state from the icons at fixed spots.

Friday

The only Sol agent. Proactive on triggers, reactive on questions.

Per-call part of the prompt, target tokens

World render 1.8k

talk 0.5k

events

Talk: last 2 min. Events: last 3 min. Last segment: the trigger.

World render: ages on everything, everyone's trails for the last 3 min.
Writes with a record list in its JSON reply (spell used or back, a reported sighting, an intent, a correction of an [L57] fact), next to say. say stays null unless worth saying.
Images only through Look: no minimap image when [MAP] is in the brief; a look_at_screen tool for questions about the screen.
If Sol is unavailable, triggers speak plain lines.

Analyst

Luna, text only. The only routine model call besides Friday.

Runs every ~10 s if the World changed, and within 3 s of a new chat or voice line.
In: World render, new lines, its last guesses. No images.
Out: facts from words ("Cho flashed", "going drake"), where unseen enemies probably are, the plan, worth a call.
Measured on the recorded game: ~5k tokens in (cached from the second call), 1.7 s warm, 7 s cold; sensible guesses with honest confidences.

Look

Luna with a small image. Rare.

The detector's unsure spot (an enemy hidden under another icon for 2 s): a 64 px crop next to the numbered candidate portraits. Answer: which are visible, and ring counts by colour; a red ring it can't name with one enemy candidate counts as them. ≤40 a game, ≥20 s apart. ~9 s.
Friday's question about the screen: one downscaled frame.

Triggers

Code, on every World change. The periodic check stays as a backstop; objective timers stay with the coach's API triggers. On the recorded game: 40 in 29 minutes, before the coach's rate limits.

enemy jungler on our side, near you, near our objective objective soon (API timers) you in their jungle, ≥2 enemies missing fight startedfight outcome (API kills) kill window: lane enemy's Flash down, jungler near Flash, TP or Ignite back within 30 s analyst: worth a call

Model calls per 30-minute game

Code: capture, minimap detector, ally frames, chat OCR, speech-to-text, game API, World, triggers.
Luna: Analyst and Look. Medium effort, standard tier, never Fast.
Sol: Friday only, for both proactive lines and answers.
Images per game: ~30 small crops.
Left out: your own HUD. You know your cooldowns.
No rationing: run until the 5-hour window is empty, then triggers speak plain lines. Cost per call from token counts (codex.call).

Build order

1Record frames ✓

2Ledger, World, record ✓

3Minimap detector, labelled test ✓

4Triggers, Friday on the World ✓

5Analyst ✓

6Ally frames, chat OCR ✓

7Look ✓

8A live game

Open

Calls per game and the 5-hour window: measure on a live game (two Analyst calls moved the meter ~1%).
Trigger volume and value: which map triggers Friday actually speaks on, and which to drop.
Stacked icons: ~4% of ally sightings in the first game are only "stacked". Is "group of 3 near drake" enough without Look?
Briar-style red halos (and big pings) hide an enemy's ring; the detector then needs a strong portrait match. Watch how often enemies are missed in fights.
Fast for Friday's direct answers: keep or drop?
When the 5-hour window is empty: spend credits or stop?

Game settings

chat timestamps on 2048×1152 fullscreen minimap 50% (MinimapScale 1.5), not flipped: still ~320 px, rect in src/capture/minimap.json Tab spell posts read "16:49 Name (Aphelios): Warwick Flash": no timer, code adds the cooldown to the chat time

Source: research/09-awareness-agents.html in the repo. Publish with research/deploy-spec.sh.