Rogue Reinforcements

Resizing starts a new floor right away.

Vision: both brains receive 8 compass rays; each reports normalized wall distance plus player, stairs, enemy, and potion visibility, and a 46th input carries the player’s HP (hp/3). v6 doubles the hidden layers (50→32→24→5); v5 added 4 pathfinding inputs — the BFS distance to the goal (stairs for the player, the player for enemies) from each neighboring tile, so both brains can see around corners. Rewards (all tunable under rewards above): each enemy scores big for damage it deals itself, the team scores for killing @, loses a little when @ escapes or when one of them is slain (cannon fodder), and gets small per-turn shaping toward the guard point (waiting costs a little). The player scores for reaching the stairs — plus a per-depth bonus, so deeper floors pay more — is penalized for dying and for each HP lost, and earns small per-turn shaping toward the stairs plus bonuses for potions, bump hits/kills, and opportunity hits/kills. Reaching the stairs descends to the next floor within the same run; only death or level collapse ends the run and restarts at floor 1. Action selection uses each brain’s Q-values directly (plus ε-greedy exploration) — there is no hand-coded steering at decision time. @ has 3 HP; anything that removes HP teaches the same avoidance. 1–3 potions (!) spawn per floor; stepping on one restores 1 HP (only when hurt — enemies ignore them). Leaving a tile next to an enemy provokes an opportunity attack: each adjacent enemy deals 1 damage. Moving into an enemy is a bump attack instead: @ stays put, the enemy loses 1 HP, and attacking never provokes. An enemy that steps away from @ takes the same opportunity attack from @, unless @ is already down. Enemies have 2 HP (shown as e at 1 HP); a slain enemy is removed at once and a reinforcement with full HP spawns 25 player turns later at least 6 tiles from @ (the live count never exceeds the enemy setting). Enemy HP is not a network input yet (future work). Each enemy adds one transition per turn to the shared brain.

Labs (separate from these rewards): the demo lab Gridseek has its own tunable reward terms in the Labs workbench. closerReward is a small training-time shaping bonus for a step that moves closer to the goal (0.02, versus 1 for reaching it); it is learned into Q-values, never added at decision time. Its stat is the greedy success rate on fixed evaluation scenarios: shaping that misfires shows up as training success without greedy success. The Composition lab has one shaping term, approachReward: a reward per tile of walking distance gained toward the objective, as a potential difference (it telescopes, so stepping back and forth earns nothing). It is learned into the router's Q-values, never added at decision time; shapingShare shows it misfiring. The Tracking room's progress nudge is two-sided the same way: closerReward per tile gained (0.1) and fartherReward per tile lost (−0.1), so a loop away and back nets zero; shapingShare watches both.

Coaching: pick a side above the dungeon. Coaching @, you move @ and every move you make is a demonstration for the player brain. Coaching E, you steer one enemy (highlighted in purple) while the player brain drives @ - @ moves first, then you answer. If your E falls, the game plays itself until you respawn as the next reinforcement; those turns add no demonstrations. A demonstration is an ordinary replay transition scored by the same rewards as the brain's own moves: Q-learning is off-policy, so it learns from moves it did not choose. Nothing is steered at decision time - press ▶ auto and the brain plays with only what it learned. agrees shows how often the coached brain's top choice already matches your move (last 100 moves); your demos in replay shows how much of its memory is you. The 👍/👎 buttons add ±1 to each brain's final move of the run that just ended.

Upgrade shop (⚡ next to the coach toggle): the side you coach earns points - @ gets 1 per stairs reached, E gets 1 per point of damage dealt to @. Points are not a reward the networks see; they only buy upgrades for the coached brain. Danger sense adds 9 inputs (enemies next to this tile and each neighbor) with zero weights, so the brain acts the same until it learns to use them; its replay memory is cleared, since it was recorded without the sense. Extra layer inserts a 24-neuron tanh layer before the outputs as 0.25 x identity with the output weights scaled by 4, so the brain's Q-values barely move when it is installed. Upgrades are saved with the brain; "reset both brains" refunds them.

Bow and composed brains (#115): the Bow is a shop purchase that only unlocks its exercise in the Training Room; it changes neither brain. Firing exists only in a composed brain: a certified router (Composition, with the Bow skill and optionally your game brain) installed as the @ brain from the Review screen or the Brain panel. Its outputs are the 8 moves, wait and fire; fire is legal when the bow is ready (1 turn reload) and the Bow skill's rules find a target in range with a clear line, and the target is picked by the router's learned target stage, never by a rule. F opens the aim phase (Tab / arrows cycle, Enter or F fires, Esc cancels; no time passes until the shot); each committed shot is a demonstration that trains the router's action and target stages. Rewards: bow hit per damage dealt (0.3), bow kill (0.5), wasted shot (-0.05), all under rewards above; the bow shots / hit rate / wasted shots stats show them misfiring. While it plays, the router and the game's own copies of its skills keep learning (the skills at a quarter of the router's step; the certified results in the Training Room never change, and the Brain panel shows how far each copy has drifted). Uninstalling restores the previous brain.

Network backend (#52): the "Burn (wasm) network backend" toggle above (or ?nn=burn) swaps the Q-network maths (forward pass, target network, per-sample SGD) from hand-written JS to a Rust/Burn module compiled to wasm. Same rules, same rewards, same replay buffer and RNG streams either way — only the matrix maths differ, and both read/write the same {w,b} brain format, so switching backends never resets or migrates a brain. The default stays JS; if the wasm module fails to load, it falls back to JS automatically with a notice by the toggle. Both backends build the network itself from one declarative shape, the RogueAI network schema (rogueai-net/1, see docs/rogueai-net.md) - changing layer sizes or activations is a spec edit, not a code change.