VALUE LANDSCAPE

A field instrument for the Bellman equation — paint a world, run synchronous value iteration, and watch the terrain and its policy settle.

V(s) = max_a Σ_s′ P(s′|s,a) · [ R(s,a,s′) + γ·V(s′) ]

Sweep0 Max Δ— Converged

−0.05 0 +0.05

Parameters

Discount γ0.90

Step cost−0.04

Slip noise0.20

Run speedmid

Bellman backup

Select the Inspect tool, then click any open cell to see its backup computed live.

How to read it

Color is each state's current value — blue positive, red negative, gray near zero. The arrow is its current greedy policy. Both are recomputed from scratch every sweep, so editing the world mid-run reshapes the landscape in front of you.

SYNCHRONOUS VALUE ITERATION · STOCHASTIC GRIDWORLD (1−NOISE INTENDED, NOISE⁄2 EACH SIDE) · TERMINAL CELLS ABSORB REWARD, NO FURTHER DISCOUNTING