Adam Optimizer: A Visual Walkthrough

Step through the process in N dimensions.

Loss Landscape Projections

Dimensions:

Press "Step" to begin. Adjust hyperparameters or dimensions and hit "Restart" to see their effect.

Learning Rate (α)

Beta 1 (β₁)

Beta 2 (β₂)

Epsilon (ε)

Weights (w)

Gradient (g)

1st Moment (m)

2nd Moment (v)

Bias-corrected m (m̂)

Bias-corrected v (v̂)

Step 0