Adam Optimizer: A Visual Walkthrough

Step through the process to see how Adam combines Momentum and RMSProp.

Loss Landscape Visualization

Press "Forward" to begin the visualization. Adjust the hyperparameters below and hit "Restart" to see their effect.

Learning Rate (α)

Beta 1 (β₁)

Beta 2 (β₂)

Epsilon (ε)

Gradient (g)

[0.00, 0.00]

1st Moment (m)

[0.00, 0.00]

2nd Moment (v)

[0.00, 0.00]

Weights (w)

[0.00, 0.00]

Bias-corrected m (m̂)

[0.00, 0.00]

Bias-corrected v (v̂)

[0.00, 0.00]

Step 0