Techniqueadvanced
Layer-wise adaptive rate scaling
Also called LARS.
Scales each layer's learning rate by the ratio of weight norm to gradient norm, for large-batch training.
Scales each layer's learning rate by the ratio of weight norm to gradient norm, for large-batch training.
About this page
The build computes these levels from the evidence in the repository. Each row says what the next level needs. Read what the levels mean.
| Ladder | Level | What the next level needs |
|---|---|---|
| Human proof | H0 no proof |
|
| Lean | L0 no Lean part |
|
| Visual | V0 none |
|
| Writing | W0 stub |
|
Tag ZN48H9 Edit this page History Report an error