Class Optimizer
java.lang.Object
ai.beestream.jitnn.nn.optim.Optimizer
Updates parameters in place from their gradients.
-
Field Summary
Fields -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionclipGradNorm(float maxNorm) Clips the gradients of this optimiser's parameters to a global L2 norm of at mostmaxNormat the nextstep(), as PyTorch'sclip_grad_norm_, and returns the norm before clipping (after loss unscaling).floatThe current learning rate.protected voidRestores whatsaveScalars(StateDict)saved.voidRestoresstateDict()into this optimiser, in place; type and state shapes must match.protected voidScalar state beyond the step count and learning rate, for checkpoints.voidsetLearningRate(float lr) Changes the learning rate; takes effect at the next step without recompiling.state()Per-parameter state by slot name, each array indexed like the parameters (null where none).The optimiser's state for a checkpoint: its type, learning rate, step count and per-parameter state (state.<slot>.<parameter index>), which refers to the live tensors.abstract voidstep()Records one update of every parameter that has a gradient.protected TensorThe statistics the next update uses (computed now when a pre-scale is pending), or null; clears them.voidzeroGrad()Clears the gradients of every parameter.
-
Field Details
-
params
-
lr
protected float lrThe learning rate. -
t
protected int tSteps taken (the bias corrections of the adaptive methods use it).
-
-
Constructor Details
-
Optimizer
-
-
Method Details
-
step
public abstract void step()Records one update of every parameter that has a gradient. -
clipGradNorm
Clips the gradients of this optimiser's parameters to a global L2 norm of at mostmaxNormat the nextstep(), as PyTorch'sclip_grad_norm_, and returns the norm before clipping (after loss unscaling). One small reduction per gradient; the gradients are scaled inside the update itself. Reading the norm is a sync point. -
takeStats
The statistics the next update uses (computed now when a pre-scale is pending), or null; clears them. -
zeroGrad
public void zeroGrad()Clears the gradients of every parameter. -
learningRate
public float learningRate()The current learning rate. -
setLearningRate
public void setLearningRate(float lr) Changes the learning rate; takes effect at the next step without recompiling. -
state
-
saveScalars
Scalar state beyond the step count and learning rate, for checkpoints. -
loadScalars
Restores whatsaveScalars(StateDict)saved. -
stateDict
The optimiser's state for a checkpoint: its type, learning rate, step count and per-parameter state (state.<slot>.<parameter index>), which refers to the live tensors. The hyper-parameters are not included: build the optimiser with the same arguments before loading. -
loadStateDict
RestoresstateDict()into this optimiser, in place; type and state shapes must match.
-