Class LRScheduler
java.lang.Object
ai.beestream.jitnn.nn.optim.LRScheduler
-
Field Summary
Fields -
Constructor Summary
ConstructorsModifierConstructorDescriptionprotectedLRScheduler(Optimizer optimizer) A schedule foroptimizer, starting from its current learning rate. -
Method Summary
Modifier and TypeMethodDescriptionstatic LRSchedulerCosine annealing from base tominLrovertMaxepochs (PyTorch's closed form).floatThe optimiser's current learning rate.voidRestores the count saved bystateDict().static LRSchedulerlr = base * gamma^m where m counts the milestones passed.protected abstract floatrate(int epoch) The learning rate forepoch(0-based).The epoch (or step) count, for checkpoints; the rate itself is restored with the optimiser.voidstep()Advances the count and sets the optimiser's rate for it.static LRSchedulerlr = base * gamma^(epoch / stepSize).static LRSchedulerwarmupCosine(Optimizer optimizer, int warmup, int total, float minRatio) Linear warm-up overwarmupsteps, then a half-cosine decay tominRatiotimes the base rate attotalsteps (Hugging Face's get_cosine_schedule_with_warmup when minRatio is 0).static LRSchedulerwarmupLinear(Optimizer optimizer, int warmup, int total) Linear warm-up from 0 to the base rate overwarmupsteps, then linear decay to 0 attotalsteps (Hugging Face's get_linear_schedule_with_warmup).
-
Field Details
-
optimizer
The optimiser whose rate this schedule sets. -
baseLr
protected final float baseLrThe optimiser's learning rate when the schedule was created. -
epoch
protected int epochEpochs (or, for warm-up schedules, steps) counted so far.
-
-
Constructor Details
-
LRScheduler
A schedule foroptimizer, starting from its current learning rate.
-
-
Method Details
-
rate
protected abstract float rate(int epoch) The learning rate forepoch(0-based). -
step
public void step()Advances the count and sets the optimiser's rate for it. -
stateDict
The epoch (or step) count, for checkpoints; the rate itself is restored with the optimiser. -
loadStateDict
Restores the count saved bystateDict(). -
currentRate
public float currentRate()The optimiser's current learning rate. -
step
lr = base * gamma^(epoch / stepSize). -
multiStep
lr = base * gamma^m where m counts the milestones passed. -
cosine
Cosine annealing from base tominLrovertMaxepochs (PyTorch's closed form). -
warmupLinear
Linear warm-up from 0 to the base rate overwarmupsteps, then linear decay to 0 attotalsteps (Hugging Face's get_linear_schedule_with_warmup). Sets the step-0 rate now. -
warmupCosine
Linear warm-up overwarmupsteps, then a half-cosine decay tominRatiotimes the base rate attotalsteps (Hugging Face's get_cosine_schedule_with_warmup when minRatio is 0). Sets the step-0 rate now.
-