Uses of Class
ai.beestream.jitnn.nn.optim.LRScheduler
Packages that use LRScheduler
Package
Description
Optimisers with PyTorch's semantics (SGD, Adam and AdamW, AdaGrad, RMSProp, AdaDelta, AdaMax,
NAdam, LAMB, LARS), learning-rate schedules (
LRScheduler) and dynamic
loss scaling for FP16 training (GradScaler).-
Uses of LRScheduler in ai.beestream.jitnn.nn.optim
Methods in ai.beestream.jitnn.nn.optim that return LRSchedulerModifier and TypeMethodDescriptionstatic LRSchedulerCosine annealing from base tominLrovertMaxepochs (PyTorch's closed form).static LRSchedulerlr = base * gamma^m where m counts the milestones passed.static LRSchedulerlr = base * gamma^(epoch / stepSize).static LRSchedulerLRScheduler.warmupCosine(Optimizer optimizer, int warmup, int total, float minRatio) Linear warm-up overwarmupsteps, then a half-cosine decay tominRatiotimes the base rate attotalsteps (Hugging Face's get_cosine_schedule_with_warmup when minRatio is 0).static LRSchedulerLRScheduler.warmupLinear(Optimizer optimizer, int warmup, int total) Linear warm-up from 0 to the base rate overwarmupsteps, then linear decay to 0 attotalsteps (Hugging Face's get_linear_schedule_with_warmup).