Uses of Class
ai.beestream.jitnn.nn.optim.Optimizer

Packages that use Optimizer
Package
Description
Optimisers with PyTorch's semantics (SGD, Adam and AdamW, AdaGrad, RMSProp, AdaDelta, AdaMax, NAdam, LAMB, LARS), learning-rate schedules (LRScheduler) and dynamic loss scaling for FP16 training (GradScaler).
  • Uses of Optimizer in ai.beestream.jitnn.nn.optim

    Modifier and Type
    Class
    Description
    final class 
    AdaDelta (Zeiler), PyTorch's semantics: decay rho, eps, learning rate (default 1), L2 weight decay.
    final class 
    AdaGrad (Duchi et al.), PyTorch's semantics: learning-rate decay, L2 weight decay, initial accumulator.
    final class 
    Adam (Kingma and Ba) with L2 weight decay, or AdamW (decoupled decay) via Adam.adamW(List, float, float); with amsgrad, AMSGrad's running maximum of the second moment (Reddi et al.).
    final class 
    AdaMax (Kingma and Ba): Adam with an infinity norm, PyTorch's semantics, L2 weight decay.
    final class 
    LAMB (You et al., 2020): Adam's bias-corrected update plus decoupled weight decay, scaled per parameter by the trust ratio |p| / |update|.
    final class 
    LARS (You et al., 2017) with momentum: a per-parameter learning rate eta |p| / (|g| + weightDecay |p|), applied to the decayed gradient and accumulated in a momentum buffer.
    final class 
    NAdam (Dozat), PyTorch's semantics: momentum decay psi (default 0.004), L2 weight decay.
    final class 
    RMSProp (Hinton), PyTorch's semantics: smoothing alpha, optional momentum and centring, L2 weight decay.
    final class 
    Stochastic gradient descent with optional (Nesterov) momentum and L2 weight decay, as in PyTorch.
    Modifier and Type
    Field
    Description
    protected final Optimizer
    LRScheduler.optimizer
    The optimiser whose rate this schedule sets.
    Methods in ai.beestream.jitnn.nn.optim with parameters of type Optimizer
    Modifier and Type
    Method
    Description
    LRScheduler.cosine(Optimizer optimizer, int tMax, float minLr)
    Cosine annealing from base to minLr over tMax epochs (PyTorch's closed form).
    LRScheduler.multiStep(Optimizer optimizer, int[] milestones, float gamma)
    lr = base * gamma^m where m counts the milestones passed.
    void
    GradScaler.step(Optimizer opt)
    Unscales (unless done already) and records the optimiser step, skipped on the device on an overflow.
    LRScheduler.step(Optimizer optimizer, int stepSize, float gamma)
    lr = base * gamma^(epoch / stepSize).
    void
    GradScaler.unscale(Optimizer opt)
    Makes opt divide its gradients by the scale at the next step (and clip after unscaling).
    LRScheduler.warmupCosine(Optimizer optimizer, int warmup, int total, float minRatio)
    Linear warm-up over warmup steps, then a half-cosine decay to minRatio times the base rate at total steps (Hugging Face's get_cosine_schedule_with_warmup when minRatio is 0).
    LRScheduler.warmupLinear(Optimizer optimizer, int warmup, int total)
    Linear warm-up from 0 to the base rate over warmup steps, then linear decay to 0 at total steps (Hugging Face's get_linear_schedule_with_warmup).
    Constructors in ai.beestream.jitnn.nn.optim with parameters of type Optimizer
    Modifier
    Constructor
    Description
    protected
    A schedule for optimizer, starting from its current learning rate.