Class Adam


public final class Adam extends Optimizer
Adam (Kingma and Ba) with L2 weight decay, or AdamW (decoupled decay) via adamW(List, float, float); with amsgrad, AMSGrad's running maximum of the second moment (Reddi et al.).
  • Field Summary

    Fields inherited from class Optimizer

    lr, params, t
  • Constructor Summary

    Constructors
    Constructor
    Description
    Adam(List<Tensor> params, float lr)
    Adam with PyTorch's defaults: betas 0.9 and 0.999, eps 1e-8, no weight decay.
    Adam(List<Tensor> params, float lr, float beta1, float beta2, float eps, float weightDecay, boolean decoupled)
    Adam with the given hyper-parameters; decoupled makes the weight decay AdamW's.
    Adam(List<Tensor> params, float lr, float beta1, float beta2, float eps, float weightDecay, boolean decoupled, boolean amsgrad)
    Adam with the given hyper-parameters; decoupled makes the weight decay AdamW's, amsgrad uses AMSGrad's maximum of the second moment.
  • Method Summary

    Modifier and Type
    Method
    Description
    static Adam
    adamW(List<Tensor> params, float lr, float weightDecay)
    AdamW: Adam with decoupled weight decay, PyTorch's default betas and eps.
    protected Map<String,Tensor[]>
    Per-parameter state by slot name, each array indexed like the parameters (null where none).
    void
    Records one update of every parameter that has a gradient.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
  • Constructor Details

    • Adam

      public Adam(List<Tensor> params, float lr)
      Adam with PyTorch's defaults: betas 0.9 and 0.999, eps 1e-8, no weight decay.
    • Adam

      public Adam(List<Tensor> params, float lr, float beta1, float beta2, float eps, float weightDecay, boolean decoupled)
      Adam with the given hyper-parameters; decoupled makes the weight decay AdamW's.
    • Adam

      public Adam(List<Tensor> params, float lr, float beta1, float beta2, float eps, float weightDecay, boolean decoupled, boolean amsgrad)
      Adam with the given hyper-parameters; decoupled makes the weight decay AdamW's, amsgrad uses AMSGrad's maximum of the second moment.
  • Method Details

    • adamW

      public static Adam adamW(List<Tensor> params, float lr, float weightDecay)
      AdamW: Adam with decoupled weight decay, PyTorch's default betas and eps.
    • step

      public void step()
      Description copied from class: Optimizer
      Records one update of every parameter that has a gradient.
      Specified by:
      step in class Optimizer
    • state

      protected Map<String,Tensor[]> state()
      Description copied from class: Optimizer
      Per-parameter state by slot name, each array indexed like the parameters (null where none).
      Overrides:
      state in class Optimizer