Class Grad

java.lang.Object
ai.beestream.jitnn.Grad

public final class Grad extends Object
Whether new operators record what autograd needs. On by default, per thread.
  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static interface 
    A try-with-resources scope that restores the previous gradient mode.
  • Method Summary

    Modifier and Type
    Method
    Description
    static Tensor
    checkpoint(Function<Tensor[],Tensor> segment, Tensor... inputs)
    Activation checkpointing, as PyTorch's torch.utils.checkpoint: runs segment on inputs and keeps none of its activations for the backward pass, which runs the segment again to get them.
    static Tensor
    clipNorm(List<Tensor> params, float maxNorm)
    Rescales the gradients of params so that their global L2 norm is at most maxNorm, as PyTorch's clip_grad_norm_: every gradient is multiplied by min(1, maxNorm / (norm + 1e-6)).
    static Grad.Scope
    Disables gradient recording until the returned scope is closed.
    static Tensor
    Runs a forward pass for inference: no gradient recording, and every tensor created inside except the returned one is internal, so its buffer is reused as soon as the computation no longer needs it.
    static Grad.Scope
    The scope form of inference(Supplier), for code that computes several outputs: no gradients, and intermediates are internal (their buffers are reused and inference fusions apply).
    static boolean
    Whether operators recorded now on this thread record what autograd needs.
    static Tensor
    Marks t as read by the caller: its buffer is never reused.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
  • Method Details

    • isEnabled

      public static boolean isEnabled()
      Whether operators recorded now on this thread record what autograd needs.
    • checkpoint

      public static Tensor checkpoint(Function<Tensor[],Tensor> segment, Tensor... inputs)
      Activation checkpointing, as PyTorch's torch.utils.checkpoint: runs segment on inputs and keeps none of its activations for the backward pass, which runs the segment again to get them. The segment's intermediates are freed after their last forward reader, so a training step holds the activations of one segment at a time instead of all of them, for about one more forward pass of compute. Gradients reach the inputs and the parameters the segment uses. Dropout draws the same masks when the segment runs again; BatchNorm in training mode updates its running statistics twice, as in PyTorch.
      for (Block b : blocks) {
          x = Grad.checkpoint(in -> b.forward(in[0]), x);
      }
      
    • disabled

      public static Grad.Scope disabled()
      Disables gradient recording until the returned scope is closed.
    • inferenceScope

      public static Grad.Scope inferenceScope()
      The scope form of inference(Supplier), for code that computes several outputs: no gradients, and intermediates are internal (their buffers are reused and inference fusions apply). Pass each output to keep(Tensor) before the scope closes, or its buffer may be reused.
    • keep

      public static Tensor keep(Tensor t)
      Marks t as read by the caller: its buffer is never reused. Returns t.
    • inference

      public static Tensor inference(Supplier<Tensor> forward)
      Runs a forward pass for inference: no gradient recording, and every tensor created inside except the returned one is internal, so its buffer is reused as soon as the computation no longer needs it. Reading those intermediates after the call is not supported.
    • clipNorm

      public static Tensor clipNorm(List<Tensor> params, float maxNorm)
      Rescales the gradients of params so that their global L2 norm is at most maxNorm, as PyTorch's clip_grad_norm_: every gradient is multiplied by min(1, maxNorm / (norm + 1e-6)). Runs lazily on the device with the rest of the step and returns the norm before clipping as a scalar tensor (reading it is a sync point).