Class CustomOp

java.lang.Object
ai.beestream.jitnn.CustomOp

public final class CustomOp extends Object
An operator defined outside jitNN: a kernel written in plain Java with TornadoVM's KernelContext, an optional host reference and an optional gradient. Once built, it is recorded, fused into the step's compiled plan, cached and differentiated like any built-in operator.

Element-wise operators on FP32 tensors take one line:

public static void swish(KernelContext kc, FloatArray x, FloatArray y, int n) {
    int i = kc.globalIdx;
    if (i < n) {
        float v = x.get(i);
        y.set(i, v / (1.0f + TornadoMath.exp(-v)));
    }
}

// d/dx x s(x) = y + s(x) (1 - y)
static final CustomOp SWISH = CustomOp.unary("swish", MyKernels::swish, v -> v / (1 + Math.exp(-v)))
        .withGradient((x, y, gy) -> gy.mul(y.add(x.sigmoid().mul(y.neg().add(1)))));

Tensor y = SWISH.apply(x);

Anything else goes through builder(String): the output shapes, then the tasks the operator adds to the plan, each with an explicit launch grid. Kernels take the KernelContext first and bound-check their thread index; they never use @Parallel.

Names are global: compiled plans are cached by the operators' names and attributes, so two operators cannot share a name. Kernel methods must not be named like a CUDA or OpenCL built-in (exp, hypot, max, ...) or a C++ keyword (not, and, ...): the generated kernel takes the method's name.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static interface 
    Gradients with respect to each input (null where none is needed), given the gradient of the first output.
    static interface 
    The gradients of an element-wise binary operator y = f(a, b): [dL/da, dL/db] from a, b, y and dL/dy.
    static final class 
    Defines a custom operator step by step; CustomOp.Builder.build() registers its name.
    static interface 
    Adds the operator's device tasks for node to the plan being built.
    static interface 
    The shape of an output, from the inputs and the integer attributes of the call.
    static interface 
    Computes node on the host: inputs are readable, outputs are allocated.
    static interface 
    The gradient of an element-wise unary operator y = f(x): dL/dx from x, y and dL/dy.
  • Method Summary

    Modifier and Type
    Method
    Description
    apply(int[] iattr, float[] fattr, Tensor... inputs)
    Applies the operator with integer and float attributes (part of the plan's cache key); returns its first output.
    apply(Tensor... inputs)
    Applies the operator; returns its first output.
    applyAll(int[] iattr, float[] fattr, Tensor... inputs)
    Applies the operator; returns all its outputs.
    static CustomOp
    binary(String name, uk.ac.manchester.tornado.api.common.TornadoFunctions.Task5<uk.ac.manchester.tornado.api.KernelContext, uk.ac.manchester.tornado.api.types.arrays.FloatArray, uk.ac.manchester.tornado.api.types.arrays.FloatArray, uk.ac.manchester.tornado.api.types.arrays.FloatArray, Integer> kernel, DoubleBinaryOperator reference)
    An element-wise operator on two FP32 tensors of the same shape.
    Starts the definition of an operator named name.
    The operator's name, unique among all operators.
     
    static CustomOp
    unary(String name, uk.ac.manchester.tornado.api.common.TornadoFunctions.Task4<uk.ac.manchester.tornado.api.KernelContext, uk.ac.manchester.tornado.api.types.arrays.FloatArray, uk.ac.manchester.tornado.api.types.arrays.FloatArray, Integer> kernel, DoubleUnaryOperator reference)
    An element-wise operator on one FP32 tensor.
    This element-wise binary operator with a gradient.
    This element-wise unary operator with a gradient.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
  • Method Details

    • builder

      public static CustomOp.Builder builder(String name)
      Starts the definition of an operator named name.
    • unary

      public static CustomOp unary(String name, uk.ac.manchester.tornado.api.common.TornadoFunctions.Task4<uk.ac.manchester.tornado.api.KernelContext, uk.ac.manchester.tornado.api.types.arrays.FloatArray, uk.ac.manchester.tornado.api.types.arrays.FloatArray, Integer> kernel, DoubleUnaryOperator reference)
      An element-wise operator on one FP32 tensor. kernel is called as kernel(kc, x, y, n) on a 1-D grid of at least n threads; reference computes one value on the host.
    • binary

      public static CustomOp binary(String name, uk.ac.manchester.tornado.api.common.TornadoFunctions.Task5<uk.ac.manchester.tornado.api.KernelContext, uk.ac.manchester.tornado.api.types.arrays.FloatArray, uk.ac.manchester.tornado.api.types.arrays.FloatArray, uk.ac.manchester.tornado.api.types.arrays.FloatArray, Integer> kernel, DoubleBinaryOperator reference)
      An element-wise operator on two FP32 tensors of the same shape. kernel is called as kernel(kc, a, b, y, n) on a 1-D grid of at least n threads.
    • withGradient

      public CustomOp withGradient(CustomOp.UnaryGradient gradient)
      This element-wise unary operator with a gradient.
    • withGradient

      public CustomOp withGradient(CustomOp.BinaryGradient gradient)
      This element-wise binary operator with a gradient.
    • name

      public String name()
      The operator's name, unique among all operators.
    • apply

      public Tensor apply(Tensor... inputs)
      Applies the operator; returns its first output.
    • apply

      public Tensor apply(int[] iattr, float[] fattr, Tensor... inputs)
      Applies the operator with integer and float attributes (part of the plan's cache key); returns its first output.
    • applyAll

      public Tensor[] applyAll(int[] iattr, float[] fattr, Tensor... inputs)
      Applies the operator; returns all its outputs.
    • toString

      public String toString()
      Overrides:
      toString in class Object