Class Emitter

java.lang.Object
ai.beestream.jitnn.engine.Emitter

public final class Emitter extends Object
What an operator sees while its device tasks are added to a task graph: the arrays bound to its tensors, constant metadata arrays, and helpers for launch grids.
  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static interface 
    Adds one task to the graph under the given task id.
  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    Work-group size for 1-D kernels and group reductions.
  • Method Summary

    Modifier and Type
    Method
    Description
    uk.ac.manchester.tornado.api.types.arrays.ShortArray
    bfloat16 tensors are stored as their raw 16-bit patterns.
    uk.ac.manchester.tornado.api.types.arrays.ByteArray
    GGUF block-quantised tensors: their raw blocks.
    static uk.ac.manchester.tornado.api.WorkerGrid
    blocks1D(int groups, int local)
    groups work-groups of local threads.
    static uk.ac.manchester.tornado.api.WorkerGrid
    blocks2D(int gx, int gy, int lx, int ly)
    gx * gy work-groups of lx * ly threads.
    static uk.ac.manchester.tornado.api.WorkerGrid
    blocks3D(int gx, int gy, int gz, int lx)
    gx * gy * gz work-groups of lx threads.
    static int
    ceilDiv(int a, int b)
    a / b rounded up, for grid sizes.
    uk.ac.manchester.tornado.api.types.arrays.IntArray
    constant(int... values)
    A constant int array uploaded once with the graph (shape and stride metadata).
    uk.ac.manchester.tornado.api.types.arrays.FloatArray
    constantF32(float... values)
    A constant float array uploaded once with the graph.
    uk.ac.manchester.tornado.api.KernelContext
    A new KernelContext, the first argument of every kernel.
    uk.ac.manchester.tornado.api.types.arrays.FloatArray
    A one-element array for optional array arguments that a kernel does not read.
    uk.ac.manchester.tornado.api.types.arrays.HalfFloatArray
    The device array of FP16 tensor t.
    uk.ac.manchester.tornado.api.types.arrays.FloatArray
    The device array of FP32 tensor t, to pass to a kernel.
    uk.ac.manchester.tornado.api.types.arrays.ByteArray
    FP8 tensors: one byte per value.
    static uk.ac.manchester.tornado.api.WorkerGrid
    grid1D(int n)
    A 1-D grid of at least n threads, in groups of LOCAL.
    static uk.ac.manchester.tornado.api.WorkerGrid
    grid2D(int x, int y, int tile)
    A 2-D grid covering x * y threads in tiles of tile * tile.
    static uk.ac.manchester.tornado.api.WorkerGrid
    groups(int groups)
    groups work-groups of LOCAL threads each.
    uk.ac.manchester.tornado.api.types.arrays.IntArray
    The device array of INT32 tensor t.
    boolean
    True when the plan runs on CUDA.
    boolean
    True when the plan runs on Metal (simdgroup matrix kernels are available).
    void
    library(ai.beestream.jitnn.engine.Native.Lib lib, String function, Emitter.TaskAdder adder)
    Adds a TornadoVM library task (a cuBLAS, cuDNN, ... or MLX call, through TaskGraph.libraryTask): it has no launch grid, and runs on the same device and stream as the kernels around it.
    void
    libraryForked(ai.beestream.jitnn.engine.Native.Lib lib, String function, Emitter.TaskAdder adder, String key, Emitter.TaskAdder join)
    A library task that starts work on a stream of its own, joined back by the task libraryJoin(Native.Lib, String, Emitter.TaskAdder, String) adds with the same key.
    void
    libraryJoin(ai.beestream.jitnn.engine.Native.Lib lib, String function, Emitter.TaskAdder adder, String key)
    The node whose tasks are being emitted, or null.
    uk.ac.manchester.tornado.api.types.arrays.ByteArray
    Q8_0 tensors: GGUF's 34-byte blocks of 32 values.
    uk.ac.manchester.tornado.api.types.arrays.FloatArray
    scratch(int numel)
    A device work buffer owned by the compiled plan (partial results between two passes).
    uk.ac.manchester.tornado.api.types.arrays.HalfFloatArray
    scratchF16(int numel)
    An FP16 device work buffer owned by the compiled plan.
    uk.ac.manchester.tornado.api.types.arrays.IntArray
    scratchI32(int numel)
    An INT32 device work buffer owned by the compiled plan.
    uk.ac.manchester.tornado.api.types.arrays.FloatArray
    sharedScratch(String role, int numel)
    A work buffer of exactly numel elements shared by every task of the plan that asks for the same role and size: tasks run in order, so a buffer one operator fills and reads is free for the next (the layout copies around library calls, which would otherwise take a buffer each).
    uk.ac.manchester.tornado.api.types.arrays.HalfFloatArray
    sharedScratchF16(String role, int numel)
    sharedScratch(String, int) for FP16 buffers.
    void
    task(uk.ac.manchester.tornado.api.WorkerGrid grid, Emitter.TaskAdder adder)
    Adds a kernel task launched over grid; adder adds it to the graph, with arrays from this emitter's accessors.
    boolean
    True when nothing reads t after this plan: an internal output (no user code holds it) that no later operator of the trace consumes and that was not requested.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
  • Field Details

    • LOCAL

      public static final int LOCAL
      Work-group size for 1-D kernels and group reductions. Kernels assume this value.
      See Also:
  • Method Details

    • node

      public Node node()
      The node whose tasks are being emitted, or null.
    • unread

      public boolean unread(Tensor t)
      True when nothing reads t after this plan: an internal output (no user code holds it) that no later operator of the trace consumes and that was not requested. An operator may then leave it unwritten.
    • f32

      public uk.ac.manchester.tornado.api.types.arrays.FloatArray f32(Tensor t)
      The device array of FP32 tensor t, to pass to a kernel.
    • i32

      public uk.ac.manchester.tornado.api.types.arrays.IntArray i32(Tensor t)
      The device array of INT32 tensor t.
    • f16

      public uk.ac.manchester.tornado.api.types.arrays.HalfFloatArray f16(Tensor t)
      The device array of FP16 tensor t.
    • q8

      public uk.ac.manchester.tornado.api.types.arrays.ByteArray q8(Tensor t)
      Q8_0 tensors: GGUF's 34-byte blocks of 32 values.
    • blocks

      public uk.ac.manchester.tornado.api.types.arrays.ByteArray blocks(Tensor t)
      GGUF block-quantised tensors: their raw blocks.
    • fp8

      public uk.ac.manchester.tornado.api.types.arrays.ByteArray fp8(Tensor t)
      FP8 tensors: one byte per value.
    • bf16

      public uk.ac.manchester.tornado.api.types.arrays.ShortArray bf16(Tensor t)
      bfloat16 tensors are stored as their raw 16-bit patterns.
    • constant

      public uk.ac.manchester.tornado.api.types.arrays.IntArray constant(int... values)
      A constant int array uploaded once with the graph (shape and stride metadata).
    • constantF32

      public uk.ac.manchester.tornado.api.types.arrays.FloatArray constantF32(float... values)
      A constant float array uploaded once with the graph.
    • dummyF32

      public uk.ac.manchester.tornado.api.types.arrays.FloatArray dummyF32()
      A one-element array for optional array arguments that a kernel does not read.
    • scratch

      public uk.ac.manchester.tornado.api.types.arrays.FloatArray scratch(int numel)
      A device work buffer owned by the compiled plan (partial results between two passes).
    • sharedScratch

      public uk.ac.manchester.tornado.api.types.arrays.FloatArray sharedScratch(String role, int numel)
      A work buffer of exactly numel elements shared by every task of the plan that asks for the same role and size: tasks run in order, so a buffer one operator fills and reads is free for the next (the layout copies around library calls, which would otherwise take a buffer each).
    • sharedScratchF16

      public uk.ac.manchester.tornado.api.types.arrays.HalfFloatArray sharedScratchF16(String role, int numel)
      sharedScratch(String, int) for FP16 buffers.
    • scratchF16

      public uk.ac.manchester.tornado.api.types.arrays.HalfFloatArray scratchF16(int numel)
      An FP16 device work buffer owned by the compiled plan.
    • scratchI32

      public uk.ac.manchester.tornado.api.types.arrays.IntArray scratchI32(int numel)
      An INT32 device work buffer owned by the compiled plan.
    • isMetal

      public boolean isMetal()
      True when the plan runs on Metal (simdgroup matrix kernels are available).
    • isCuda

      public boolean isCuda()
      True when the plan runs on CUDA.
    • context

      public uk.ac.manchester.tornado.api.KernelContext context()
      A new KernelContext, the first argument of every kernel.
    • task

      public void task(uk.ac.manchester.tornado.api.WorkerGrid grid, Emitter.TaskAdder adder)
      Adds a kernel task launched over grid; adder adds it to the graph, with arrays from this emitter's accessors.
    • library

      public void library(ai.beestream.jitnn.engine.Native.Lib lib, String function, Emitter.TaskAdder adder)
      Adds a TornadoVM library task (a cuBLAS, cuDNN, ... or MLX call, through TaskGraph.libraryTask): it has no launch grid, and runs on the same device and stream as the kernels around it. Its arrays must come from this emitter's accessors.
    • libraryForked

      public void libraryForked(ai.beestream.jitnn.engine.Native.Lib lib, String function, Emitter.TaskAdder adder, String key, Emitter.TaskAdder join)
      A library task that starts work on a stream of its own, joined back by the task libraryJoin(Native.Lib, String, Emitter.TaskAdder, String) adds with the same key. A plan captured into CUDA graphs captures each task graph on its own, and a capture may not end with the side stream still forked: when the two tasks land in different task graphs, join (the same task as the joining one) is added at the end of the forking task's graph instead, and the later joining task is left out.
    • libraryJoin

      public void libraryJoin(ai.beestream.jitnn.engine.Native.Lib lib, String function, Emitter.TaskAdder adder, String key)
    • grid1D

      public static uk.ac.manchester.tornado.api.WorkerGrid grid1D(int n)
      A 1-D grid of at least n threads, in groups of LOCAL.
    • groups

      public static uk.ac.manchester.tornado.api.WorkerGrid groups(int groups)
      groups work-groups of LOCAL threads each.
    • grid2D

      public static uk.ac.manchester.tornado.api.WorkerGrid grid2D(int x, int y, int tile)
      A 2-D grid covering x * y threads in tiles of tile * tile.
    • blocks1D

      public static uk.ac.manchester.tornado.api.WorkerGrid blocks1D(int groups, int local)
      groups work-groups of local threads.
    • blocks2D

      public static uk.ac.manchester.tornado.api.WorkerGrid blocks2D(int gx, int gy, int lx, int ly)
      gx * gy work-groups of lx * ly threads.
    • blocks3D

      public static uk.ac.manchester.tornado.api.WorkerGrid blocks3D(int gx, int gy, int gz, int lx)
      gx * gy * gz work-groups of lx threads.
    • ceilDiv

      public static int ceilDiv(int a, int b)
      a / b rounded up, for grid sizes.