Class Emitter
java.lang.Object
ai.beestream.jitnn.engine.Emitter
What an operator sees while its device tasks are added to a task graph: the arrays bound to
its tensors, constant metadata arrays, and helpers for launch grids.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic interfaceAdds one task to the graph under the given task id. -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final intWork-group size for 1-D kernels and group reductions. -
Method Summary
Modifier and TypeMethodDescriptionuk.ac.manchester.tornado.api.types.arrays.ShortArraybfloat16 tensors are stored as their raw 16-bit patterns.uk.ac.manchester.tornado.api.types.arrays.ByteArrayGGUF block-quantised tensors: their raw blocks.static uk.ac.manchester.tornado.api.WorkerGridblocks1D(int groups, int local) groupswork-groups oflocalthreads.static uk.ac.manchester.tornado.api.WorkerGridblocks2D(int gx, int gy, int lx, int ly) gx * gywork-groups oflx * lythreads.static uk.ac.manchester.tornado.api.WorkerGridblocks3D(int gx, int gy, int gz, int lx) gx * gy * gzwork-groups oflxthreads.static intceilDiv(int a, int b) a / b rounded up, for grid sizes.uk.ac.manchester.tornado.api.types.arrays.IntArrayconstant(int... values) A constant int array uploaded once with the graph (shape and stride metadata).uk.ac.manchester.tornado.api.types.arrays.FloatArrayconstantF32(float... values) A constant float array uploaded once with the graph.uk.ac.manchester.tornado.api.KernelContextcontext()A newKernelContext, the first argument of every kernel.uk.ac.manchester.tornado.api.types.arrays.FloatArraydummyF32()A one-element array for optional array arguments that a kernel does not read.uk.ac.manchester.tornado.api.types.arrays.HalfFloatArrayThe device array of FP16 tensort.uk.ac.manchester.tornado.api.types.arrays.FloatArrayThe device array of FP32 tensort, to pass to a kernel.uk.ac.manchester.tornado.api.types.arrays.ByteArrayFP8 tensors: one byte per value.static uk.ac.manchester.tornado.api.WorkerGridgrid1D(int n) A 1-D grid of at leastnthreads, in groups ofLOCAL.static uk.ac.manchester.tornado.api.WorkerGridgrid2D(int x, int y, int tile) A 2-D grid coveringx * ythreads in tiles oftile * tile.static uk.ac.manchester.tornado.api.WorkerGridgroups(int groups) groupswork-groups ofLOCALthreads each.uk.ac.manchester.tornado.api.types.arrays.IntArrayThe device array of INT32 tensort.booleanisCuda()True when the plan runs on CUDA.booleanisMetal()True when the plan runs on Metal (simdgroup matrix kernels are available).voidlibrary(ai.beestream.jitnn.engine.Native.Lib lib, String function, Emitter.TaskAdder adder) Adds a TornadoVM library task (a cuBLAS, cuDNN, ... or MLX call, throughTaskGraph.libraryTask): it has no launch grid, and runs on the same device and stream as the kernels around it.voidlibraryForked(ai.beestream.jitnn.engine.Native.Lib lib, String function, Emitter.TaskAdder adder, String key, Emitter.TaskAdder join) A library task that starts work on a stream of its own, joined back by the tasklibraryJoin(Native.Lib, String, Emitter.TaskAdder, String)adds with the samekey.voidlibraryJoin(ai.beestream.jitnn.engine.Native.Lib lib, String function, Emitter.TaskAdder adder, String key) The task that joins the side worklibraryForked(Native.Lib, String, Emitter.TaskAdder, String, Emitter.TaskAdder)started withkey.node()The node whose tasks are being emitted, or null.uk.ac.manchester.tornado.api.types.arrays.ByteArrayQ8_0 tensors: GGUF's 34-byte blocks of 32 values.uk.ac.manchester.tornado.api.types.arrays.FloatArrayscratch(int numel) A device work buffer owned by the compiled plan (partial results between two passes).uk.ac.manchester.tornado.api.types.arrays.HalfFloatArrayscratchF16(int numel) An FP16 device work buffer owned by the compiled plan.uk.ac.manchester.tornado.api.types.arrays.IntArrayscratchI32(int numel) An INT32 device work buffer owned by the compiled plan.uk.ac.manchester.tornado.api.types.arrays.FloatArraysharedScratch(String role, int numel) A work buffer of exactlynumelelements shared by every task of the plan that asks for the same role and size: tasks run in order, so a buffer one operator fills and reads is free for the next (the layout copies around library calls, which would otherwise take a buffer each).uk.ac.manchester.tornado.api.types.arrays.HalfFloatArraysharedScratchF16(String role, int numel) sharedScratch(String, int)for FP16 buffers.voidtask(uk.ac.manchester.tornado.api.WorkerGrid grid, Emitter.TaskAdder adder) Adds a kernel task launched overgrid;adderadds it to the graph, with arrays from this emitter's accessors.booleanTrue when nothing readstafter this plan: an internal output (no user code holds it) that no later operator of the trace consumes and that was not requested.
-
Field Details
-
LOCAL
public static final int LOCALWork-group size for 1-D kernels and group reductions. Kernels assume this value.- See Also:
-
-
Method Details
-
node
The node whose tasks are being emitted, or null. -
unread
True when nothing readstafter this plan: an internal output (no user code holds it) that no later operator of the trace consumes and that was not requested. An operator may then leave it unwritten. -
f32
The device array of FP32 tensort, to pass to a kernel. -
i32
The device array of INT32 tensort. -
f16
The device array of FP16 tensort. -
q8
Q8_0 tensors: GGUF's 34-byte blocks of 32 values. -
blocks
GGUF block-quantised tensors: their raw blocks. -
fp8
FP8 tensors: one byte per value. -
bf16
bfloat16 tensors are stored as their raw 16-bit patterns. -
constant
public uk.ac.manchester.tornado.api.types.arrays.IntArray constant(int... values) A constant int array uploaded once with the graph (shape and stride metadata). -
constantF32
public uk.ac.manchester.tornado.api.types.arrays.FloatArray constantF32(float... values) A constant float array uploaded once with the graph. -
dummyF32
public uk.ac.manchester.tornado.api.types.arrays.FloatArray dummyF32()A one-element array for optional array arguments that a kernel does not read. -
scratch
public uk.ac.manchester.tornado.api.types.arrays.FloatArray scratch(int numel) A device work buffer owned by the compiled plan (partial results between two passes). -
scratchF16
public uk.ac.manchester.tornado.api.types.arrays.HalfFloatArray scratchF16(int numel) An FP16 device work buffer owned by the compiled plan. -
scratchI32
public uk.ac.manchester.tornado.api.types.arrays.IntArray scratchI32(int numel) An INT32 device work buffer owned by the compiled plan. -
isMetal
public boolean isMetal()True when the plan runs on Metal (simdgroup matrix kernels are available). -
isCuda
public boolean isCuda()True when the plan runs on CUDA. -
context
public uk.ac.manchester.tornado.api.KernelContext context()A newKernelContext, the first argument of every kernel. -
task
Adds a kernel task launched overgrid;adderadds it to the graph, with arrays from this emitter's accessors. -
library
public void library(ai.beestream.jitnn.engine.Native.Lib lib, String function, Emitter.TaskAdder adder) Adds a TornadoVM library task (a cuBLAS, cuDNN, ... or MLX call, throughTaskGraph.libraryTask): it has no launch grid, and runs on the same device and stream as the kernels around it. Its arrays must come from this emitter's accessors. -
libraryForked
public void libraryForked(ai.beestream.jitnn.engine.Native.Lib lib, String function, Emitter.TaskAdder adder, String key, Emitter.TaskAdder join) A library task that starts work on a stream of its own, joined back by the tasklibraryJoin(Native.Lib, String, Emitter.TaskAdder, String)adds with the samekey. A plan captured into CUDA graphs captures each task graph on its own, and a capture may not end with the side stream still forked: when the two tasks land in different task graphs,join(the same task as the joining one) is added at the end of the forking task's graph instead, and the later joining task is left out. -
libraryJoin
public void libraryJoin(ai.beestream.jitnn.engine.Native.Lib lib, String function, Emitter.TaskAdder adder, String key) The task that joins the side worklibraryForked(Native.Lib, String, Emitter.TaskAdder, String, Emitter.TaskAdder)started withkey. -
grid1D
public static uk.ac.manchester.tornado.api.WorkerGrid grid1D(int n) A 1-D grid of at leastnthreads, in groups ofLOCAL. -
groups
public static uk.ac.manchester.tornado.api.WorkerGrid groups(int groups) groupswork-groups ofLOCALthreads each. -
grid2D
public static uk.ac.manchester.tornado.api.WorkerGrid grid2D(int x, int y, int tile) A 2-D grid coveringx * ythreads in tiles oftile * tile. -
blocks1D
public static uk.ac.manchester.tornado.api.WorkerGrid blocks1D(int groups, int local) groupswork-groups oflocalthreads. -
blocks2D
public static uk.ac.manchester.tornado.api.WorkerGrid blocks2D(int gx, int gy, int lx, int ly) gx * gywork-groups oflx * lythreads. -
blocks3D
public static uk.ac.manchester.tornado.api.WorkerGrid blocks3D(int gx, int gy, int gz, int lx) gx * gy * gzwork-groups oflxthreads. -
ceilDiv
public static int ceilDiv(int a, int b) a / b rounded up, for grid sizes.
-