Class JitnnConfig
java.lang.Object
ai.beestream.jitnn.JitnnConfig
jitNN's options, in one immutable object. Each option has a key (
jitnn.*), a type, a
default and a description; docs/CONFIGURATION.md lists them all.
The runtime reads its configuration once, the first time it needs an option, and keeps it:
by default the jitnn.* system properties at that moment (fromSystemProperties()).
To configure jitNN from code, install(JitnnConfig) a configuration before the first tensor
operation:
JitnnConfig.install(JitnnConfig.builder()
.set(JitnnConfig.PLAN_CACHE, 64)
.set(JitnnConfig.NATIVE, "auto")
.build());
Only the options marked changeable at run time
(determinism, autotuning, the GEMM and transpose variants, the CPU backend's library choice)
can change afterwards, through change(JitnnConfig.Option, T) or their setters such as
JitnnRuntime.setDeterministic; the others are start-time options.
An invalid value (a number that does not parse, a value outside an option's choices) fails
with an IllegalArgumentException naming the key.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic final classBuilds a configuration option by option.static final classOne option: its key, type, default and description.static enumThe groups options are listed in. -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final JitnnConfig.Option<String> jitnn.autotune.static final JitnnConfig.Option<String> jitnn.autotune.dir.static final JitnnConfig.Option<Double> jitnn.autotune.margin.static final JitnnConfig.Option<Integer> jitnn.autotune.reps.static final JitnnConfig.Option<Integer> jitnn.autotune.runs.static final JitnnConfig.Option<Boolean> jitnn.autotune.verbose.static final JitnnConfig.Option<String> jitnn.backend.static final JitnnConfig.Option<Boolean> jitnn.binaryRepeat.static final JitnnConfig.Option<Boolean> jitnn.channelsLast.static final JitnnConfig.Option<Boolean> jitnn.channelsLast.verbose.static final JitnnConfig.Option<Integer> jitnn.concurrency.static final JitnnConfig.Option<String> jitnn.conv.static final JitnnConfig.Option<Boolean> jitnn.convPhases.static final JitnnConfig.Option<Integer> jitnn.conv.weightChunk.static final JitnnConfig.Option<Integer> jitnn.conv.weightGroups.static final JitnnConfig.Option<String> jitnn.conv1x1.static final JitnnConfig.Option<String> jitnn.conv1x1.backward.static final JitnnConfig.Option<String> jitnn.conv1x1.gemm.static final JitnnConfig.Option<String> jitnn.cpu.blas.static final JitnnConfig.Option<String> jitnn.cpu.bnns.static final JitnnConfig.Option<Integer> jitnn.cpu.gemm.kc.static final JitnnConfig.Option<Integer> jitnn.cpu.gemm.mcPanels.static final JitnnConfig.Option<Integer> jitnn.cpu.gemm.nc.static final JitnnConfig.Option<Long> jitnn.cpu.gemm.tileMacs.static final JitnnConfig.Option<String> jitnn.cpu.native.choose.static final JitnnConfig.Option<String> jitnn.cpu.onednn.static final JitnnConfig.Option<String> jitnn.cpu.opencl.static final JitnnConfig.Option<String> jitnn.cpu.opencl.choose.static final JitnnConfig.Option<Long> jitnn.cpu.poolBytes.static final JitnnConfig.Option<Boolean> jitnn.cpu.reuse.static final JitnnConfig.Option<Integer> jitnn.cpu.reuse.traces.static final JitnnConfig.Option<Long> jitnn.cpu.spinMicros.static final JitnnConfig.Option<Boolean> jitnn.cpu.stats.static final JitnnConfig.Option<Boolean> jitnn.cpu.stats.shapes.static final JitnnConfig.Option<Integer> jitnn.cpu.threads.static final JitnnConfig.Option<Boolean> jitnn.cudaGraph.static final JitnnConfig.Option<Boolean> jitnn.debugKeys.static final JitnnConfig.Option<Boolean> jitnn.deterministic.static final JitnnConfig.Option<String> jitnn.device.static final JitnnConfig.Option<String> jitnn.djl.fallback.static final JitnnConfig.Option<Long> jitnn.djl.persistentBytes.static final JitnnConfig.Option<Boolean> jitnn.dynamicShapes.static final JitnnConfig.Option<String> jitnn.flash.static final JitnnConfig.Option<Boolean> jitnn.flash.mma.static final JitnnConfig.Option<Boolean> jitnn.fuse.static final JitnnConfig.Option<Boolean> jitnn.fuse.elementwise.static final JitnnConfig.Option<Boolean> jitnn.fuse.program.static final JitnnConfig.Option<Boolean> jitnn.fusedAttention.static final JitnnConfig.Option<String> jitnn.gemm.static final JitnnConfig.Option<Integer> jitnn.gguf.tiledRows.static final JitnnConfig.Option<Integer> jitnn.graph.maxTasks.static final JitnnConfig.Option<String> jitnn.halfAttention.static final JitnnConfig.Option<Long> jitnn.hostPool.bytes.static final JitnnConfig.Option<String> jitnn.native.static final JitnnConfig.Option<Boolean> jitnn.native.approximateGelu.static final JitnnConfig.Option<Boolean> jitnn.native.attentionFp16.static final JitnnConfig.Option<Boolean> jitnn.native.biasEpilogue.static final JitnnConfig.Option<Long> jitnn.native.convMinFlops.static final JitnnConfig.Option<String> jitnn.native.cudnnChannelsLast.static final JitnnConfig.Option<Long> jitnn.native.mlxConvMinFlops.static final JitnnConfig.Option<Long> jitnn.native.mlxCopyMinFlops.static final JitnnConfig.Option<Long> jitnn.native.mlxGatherMin.static final JitnnConfig.Option<Boolean> jitnn.native.tf32.static final JitnnConfig.Option<String> jitnn.native.unavailable.static final JitnnConfig.Option<Boolean> jitnn.native.verbose.static final JitnnConfig.Option<String> jitnn.onnx.buckets.static final JitnnConfig.Option<Integer> jitnn.onnx.fusedAttentionRows.static final JitnnConfig.Option<Boolean> jitnn.onnx.halfActivations.static final JitnnConfig.Option<Boolean> jitnn.onnx.int8.static final JitnnConfig.Option<Boolean> jitnn.onnx.rewrite.static final JitnnConfig.Option<Long> jitnn.parallelCopyBytes.static final JitnnConfig.Option<Integer> jitnn.plan.cache.static final JitnnConfig.Option<Boolean> jitnn.plan.carry.static final JitnnConfig.Option<Long> jitnn.plan.idleBytes.static final JitnnConfig.Option<Long> jitnn.plan.idleExecutions.static final JitnnConfig.Option<Boolean> jitnn.plan.memo.static final JitnnConfig.Option<String> jitnn.printKernels.static final JitnnConfig.Option<Boolean> jitnn.profile.static final JitnnConfig.Option<Long> jitnn.promoteInputBytes.static final JitnnConfig.Option<Boolean> jitnn.reuseBuffers.static final JitnnConfig.Option<String> jitnn.reuseBuffers.exact.static final JitnnConfig.Option<String> jitnn.rnn.static final JitnnConfig.Option<String> jitnn.rownorm.static final JitnnConfig.Option<Boolean> jitnn.splitK.static final JitnnConfig.Option<Integer> jitnn.splitK.groups.static final JitnnConfig.Option<Boolean> jitnn.tensorCores.static final JitnnConfig.Option<String> jitnn.transpose.static final JitnnConfig.Option<String> jitnn.tuning.static final JitnnConfig.Option<Boolean> jitnn.verbose.static final JitnnConfig.Option<String> jitnn.winograd.static final JitnnConfig.Option<Boolean> jitnn.withNative. -
Method Summary
Modifier and TypeMethodDescriptionstatic JitnnConfig.Builderbuilder()A builder starting from the defaults.static JitnnConfig.BuilderA builder starting from the system properties.static <T> voidchange(JitnnConfig.Option<T> option, T value) Changes an option that may change at run time (JitnnConfig.Option.changeableAtRunTime()); operations recorded afterwards see the new value.static JitnnConfigcurrent()The runtime's configuration.static <T> booleandefaultIfUnset(JitnnConfig.Option<T> option, T value) Givesoptiona different default before the runtime starts, unless it was set: for integrations such as the DJL engine.static JitnnConfigdefaults()All defaults.booleanstatic JitnnConfigfromProperties(Properties properties) The options set inproperties(jitnn.*keys); other keys are ignored.static JitnnConfigThe options set as system properties now.<T> Tget(JitnnConfig.Option<T> option) The value ofoption: the one set, else its default (null for a machine-dependent default).inthashCode()static voidinstall(JitnnConfig config) Makesconfigthe runtime's configuration.booleanisSet(JitnnConfig.Option<?> option) Whetheroptionwas set (as a system property or with the builder) rather than left at its default.static StringmarkdownTable(JitnnConfig.Section section) The Markdown table of a section's options (property, default, meaning), asdocs/CONFIGURATION.mdshows it.static List<JitnnConfig.Option<?>> options()Every option, in documentation order.The options that were set, askey=valueproperties.toString()<T> JitnnConfigwith(JitnnConfig.Option<T> option, T value) A copy withoptionset tovalue(null: back to its default).
-
Field Details
-
BACKEND
jitnn.backend. -
DEVICE
jitnn.device. -
CONCURRENCY
jitnn.concurrency. -
CPU_THREADS
jitnn.cpu.threads. -
CPU_SPIN_MICROS
jitnn.cpu.spinMicros. -
CPU_POOL_BYTES
jitnn.cpu.poolBytes. -
CPU_GEMM_KC
jitnn.cpu.gemm.kc. -
CPU_GEMM_MC_PANELS
jitnn.cpu.gemm.mcPanels. -
CPU_GEMM_NC
jitnn.cpu.gemm.nc. -
CPU_GEMM_TILE_MACS
jitnn.cpu.gemm.tileMacs. -
CPU_OPENCL
jitnn.cpu.opencl. -
CPU_OPENCL_CHOOSE
jitnn.cpu.opencl.choose. -
CPU_BLAS
jitnn.cpu.blas. -
CPU_ONEDNN
jitnn.cpu.onednn. -
CPU_BNNS
jitnn.cpu.bnns. -
CPU_NATIVE_CHOOSE
jitnn.cpu.native.choose. -
CPU_REUSE
jitnn.cpu.reuse. -
CPU_REUSE_TRACES
jitnn.cpu.reuse.traces. -
CPU_STATS
jitnn.cpu.stats. -
CPU_STATS_SHAPES
jitnn.cpu.stats.shapes. -
PLAN_CACHE
jitnn.plan.cache. -
PLAN_IDLE_BYTES
jitnn.plan.idleBytes. -
PLAN_IDLE_EXECUTIONS
jitnn.plan.idleExecutions. -
PLAN_CARRY
jitnn.plan.carry. -
PLAN_MEMO
jitnn.plan.memo. -
PROMOTE_INPUT_BYTES
jitnn.promoteInputBytes. -
CUDA_GRAPH
jitnn.cudaGraph. -
DYNAMIC_SHAPES
jitnn.dynamicShapes. -
GRAPH_MAX_TASKS
jitnn.graph.maxTasks. -
HOST_POOL_BYTES
jitnn.hostPool.bytes. -
PARALLEL_COPY_BYTES
jitnn.parallelCopyBytes. -
REUSE_BUFFERS
jitnn.reuseBuffers. -
REUSE_BUFFERS_EXACT
jitnn.reuseBuffers.exact. -
VERBOSE
jitnn.verbose. -
DEBUG_KEYS
jitnn.debugKeys. -
PRINT_KERNELS
jitnn.printKernels. -
PROFILE
jitnn.profile. -
GEMM
jitnn.gemm. -
TUNING
jitnn.tuning. -
FUSE_PROGRAM
jitnn.fuse.program. -
DETERMINISTIC
jitnn.deterministic. -
AUTOTUNE
jitnn.autotune. -
AUTOTUNE_DIR
jitnn.autotune.dir. -
AUTOTUNE_MARGIN
jitnn.autotune.margin. -
AUTOTUNE_REPS
jitnn.autotune.reps. -
AUTOTUNE_RUNS
jitnn.autotune.runs. -
AUTOTUNE_VERBOSE
jitnn.autotune.verbose. -
SPLIT_K
jitnn.splitK. -
SPLIT_K_GROUPS
jitnn.splitK.groups. -
TENSOR_CORES
jitnn.tensorCores. -
TRANSPOSE
jitnn.transpose. -
BINARY_REPEAT
jitnn.binaryRepeat. -
FUSED_ATTENTION
jitnn.fusedAttention. -
FLASH
jitnn.flash. -
FLASH_MMA
jitnn.flash.mma. -
ROWNORM
jitnn.rownorm. -
CONV
jitnn.conv. -
WINOGRAD
jitnn.winograd. -
CONV1X1
jitnn.conv1x1. -
CONV1X1_GEMM
jitnn.conv1x1.gemm. -
CONV1X1_BACKWARD
jitnn.conv1x1.backward. -
CONV_PHASES
jitnn.convPhases. -
CONV_WEIGHT_GROUPS
jitnn.conv.weightGroups. -
CONV_WEIGHT_CHUNK
jitnn.conv.weightChunk. -
RNN
jitnn.rnn. -
GGUF_TILED_ROWS
jitnn.gguf.tiledRows. -
FUSE
jitnn.fuse. -
CHANNELS_LAST
jitnn.channelsLast. -
CHANNELS_LAST_VERBOSE
jitnn.channelsLast.verbose. -
FUSE_ELEMENTWISE
jitnn.fuse.elementwise. -
ONNX_REWRITE
jitnn.onnx.rewrite. -
ONNX_INT8
jitnn.onnx.int8. -
ONNX_BUCKETS
jitnn.onnx.buckets. -
ONNX_HALF_ACTIVATIONS
jitnn.onnx.halfActivations. -
ONNX_FUSED_ATTENTION_ROWS
jitnn.onnx.fusedAttentionRows. -
DJL_FALLBACK
jitnn.djl.fallback. -
DJL_PERSISTENT_BYTES
jitnn.djl.persistentBytes. -
NATIVE
jitnn.native. -
WITH_NATIVE
jitnn.withNative. -
HALF_ATTENTION
jitnn.halfAttention. -
NATIVE_APPROXIMATE_GELU
jitnn.native.approximateGelu. -
NATIVE_BIAS_EPILOGUE
jitnn.native.biasEpilogue. -
NATIVE_TF32
jitnn.native.tf32. -
NATIVE_CONV_MIN_FLOPS
jitnn.native.convMinFlops. -
NATIVE_MLX_CONV_MIN_FLOPS
jitnn.native.mlxConvMinFlops. -
NATIVE_MLX_COPY_MIN_FLOPS
jitnn.native.mlxCopyMinFlops. -
NATIVE_MLX_GATHER_MIN
jitnn.native.mlxGatherMin. -
NATIVE_CUDNN_CHANNELS_LAST
jitnn.native.cudnnChannelsLast. -
NATIVE_ATTENTION_FP16
jitnn.native.attentionFp16. -
NATIVE_UNAVAILABLE
jitnn.native.unavailable. -
NATIVE_VERBOSE
jitnn.native.verbose.
-
-
Method Details
-
options
Every option, in documentation order. -
get
The value ofoption: the one set, else its default (null for a machine-dependent default). -
isSet
Whetheroptionwas set (as a system property or with the builder) rather than left at its default. -
with
A copy withoptionset tovalue(null: back to its default). -
toProperties
The options that were set, askey=valueproperties. -
equals
-
hashCode
-
toString
-
defaults
All defaults. -
fromProperties
The options set inproperties(jitnn.*keys); other keys are ignored. -
fromSystemProperties
The options set as system properties now. -
builder
A builder starting from the defaults. -
builderFromSystemProperties
A builder starting from the system properties. -
current
The runtime's configuration. The first call fixes it: the installed one, else the system properties at that moment. -
install
Makesconfigthe runtime's configuration. Call it before the first tensor operation; once the runtime has read its configuration, installing a different one fails with anIllegalStateException(usechange(JitnnConfig.Option, T)for the options that may change). -
defaultIfUnset
Givesoptiona different default before the runtime starts, unless it was set: for integrations such as the DJL engine. True if the value was applied. -
change
Changes an option that may change at run time (JitnnConfig.Option.changeableAtRunTime()); operations recorded afterwards see the new value. Other options fail with anIllegalArgumentException. -
markdownTable
The Markdown table of a section's options (property, default, meaning), asdocs/CONFIGURATION.mdshows it.
-