Class Llama
java.lang.Object
ai.beestream.jitnn.nn.Module
ai.beestream.jitnn.nn.models.Llama
- All Implemented Interfaces:
AutoCloseable
A Llama-style decoder: token embedding, pre-RMSNorm blocks of causal attention with rotary
position embedding (rotate-half form) and a SwiGLU feed-forward, a final RMSNorm and an untied
output projection; no biases. Seeds are handed out in construction order
(scripts/pytorch/transformers.py builds the same model). With fewer key/value heads than query
heads the attention is grouped-query (GQA), as Llama 2 70B and Llama 3;
fromHuggingFace(Path)
builds a model from a Hugging Face Llama checkpoint.-
Constructor Summary
ConstructorsConstructorDescriptionLlama(int vocab, int dim, int heads, int layers, int hidden, float ropeBase, long seed) A decoder overvocabtokens withlayersblocks ofdimfeatures inheadsheads, a feed-forward ofhiddenfeatures and rotary baseropeBase, initialised fromseed.Llama(int vocab, int dim, int heads, int kvHeads, int layers, int hidden, float ropeBase, float eps, boolean tied, boolean qkvBias, long seed) A Llama withkvHeadskey/value heads (grouped-query attention when fewer thanheads), RMSNorm epsiloneps, the output projection tied to the token embedding whentied, and biases on the q, k and v projections whenqkvBias. -
Method Summary
Modifier and TypeMethodDescriptioncheckpointing(boolean on) Withon, training keeps no activations inside the blocks: each block runs again in the backward pass (Grad.checkpoint(Function, Tensor...)), for less memory and about a third more compute.Logits [B, T, vocab] for INT32 tokens [B, T].static LlamafromHuggingFace(Path dir) A Llama from a Hugging Face checkpoint directory (config.json and safetensors, asHfCheckpoint): model_type llama (or mistral without a sliding window, qwen2 with its q, k, v biases), grouped-query attention, tied or separate output weights, and linear or Llama 3.1 RoPE scaling.ropeScaling(RopeScaling scaling) Rescales the RoPE frequencies of every block (Llama 3.1's rope_scaling); null for none.Methods inherited from class Module
buffer, buffers, child, close, isClosed, isTraining, loadStateDict, parameter, parameterCount, parameters, stateDict, train, zeroGrad
-
Constructor Details
-
Llama
public Llama(int vocab, int dim, int heads, int layers, int hidden, float ropeBase, long seed) A decoder overvocabtokens withlayersblocks ofdimfeatures inheadsheads, a feed-forward ofhiddenfeatures and rotary baseropeBase, initialised fromseed. -
Llama
public Llama(int vocab, int dim, int heads, int kvHeads, int layers, int hidden, float ropeBase, float eps, boolean tied, boolean qkvBias, long seed) A Llama withkvHeadskey/value heads (grouped-query attention when fewer thanheads), RMSNorm epsiloneps, the output projection tied to the token embedding whentied, and biases on the q, k and v projections whenqkvBias.
-
-
Method Details
-
checkpointing
Withon, training keeps no activations inside the blocks: each block runs again in the backward pass (Grad.checkpoint(Function, Tensor...)), for less memory and about a third more compute. -
ropeScaling
Rescales the RoPE frequencies of every block (Llama 3.1's rope_scaling); null for none. -
fromHuggingFace
A Llama from a Hugging Face checkpoint directory (config.json and safetensors, asHfCheckpoint): model_type llama (or mistral without a sliding window, qwen2 with its q, k, v biases), grouped-query attention, tied or separate output weights, and linear or Llama 3.1 RoPE scaling. A head dimension other than hidden_size / heads is not supported.- Throws:
IOException
-
forward
-