Uses of Class
ai.beestream.jitnn.nn.models.Bert

Packages that use Bert
Package
Description
Complete models: ResNet, MobileNetV2, GPT-2, a Llama-style decoder, BERT, a VAE, DCGAN, U-Net and character RNNs, and the embedding models word2vec, doc2vec and GloVe.
  • Uses of Bert in ai.beestream.jitnn.nn.models

    Methods in ai.beestream.jitnn.nn.models that return Bert
    Modifier and Type
    Method
    Description
    Bert.checkpointing(boolean on)
    With on, training keeps no activations inside the blocks: each block runs again in the backward pass (Grad.checkpoint(Function, Tensor...)), for less memory and about a third more compute.
    static Bert
    A BERT-family encoder from a Hugging Face model directory: config.json and safetensors weights, by model_type: bert (bert-base-uncased and other BERT checkpoints, as BertModel, BertForMaskedLM or BertForPreTraining); roberta (roberta-base and the like): positions counted from the padding index + 1, skipping padding, as RoBERTa does; LayerNorm epsilon 1e-5; distilbert: no token types, no pooler. The MLM head is loaded when the checkpoint has it; otherwise it keeps its initial values.