Package ai.beestream.jitnn.nn.models
package ai.beestream.jitnn.nn.models
Complete models: ResNet, MobileNetV2, GPT-2, a Llama-style decoder, BERT, a VAE, DCGAN, U-Net and
character RNNs, and the embedding models word2vec, doc2vec and GloVe. Weights are initialised so that
PyTorch can reproduce them (
Init).-
ClassDescriptionA BERT-style encoder for masked language modelling: token and learned position embeddings with LayerNorm, post-LayerNorm blocks of bidirectional attention (with an additive padding mask) and a GELU MLP of 4 x dim, and an MLM head (dense, GELU, LayerNorm, then the token embedding as the decoder with its own bias).A recurrent language model: token embedding, a stack of LSTM or GRU layers, a linear head over the vocabulary.DCGAN (Radford et al., 2016) for 28x28 single-channel images: a generator from a latent vector through a 7x7 feature map and two stride-2 transposed convolutions to a tanh image, and a discriminator of two stride-2 convolutions with LeakyReLU(0.2) and a linear logit.Images [B, 1, 28, 28] to logits [B, 1].z [B, latent] to images [B, 1, 28, 28] in [-1, 1].Paragraph vectors (Le and Mikolov, 2014) with negative sampling.How a document's words are predicted.GloVe (Pennington et al., 2014): word and context vectors with biases fitted to the logarithm of the co-occurrence counts, weighted by f(x) = min(1, (x / xMax)^alpha).A GPT-2 style decoder: token and learned position embeddings, pre-LayerNorm blocks of causal multi-head attention and a GELU (tanh form) MLP of 4 x dim, a final LayerNorm, and logits from the token embedding (tied weights).A Hugging Face model directory:
config.jsonand the weights inmodel.safetensorsor in shards listed bymodel.safetensors.index.json.A Llama-style decoder: token embedding, pre-RMSNorm blocks of causal attention with rotary position embedding (rotate-half form) and a SwiGLU feed-forward, a final RMSNorm and an untied output projection; no biases.MobileNetV2 (Sandler et al., 2018), torchvision's layout: inverted residual blocks (1x1 expand, 3x3 depthwise, 1x1 linear projection) with ReLU6.ResNet (He et al., 2016).A small U-Net (Ronneberger et al., 2015) for 28x28 inputs with two down-sampling levels: double 3x3 convolutions with ReLU, max pooling, a bottleneck, then one level up through a stride-2 transposed convolution and one through bilinear upsampling and a 1x1 convolution, each concatenated with the encoder's features of that size; a 1x1 convolution gives one logit per pixel.A fully connected variational autoencoder (Kingma and Welling, 2014), the layout of PyTorch's MNIST example: 784-400 encoder to a Gaussian latent (mean, log-variance), 400-784 decoder with a sigmoid output.Reconstruction probabilities [B, in] and the latent mean and log-variance [B, latent].word2vec (Mikolov et al., 2013) with negative sampling: an input embedding (the word vectors) and an output embedding, initialised like the original C code (input uniform in +-0.5 / dim, output zero).Which word predicts which.