Package ai.beestream.jitnn.io
package ai.beestream.jitnn.io
Checkpoints: named tensors and values (
StateDict) saved and loaded in
Hugging Face's safetensors format (SafeTensors).-
ClassDescriptionA small JSON reader for the files jitNN loads (safetensors headers, Hugging Face configs and tokenizers): objects become
LinkedHashMaps in file order, arraysLists, integersLong, other numbersDouble.Reads and writes the safetensors format (Hugging Face): an 8-byte little-endian header length, a JSON header that maps each tensor name to its dtype, shape and byte range, optional__metadata__string pairs, then the raw little-endian data. jitNN's checkpoints use it, so they open in Python withsafetensors.torch.load_file.One tensor of a file: its type (null when jitNN has no such type, e.g.A file's header: the tensors, in file order, and the metadata.The data to write for one tensor: its type, shape, and a segment of exactly its bytes.Named tensors and string values: the state of a model, an optimiser or a whole training run, as PyTorch'sstate_dict.