Uses of Package
ai.beestream.jitnn.nn.text
Packages that use ai.beestream.jitnn.nn.text
Package
Description
Vocabularies and training examples for the embedding models in
ai.beestream.jitnn.nn.models.-
Classes in ai.beestream.jitnn.nn.text used by ai.beestream.jitnn.nn.textClassDescriptionA Hugging Face tokenizer read from its
tokenizer.json, for the two families jitNN's models use: WordPiece with the BERT normaliser and pre-tokeniser (BERT and its relatives), and byte-level BPE (GPT-2, RoBERTa).Encodings padded to the longest one:[batch, length]ids and type ids, and an attention mask of 1 for tokens and 0 for padding.The ids, type ids and tokens of one text (or pair), special tokens included.Word ids by descending frequency, the words seen at leastminCounttimes.