Class Vocabulary

java.lang.Object
ai.beestream.jitnn.nn.text.Vocabulary

public final class Vocabulary extends Object
Word ids by descending frequency, the words seen at least minCount times.
  • Method Details

    • build

      public static Vocabulary build(List<String[]> sentences, int minCount)
      The vocabulary of the words in sentences seen at least minCount times.
    • size

      public int size()
      The number of words.
    • id

      public int id(String word)
      The id of word, or -1 when it is not in the vocabulary.
    • word

      public String word(int id)
      The word with id id.
    • counts

      public long[] counts()
      Each word's count, indexed by id.
    • encode

      public List<int[]> encode(List<String[]> sentences)
      Each sentence as ids, words outside the vocabulary dropped.