Class GloVe

java.lang.Object
ai.beestream.jitnn.nn.Module
ai.beestream.jitnn.nn.models.GloVe
All Implemented Interfaces:
AutoCloseable

public final class GloVe extends Module
GloVe (Pennington et al., 2014): word and context vectors with biases fitted to the logarithm of the co-occurrence counts, weighted by f(x) = min(1, (x / xMax)^alpha). The paper trains it with AdaGrad; the word vectors are the sum of the two tables.
  • Constructor Details

    • GloVe

      public GloVe(int vocab, int dim, long seed)
      GloVe with the paper's weighting (xMax 100, alpha 3/4).
    • GloVe

      public GloVe(int vocab, int dim, float xMax, float alpha, long seed)
      GloVe over vocab words with vectors of dim and weighting f(x) = min(1, (x / xMax)^alpha), initialised from seed.
  • Method Details

    • loss

      public Tensor loss(int[] rows, int[] cols, float[] x)
      The loss of a batch of co-occurrences x at (rows, cols), all > 0.
    • forward

      public Tensor forward(Tensor x)
      The word vectors W + W~ of ids x.
      Specified by:
      forward in class Module
    • vectors

      public float[] vectors()
      All word vectors [vocab, dim] on the host.
    • dim

      public int dim()
      The vector size.