Record Class Tokenizer.Batch
java.lang.Object
java.lang.Record
ai.beestream.jitnn.nn.text.Tokenizer.Batch
- Enclosing class:
Tokenizer
public static record Tokenizer.Batch(int[] idArray, int[] typeIdArray, int[] attention, int batch, int length)
extends Record
Encodings padded to the longest one:
[batch, length] ids and type ids, and an
attention mask of 1 for tokens and 0 for padding.-
Constructor Summary
ConstructorsConstructorDescriptionBatch(int[] idArray, int[] typeIdArray, int[] attention, int batch, int length) Creates an instance of aBatchrecord class. -
Method Summary
Modifier and TypeMethodDescriptionAn additive attention mask [batch, 1, 1, length]: 0 for tokens, -10000 for padding.int[]Returns the value of theattentionrecord component.intbatch()Returns the value of thebatchrecord component.final booleanIndicates whether some other object is "equal to" this one.final inthashCode()Returns a hash code value for this object.int[]idArray()Returns the value of theidArrayrecord component.ids()The token ids as an INT32 tensor[batch, length].intlength()Returns the value of thelengthrecord component.final StringtoString()Returns a string representation of this record class.int[]Returns the value of thetypeIdArrayrecord component.typeIds()The token type (segment) ids as an INT32 tensor[batch, length].
-
Constructor Details
-
Batch
public Batch(int[] idArray, int[] typeIdArray, int[] attention, int batch, int length) Creates an instance of aBatchrecord class.- Parameters:
idArray- the value for theidArrayrecord componenttypeIdArray- the value for thetypeIdArrayrecord componentattention- the value for theattentionrecord componentbatch- the value for thebatchrecord componentlength- the value for thelengthrecord component
-
-
Method Details
-
ids
The token ids as an INT32 tensor[batch, length]. -
typeIds
The token type (segment) ids as an INT32 tensor[batch, length]. -
additiveMask
An additive attention mask [batch, 1, 1, length]: 0 for tokens, -10000 for padding. -
toString
-
hashCode
-
equals
Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. Reference components are compared withObjects::equals(Object,Object); primitive components are compared with thecomparemethod from their corresponding wrapper classes. -
idArray
public int[] idArray()Returns the value of theidArrayrecord component.- Returns:
- the value of the
idArrayrecord component
-
typeIdArray
public int[] typeIdArray()Returns the value of thetypeIdArrayrecord component.- Returns:
- the value of the
typeIdArrayrecord component
-
attention
public int[] attention()Returns the value of theattentionrecord component.- Returns:
- the value of the
attentionrecord component
-
batch
public int batch()Returns the value of thebatchrecord component.- Returns:
- the value of the
batchrecord component
-
length
public int length()Returns the value of thelengthrecord component.- Returns:
- the value of the
lengthrecord component
-