Class DataLoader
java.lang.Object
ai.beestream.jitnn.data.DataLoader
- All Implemented Interfaces:
AutoCloseable
Batches of a
Dataset for training, as PyTorch's DataLoader: shuffled per epoch
from a seed, optional per-sample transforms (augmentation), the last short batch kept or dropped,
and batches assembled ahead by worker threads while the device computes. Batches come out in
order and do not depend on the number of workers: shuffling and transforms draw from random
streams derived from the seed, the epoch and the sample.
try (DataLoader loader = DataLoader.builder(train, 128).shuffle(42).transform(Transforms.randomCrop(4)).workers(4).build()) {
for (int epoch = 0; epoch < epochs; epoch++) {
for (DataLoader.Batch b : loader.epoch(epoch)) {
Tensor loss = F.crossEntropy(model.forward(b.x()), b.y());
...
}
}
}
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic final recordOne batch: samples [size, ...sampleShape] and labels [size], host tensors.static final classOptions of aDataLoader: shuffling, worker threads, prefetching. -
Method Summary
Modifier and TypeMethodDescriptionintbatches()Batches per epoch.static DataLoader.BuilderA builder for a loader ofdatain batches ofbatchSize.voidclose()epoch(int epoch) The batches of epochepoch(its shuffle and augmentation draws).
-
Method Details
-
builder
A builder for a loader ofdatain batches ofbatchSize. -
batches
public int batches()Batches per epoch. -
epoch
The batches of epochepoch(its shuffle and augmentation draws). -
close
public void close()- Specified by:
closein interfaceAutoCloseable
-