Uses of Interface
ai.beestream.jitnn.dist.Collective
Packages that use Collective
Package
Description
Several devices: collectives (all-reduce, broadcast, reduce-scatter, all-gather) among ranks,
staged through the host (
HostCollective) or run by NCCL on NVIDIA GPUs
(NcclCollective), and ProcessGroup, which
runs one rank per device.Neural-network modules after PyTorch's
torch.nn: Module and
Sequential, linear and convolution layers, pooling, normalisation,
attention, embeddings, recurrent layers (LSTM, GRU), graph convolution, activations, and data-parallel
training (DataParallel).-
Uses of Collective in ai.beestream.jitnn.dist
Classes in ai.beestream.jitnn.dist that implement CollectiveModifier and TypeClassDescriptionfinal classCollectives staged through host memory: each rank copies its tensor to the host, the reduction runs on the host in rank order (so every rank gets bit-identical results, run after run), and each rank copies the result back.final classCollectives on the GPUs, through NCCL (TornadoVM's tornado-nccl): one rank per CUDA device, in one process.Methods in ai.beestream.jitnn.dist with parameters of type CollectiveModifier and TypeMethodDescriptionstatic <T> List<T> ProcessGroup.run(String[] devices, Collective collective, BiFunction<Integer, Collective, T> body) Runsbody(rank, collective)on every rank and returns the ranks' results in rank order.Method parameters in ai.beestream.jitnn.dist with type arguments of type CollectiveModifier and TypeMethodDescriptionstatic <T> List<T> ProcessGroup.run(String[] devices, Collective collective, BiFunction<Integer, Collective, T> body) Runsbody(rank, collective)on every rank and returns the ranks' results in rank order.static <T> List<T> ProcessGroup.run(String[] devices, BiFunction<Integer, Collective, T> body) -
Uses of Collective in ai.beestream.jitnn.nn
Constructors in ai.beestream.jitnn.nn with parameters of type CollectiveModifierConstructorDescriptionDataParallel(Module model, int rank, Collective collective) Wraps rankrank's replicamodel; ranks synchronise throughcollective.SyncBatchNorm2d(int channels, Collective collective, int rank) Synchronised batch normalisation for rankrank, with PyTorch's defaults (momentum 0.1, eps 1e-5).SyncBatchNorm2d(int channels, Collective collective, int rank, float momentum, float eps) Synchronised batch normalisation for rankrankof anNcclCollective; with a null collective (or one rank) it is plain batch normalisation.