Uses of Package
ai.beestream.jitnn.dist
Packages that use ai.beestream.jitnn.dist
Package
Description
Several devices: collectives (all-reduce, broadcast, reduce-scatter, all-gather) among ranks,
staged through the host (
HostCollective) or run by NCCL on NVIDIA GPUs
(NcclCollective), and ProcessGroup, which
runs one rank per device.Neural-network modules after PyTorch's
torch.nn: Module and
Sequential, linear and convolution layers, pooling, normalisation,
attention, embeddings, recurrent layers (LSTM, GRU), graph convolution, activations, and data-parallel
training (DataParallel).-
Classes in ai.beestream.jitnn.dist used by ai.beestream.jitnn.distClassDescriptionCollective communication among the ranks of a group, each rank one thread and usually one device (after torch.distributed).How a reduction combines the ranks' tensors, element-wise.
-
Classes in ai.beestream.jitnn.dist used by ai.beestream.jitnn.nnClassDescriptionCollective communication among the ranks of a group, each rank one thread and usually one device (after torch.distributed).