Class HostCollective

java.lang.Object
ai.beestream.jitnn.dist.HostCollective
All Implemented Interfaces:
Collective

public final class HostCollective extends Object implements Collective
Collectives staged through host memory: each rank copies its tensor to the host, the reduction runs on the host in rank order (so every rank gets bit-identical results, run after run), and each rank copies the result back. It works with any mix of devices and backends (CUDA, OpenCL, Metal, even different backends in one group); the price is a device-host round trip per call.
  • Constructor Details

    • HostCollective

      public HostCollective(int worldSize)
      Collectives among worldSize ranks of this process.
  • Method Details

    • worldSize

      public int worldSize()
      Description copied from interface: Collective
      The number of ranks, over every process.
      Specified by:
      worldSize in interface Collective
    • allReduce

      public void allReduce(int rank, Tensor tensor, Collective.ReduceOp op)
      Description copied from interface: Collective
      Every rank's tensor replaced, in place, by the reduction over all ranks.
      Specified by:
      allReduce in interface Collective
    • broadcast

      public void broadcast(int rank, Tensor tensor, int root)
      Description copied from interface: Collective
      Every rank's tensor replaced, in place, by rank root's.
      Specified by:
      broadcast in interface Collective
    • reduceScatter

      public void reduceScatter(int rank, Tensor input, Tensor output, Collective.ReduceOp op)
      Description copied from interface: Collective
      input [worldSize * n, ...] on every rank; output [n, ...] receives on rank r the reduction of every rank's r-th slice.
      Specified by:
      reduceScatter in interface Collective
    • allGather

      public void allGather(int rank, Tensor input, Tensor output)
      Description copied from interface: Collective
      input [n, ...] on every rank; output [worldSize * n, ...] receives every rank's input, in rank order.
      Specified by:
      allGather in interface Collective
    • abort

      public void abort()
      Releases every rank waiting in a collective with an error (called when a rank fails).
      Specified by:
      abort in interface Collective
    • barrier

      public void barrier(int rank)
      Description copied from interface: Collective
      Blocks until every rank has called it.
      Specified by:
      barrier in interface Collective