Package ai.beestream.jitnn.ml


package ai.beestream.jitnn.ml
Classical machine learning after scikit-learn, run on the device: k-means, k-nearest neighbours, PCA, linear and logistic regression, linear SVM, naive Bayes, decision trees, random forests, gradient boosting and t-SNE.
  • Class
    Description
    Gradient-boosted trees with second-order (Newton) leaves on binned features, after scikit-learn's HistGradientBoosting and XGBoost: squared error for regression, the logistic loss for two classes and the softmax loss (one tree per class and round) for more.
    k-means clustering with Lloyd's algorithm, after scikit-learn's KMeans: k-means++ seeding (or given initial centres), nInit restarts keeping the lowest inertia, and stopping when the assignments no longer change or the centres move less than tol times the mean feature variance.
    k-nearest neighbours by brute force on the device (Euclidean distance, uniform weights), after scikit-learn's KNeighborsClassifier and KNeighborsRegressor.
    Least squares with an optional L2 penalty alpha (ridge regression), solved in closed form from the normal equations (X^T X + alpha I) w = X^T y on centred data, after scikit-learn's LinearRegression (alpha 0) and Ridge.
    Linear support vector classification with the squared hinge loss, one-vs-rest, after scikit-learn's LinearSVC (liblinear, intercept_scaling 1): for each class it minimises ||[w, b]||^2 / 2 + C sum_i max(0, 1 - y_i (w . x_i + b))^2, the intercept penalised like a weight, with L-BFGS (the objective is smooth).
    L2-regularised logistic regression fitted with L-BFGS, after scikit-learn's LogisticRegression (lbfgs): it minimises C sum_i loss_i + ||W||^2 / 2 with the intercept unpenalised; a sigmoid for two classes, the multinomial (softmax) loss for more.
    Naive Bayes classifiers after scikit-learn's GaussianNB (per-class means and variances, the variances smoothed by varSmoothing times the largest feature variance) and MultinomialNB (feature counts with additive smoothing alpha).
    Which likelihood the features follow.
    Principal component analysis from the eigendecomposition of the covariance matrix, after scikit-learn's PCA: components as rows, ordered by explained variance, each signed so that its largest-magnitude entry is positive (scikit-learn's svd_flip convention).
    Random forests (Breiman, 2001) and single decision trees on binned features: trees grown on bootstrap samples (weights drawn on the host) with a random subset of the features at each split, predictions averaged.
    What the forest predicts.
    Exact t-SNE (van der Maaten and Hinton, 2008) on the device, after scikit-learn's TSNE with method="exact": Gaussian affinities calibrated per point to the perplexity by bisection on the precision, symmetrised; a Student-t kernel in the embedding; gradient descent with momentum (0.5, then 0.8), early exaggeration (12 for 250 iterations), per-parameter gains (+0.2 / x0.8, at least 0.01) and the learning rate max(n / 12 / 4, 50).