Interface RopeScaling

All Known Implementing Classes:
RopeScaling.Linear, RopeScaling.Llama3

public sealed interface RopeScaling permits RopeScaling.Linear, RopeScaling.Llama3
A rescaling of the rotary position embedding's frequencies, as Hugging Face's rope_scaling.
  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Interface
    Description
    static final record 
    Positions divided by factor.
    static final record 
    Llama 3.1's rescaling; see llama3(double, double, double, int).
  • Method Summary

    Modifier and Type
    Method
    Description
    double
    apply(double frequency)
    The frequency used in place of frequency (radians per position).
    linear(double factor)
    Positions divided by factor (rope_type linear).
    llama3(double factor, double lowFreqFactor, double highFreqFactor, int originalContext)
    Llama 3.1's rescaling (rope_type llama3): frequencies with wavelengths above originalContext / lowFreqFactor are divided by factor, those below originalContext / highFreqFactor are kept, and the band between is interpolated.
  • Method Details

    • apply

      double apply(double frequency)
      The frequency used in place of frequency (radians per position).
    • linear

      static RopeScaling linear(double factor)
      Positions divided by factor (rope_type linear).
    • llama3

      static RopeScaling llama3(double factor, double lowFreqFactor, double highFreqFactor, int originalContext)
      Llama 3.1's rescaling (rope_type llama3): frequencies with wavelengths above originalContext / lowFreqFactor are divided by factor, those below originalContext / highFreqFactor are kept, and the band between is interpolated.