Interface RopeScaling
- All Known Implementing Classes:
RopeScaling.Linear, RopeScaling.Llama3
A rescaling of the rotary position embedding's frequencies, as Hugging Face's
rope_scaling.-
Nested Class Summary
Nested ClassesModifier and TypeInterfaceDescriptionstatic final recordPositions divided byfactor.static final recordLlama 3.1's rescaling; seellama3(double, double, double, int). -
Method Summary
Modifier and TypeMethodDescriptiondoubleapply(double frequency) The frequency used in place offrequency(radians per position).static RopeScalinglinear(double factor) Positions divided byfactor(rope_type linear).static RopeScalingllama3(double factor, double lowFreqFactor, double highFreqFactor, int originalContext) Llama 3.1's rescaling (rope_type llama3): frequencies with wavelengths above originalContext / lowFreqFactor are divided byfactor, those below originalContext / highFreqFactor are kept, and the band between is interpolated.
-
Method Details
-
apply
double apply(double frequency) The frequency used in place offrequency(radians per position). -
linear
Positions divided byfactor(rope_type linear). -
llama3
static RopeScaling llama3(double factor, double lowFreqFactor, double highFreqFactor, int originalContext) Llama 3.1's rescaling (rope_type llama3): frequencies with wavelengths above originalContext / lowFreqFactor are divided byfactor, those below originalContext / highFreqFactor are kept, and the band between is interpolated.
-