GenerationConfigTopP Property

Optional. The maximum cumulative probability of tokens to consider when sampling. The model uses combined Top-k and Top-p (nucleus) sampling. Tokens are sorted based on their assigned probabilities so that only the most likely tokens are considered. Top-k sampling directly limits the maximum number of tokens to consider, while Nucleus sampling limits the number of tokens based on the cumulative probability. Note: The default value varies by Model and is specified by the TopP attribute returned from the [!:getModel] function. An empty TopK attribute indicates that the model doesn't apply top-k sampling and doesn't allow setting TopK on requests.

public double? TopP { get; set; }