type
Post
Created date
Apr 21, 2024 05:51 AM
category
LLM
tags
Prompt Engineering
status
Published
Language
English
From
summary
slug
llm-param-settings
password
Author
Priority
Featured
Featured
Cover
Origin
Type
URL
Youtube
Youtube
icon
notion image
Parameter
Purpose
Temperature
Controls the randomness of responses. Lower values yield more deterministic results; higher values promote diversity and creativity.
Top P
Controls the range of token probabilities considered (nucleus sampling). Lower values yields more confident and factual responses.
Max Length
Determines the maximum number of tokens in a response. Setting this helps you prevent long and/or irrelevant response and manage computational costs.
Stop Sequences
Controls specific strings that, when generated, will halt further token generation. Useful for controlling the structure of outputs. For example, you can tell the model to generate lists that have no more than 10 items by adding "11" as a stop sequence.
Frequency Penalty
Applies a penalty to tokens that have appeared frequently. The higher the frequency penalty, the less likely a word will appear again.
Presence Penalty
Penalises tokens based on their occurrence to avoid repeated phrases, aiding in generating more diverse content. - A token that appears twice and a token that appears 10 times are penalized the same. - If you want the model to generate diverse or creative text, you might want to use a higher presence penalty. Or, if you need the model to stay focused, try using a lower presence penalty.
Note: The general suggestion is to adjust temperature or Top P but not both.
 
On LLM BenchmarkFundamental concepts on Neural Network