Let’s take the same example of startup ideas brainstorming. Now, set the top-k to 1. This means the model can choose from exactly 1 option. So, no matter how many times you generate, or what temperature value you set, the output will be the same every time, as below.
AI
-
Temperature vs Top-k/Top-p: Token Selection Explained
By
–
How is it different from temperature? In a nutshell:
• Temperature – how to choose between tokens
• Top-k/top-p – how many tokens to choose from As a result,
• Lower top-k/top-p = more predictable
• Higher top-k/top-p = more creative -
Top-k and Top-p: Controlling Model Token Selection
By
–
2. Top-k (also top-p) – controls the list from which the model can make its choices. Increasing the top-k increases the number of tokens that the model can choose from, and vice versa.
-
Auto Diagram Creator: AI Tool with Creative Temperature Settings
By
–
Now with a higher temperature (1.0), we get an idea of an auto diagram creator. Definitely more ambitious!
-
Low Temperature AI Generates Automatic To-Do List App
By
–
With a low temperature (0.2), we get an idea of an automatic to-do list app. Interesting, but nothing revolutionary. (model-generated text in bold)
-
Brainstorming Startup Ideas in Productivity Industry
By
–
Let’s take an example: brainstorming startup ideas. Using this prompt below, we want to get ideas in the productivity industry.
-
Temperature Controls Model Token Generation Creativity
By
–
1. Temperature – controls how the model chooses from its next choice of tokens. Increasing the temperature makes generating tokens with lower likelihoods more probable, and vice versa. As a result:
• Lower temperature = more predictable
• Higher temperature = more creative -
Understanding Token Likelihood Parameters in Language Models
By
–
It does this by giving a “likelihood” number to each token. So, for the phrase “I like to bake …”, “cookies” has a higher likelihood than “chairs.” “chairs” can still appear, but “cookies” has a much higher chance. But we can change this behavior with those 3 parameters.
-
Adjusting Model Parameters: Temperature, Top-k, Top-p
By
–
So, how can you do this? By adjusting the model parameters. There are 3 parameters you can adjust:
1. Temperature
2. Top-k
3. Top-p But before we see how to use them, let’s first understand how the model selects the next token to generate. -
Harmonai joins Replicate with open source audio ML models
By
–
Also, welcome @harmonai_org to Replicate! They're a open source collective of audio ML hackers. This is just the first of many models they're working on and we're looking forward to all those robo-tunes.