Temperature
A parameter controlling randomness in AI outputs. Lower temperature (0-0.3) gives consistent, focused responses; higher (0.7-1.0) gives more creative, varied ones.
In-Depth Explanation
Temperature is a parameter that controls the randomness of AI model outputs. It affects how the model selects from possible next tokens - lower temperatures make it more deterministic, higher temperatures more creative.
How temperature works:
- Model predicts probability distribution over possible next tokens
- Temperature scales these probabilities before sampling
- Low temperature sharpens the distribution (top choice more likely)
- High temperature flattens it (more tokens become viable)
Temperature settings:
- 0: Deterministic - always picks highest probability token
- 0.1-0.3: Very focused, consistent, good for factual tasks
- 0.5-0.7: Balanced, good default for most tasks
- 0.8-1.0: Creative, varied, good for brainstorming
- >1.0: Very random, may become incoherent
Temperature recommendations by task:
- Data extraction: 0
- Q&A, factual: 0.1-0.3
- General assistant: 0.5-0.7
- Creative writing: 0.7-0.9
- Brainstorming: 0.8-1.0
Note: Temperature interacts with other parameters like top_p (nucleus sampling). Usually, adjust one or the other, not both.
Business Context
Use low temperature for factual tasks like data extraction; higher for creative tasks like marketing copy. Default 0.7 works for most cases.
How Clever Ops Uses This
We tune temperature settings for each use case with our Australian clients. Factual customer support gets low temperature; content generation gets higher values for more engaging outputs.
Example Use Case
"Temperature 0 for invoice data extraction where consistency is critical; temperature 0.8 for brainstorming marketing taglines where variety is valued."