Completion

In the context of language models, a completion is the text generated by the model in response to an input prompt. The model "completes" the prompt by predicting and generating the most likely continuation based on its training.

The completion process works by repeatedly predicting the next most likely token, then appending it to the sequence and predicting again. This autoregressive process continues until a stopping condition is met (max tokens, stop sequence, or end token).

Key aspects of completions:

• Token-based: Completions are generated and measured in tokens, not words
• Probabilistic: Each generation involves sampling from a probability distribution
• Controllable: Parameters like temperature and top_p affect completion randomness
• Stop conditions: You can specify sequences that end generation

Understanding completions is essential for:

• Estimating and controlling API costs
• Designing prompts that elicit desired responses
• Implementing streaming for real-time applications
• Debugging unexpected model behaviour