In-Context Learning
The ability of LLMs to learn and adapt their behaviour based on examples and instructions provided in the prompt, without model updates.
In-Depth Explanation
In-context learning (ICL) is the remarkable ability of large language models to adapt to new tasks based solely on examples provided in the prompt - no gradient updates or model changes required.
How in-context learning works:
- The model receives a prompt with instructions and/or examples
- Through its attention mechanism, it identifies patterns in the examples
- It applies these patterns to generate appropriate outputs
- All of this happens at inference time, using the model's existing weights
Forms of in-context learning:
- Zero-shot: Task description only, no examples
- One-shot: Single example provided
- Few-shot: Multiple examples demonstrating the pattern
- Instruction following: Detailed natural language instructions
Why ICL is powerful:
- Instant adaptation without training
- Easy to iterate and experiment
- Works across diverse domains
- Accessible without ML expertise
- Preserves base model capabilities
Research shows ICL emerges with scale - smaller models don't exhibit this ability as strongly.
Business Context
In-context learning enables rapid customisation - you can change AI behaviour instantly by updating prompts rather than retraining models.
How Clever Ops Uses This
We leverage in-context learning extensively at Clever Ops, enabling rapid prototyping and iteration for Australian business clients without the delays of model training.
Example Use Case
"Providing 3 examples of your preferred email format so the AI mimics your style for all future emails."