Fine-Tuning
Adapting a pre-trained model to a specific task or domain by training it further on specialised data. Creates a new model variant.
In-Depth Explanation
Fine-tuning takes a pre-trained model and continues training it on your specific dataset, specialising its capabilities for your use case. This creates a custom model variant with knowledge and behaviours tailored to your needs.
The fine-tuning process:
- Start with a capable pre-trained model (GPT-4, Llama, etc.)
- Prepare a dataset of examples in your domain
- Train the model on your data (typically for a few epochs)
- The model's weights are adjusted to perform better on your specific task
- You now have a custom model deployment
When to consider fine-tuning:
- Do fine-tune when you need: consistent formatting, domain-specific knowledge baked in, particular tone/style, reduced prompt length, higher throughput
- Don't fine-tune when: prompting achieves good results, data changes frequently, you lack sufficient training examples (<100), or RAG can provide the knowledge
Modern efficient techniques (LoRA, QLoRA) have made fine-tuning much more accessible, reducing compute requirements by 90% or more.
Business Context
Fine-tuning is best when you need consistent, specialised behaviour that can't be achieved through prompting alone. Consider it for high-volume, specialised tasks.
How Clever Ops Uses This
We help Australian businesses determine when fine-tuning is worth the investment versus using RAG or advanced prompting. Often, clever prompting achieves 80% of fine-tuning benefits at 10% of the cost.
Example Use Case
"Fine-tuning a model on your company's writing style for consistent brand voice across all AI-generated content."
Frequently Asked Questions
Related Terms
Related Resources
Training
The process of teaching an AI model by exposing it to data and adjusting its par...
LoRA (Low-Rank Adaptation)
An efficient fine-tuning technique that trains only a small number of additional...
QLoRA (Quantized LoRA)
An even more efficient fine-tuning technique that combines quantisation with LoR...
What is RAG (Retrieval Augmented Generation)?
Learn how RAG combines the power of large language models with your business data to provide accurat...
Fine-tuning vs RAG vs Prompt Engineering: Complete Comparison
Understand the differences between fine-tuning, RAG, and prompt engineering. Learn when to use each ...
Learning Centre
Guides, articles, and resources on AI and automation.
AI & Automation Services
Explore our full AI automation service offering.
AI Readiness Assessment
Check if your business is ready for AI automation.