Weights
The numerical values in neural networks that are learned during training. They determine how strongly inputs influence outputs.
In-Depth Explanation
Weights are the learned parameters in neural networks that determine how information flows from inputs to outputs. During training, weights are adjusted to minimize prediction errors, encoding the patterns the model has learned.
How weights work:
- Each connection between neurons has a weight
- Inputs are multiplied by their weights
- Products are summed and passed through activation functions
- The output depends on input values and weight values
Weight characteristics:
- Initialized randomly: Starting points for training
- Updated by gradients: Adjusted to reduce errors
- Encode knowledge: What the model "knows"
- Determine behavior: How the model responds
Weight-related concepts:
- Model parameters: Total number of weights
- Weight decay: Regularisation technique
- Weight sharing: Same weights for multiple inputs
- Pre-trained weights: Weights from training on large data
Business Context
Weights are what make a model "know" things. Fine-tuning adjusts weights; RAG adds knowledge without changing weights.
How Clever Ops Uses This
Understanding weights helps us explain to Australian businesses the difference between fine-tuning (changes weights) and RAG (uses weights as-is but adds context).
Example Use Case
"A model's weights encode patterns like "pizza" being related to "food" and "restaurant" - these associations are learned during training."