What is the difference between weights and parameters?

Weights are the connection strengths between neurons. Parameters is the broader term including weights plus biases (offset values). When people say "7B parameter model," they mean ~7 billion weights and biases.

Can I see a model's weights?

For open-source models (Llama, Mistral), yes - you can download and inspect weights. For closed models (GPT-4, Claude), weights are proprietary and not accessible.

Why are weights called "model weights"?

They "weigh" or determine the importance of different inputs and pathways. A high weight means that input strongly influences the output; a low weight means it has little effect.

What happens to weights during fine-tuning?

Fine-tuning adjusts weights to improve performance on specific tasks. Starting from pre-trained weights (general knowledge), fine-tuning specialises them for your use case.

Book Free Assessment Calculator

Weights

The numerical values in neural networks that are learned during training. They determine how strongly inputs influence outputs.

In-Depth Explanation

Weights are the learned parameters in neural networks that determine how information flows from inputs to outputs. During training, weights are adjusted to minimize prediction errors, encoding the patterns the model has learned.

How weights work:

Each connection between neurons has a weight
Inputs are multiplied by their weights
Products are summed and passed through activation functions
The output depends on input values and weight values

Weight characteristics:

Initialized randomly: Starting points for training
Updated by gradients: Adjusted to reduce errors
Encode knowledge: What the model "knows"
Determine behavior: How the model responds

Weight-related concepts:

Model parameters: Total number of weights
Weight decay: Regularisation technique
Weight sharing: Same weights for multiple inputs
Pre-trained weights: Weights from training on large data

Business Context

Weights are what make a model "know" things. Fine-tuning adjusts weights; RAG adds knowledge without changing weights.

How Clever Ops Uses This

Understanding weights helps us explain to Australian businesses the difference between fine-tuning (changes weights) and RAG (uses weights as-is but adds context).