Decoder
The component of a transformer model that generates output sequences. GPT-style models are "decoder-only" architectures optimised for text generation.
In-Depth Explanation
In transformer architecture, the decoder is responsible for generating output sequences one token at a time. Modern generative AI models like GPT, Claude, and Llama are "decoder-only" architectures, meaning they consist entirely of decoder layers.
The decoder works through autoregressive generation:
- Takes all previous tokens as input
- Processes through multiple transformer layers with self-attention
- Predicts the probability distribution for the next token
- Samples from that distribution to select the next token
- Appends the new token and repeats
Key characteristics of decoder models:
- Causal attention: Can only attend to previous tokens (no "peeking ahead")
- Autoregressive: Generate one token at a time, left to right
- Versatile: Can be prompted for many different tasks
- Efficient inference: Optimised for text generation workflows
Decoder-only models contrast with encoder-only models (like BERT, for understanding) and encoder-decoder models (like T5, for translation and summarisation).
Business Context
Decoder models excel at content generation, coding, and conversational AI - the most common business applications of generative AI.
How Clever Ops Uses This
We primarily work with decoder models for our Australian business clients, as they power the chatbots, content generators, and coding assistants most businesses need.
Example Use Case
"GPT models use decoder architecture to generate text one token at a time, each token informed by all previous tokens in the sequence."