Recurrent Neural Network
RNN
Neural network designed to process sequential data by maintaining internal state. Used for time series, text, and other sequential tasks before transformers became dominant.
In-Depth Explanation
Recurrent Neural Networks (RNNs) process sequences by maintaining hidden state that carries information across time steps. They were foundational for NLP before transformers.
How RNNs work:
- Process input one step at a time
- Hidden state carries information forward
- Output depends on current input AND history
- Same weights applied at each step
RNN variants:
- Vanilla RNN: Simple but struggles with long sequences
- LSTM: Long Short-Term Memory, gates control information flow
- GRU: Gated Recurrent Unit, simplified LSTM
- Bidirectional: Process sequences both directions
Limitations addressed by transformers:
- Sequential processing (can't parallelise)
- Difficulty with very long dependencies
- Vanishing/exploding gradients (even with LSTM)
RNNs still useful for:
- Real-time streaming processing
- Resource-constrained environments
- Simple sequential tasks
Business Context
While transformers now dominate, understanding RNNs helps grasp sequential processing concepts and may be relevant for specific time-series applications.
How Clever Ops Uses This
We use transformer-based models for most applications but may recommend RNN variants for specific time-series or streaming use cases.
Example Use Case
"An LSTM model predicting equipment failure based on sensor readings over time, where the sequential pattern matters more than absolute values."
Frequently Asked Questions
Related Terms
Related Resources
Transformer
The neural network architecture behind modern LLMs. Uses attention mechanisms to...
Neural Network
A computing system inspired by biological brains, consisting of interconnected n...
Learning Centre
Guides, articles, and resources on AI and automation.
AI & Automation Services
Explore our full AI automation service offering.
AI Readiness Assessment
Check if your business is ready for AI automation.