Recurrent Neural Network
RNN
Neural network designed to process sequential data by maintaining internal state. Used for time series, text, and other sequential tasks before transformers became dominant.
In-Depth Explanation
Recurrent Neural Networks (RNNs) process sequences by maintaining hidden state that carries information across time steps. They were foundational for NLP before transformers.
How RNNs work:
- Process input one step at a time
- Hidden state carries information forward
- Output depends on current input AND history
- Same weights applied at each step
RNN variants:
- Vanilla RNN: Simple but struggles with long sequences
- LSTM: Long Short-Term Memory, gates control information flow
- GRU: Gated Recurrent Unit, simplified LSTM
- Bidirectional: Process sequences both directions
Limitations addressed by transformers:
- Sequential processing (can't parallelise)
- Difficulty with very long dependencies
- Vanishing/exploding gradients (even with LSTM)
RNNs still useful for:
- Real-time streaming processing
- Resource-constrained environments
- Simple sequential tasks
Business Context
While transformers now dominate, understanding RNNs helps grasp sequential processing concepts and may be relevant for specific time-series applications.
How Clever Ops Uses This
We use transformer-based models for most applications but may recommend RNN variants for specific time-series or streaming use cases.
Example Use Case
"An LSTM model predicting equipment failure based on sensor readings over time, where the sequential pattern matters more than absolute values."