Topics
Neural networks, explained
53 subjects in modern deep learning. Each page explains the idea from its founding papers, then follows the news on it.
Foundations
- Neural networks
- Backpropagation
- Optimizers and gradient descent
- Activation functions
- Normalization layers
- Embeddings1
- Scaling laws
- Tokenization2
Architectures
- Transformers
- Attention1
- Mixture of experts7
- State space models
- Recurrent neural networks
- Convolutional neural networks
- Vision transformers
- Diffusion models
- Autoencoders and VAEs
- Generative adversarial networks
- Graph neural networks
- World models2
Language models
Vision, audio and multimodal
- Multimodal models7
- Vision-language models3
- Image generation
- Video generation
- Speech recognition1
- Text-to-speech1
Reinforcement learning and post-training
- Reinforcement learning
- RLHF and preference tuning1
Training
- Fine-tuning1
- Knowledge distillation2
- Synthetic data
- Distributed training1
Inference and deployment
Evaluation
Interpretability and safety
Beats
Greyed subjects have no page yet; new explainers publish daily. Definitions from every page are collected in the glossary.