Neural Networks From Scratch
Build, Train and Understand Every Piece
- Pages
- 80
- Chapters
- 42
- Level
- Intermediate
- Format
- E-book
About this book
Deep-learning libraries make it easy to train a network without understanding it. This book takes the opposite route: you build one from scratch, piece by piece, until nothing is mysterious.
You will implement neurons and layers, activation functions, loss functions, backpropagation and gradient descent, then improve training with better initialisation, momentum and adaptive optimisers, normalisation, dropout and early stopping. Short Python examples using only basic arrays accompany each step, alongside small calculations you can check by hand.
The final chapters connect your from-scratch network to the convolutional and transformer architectures used today.
What you will learn
- How neurons, layers and activations compute
- Loss functions for regression and classification
- Backpropagation derived and computed step by step
- Optimisers: momentum, RMSProp and Adam
- Initialisation, normalisation, dropout and early stopping
- How your network connects to modern architectures
Table of contents
- IntroductionIntroduction
- Chapter 1What a Neural Network Is
- Chapter 2A Single Neuron
- Chapter 3Activation Functions
- Chapter 4From Neurons to Layers
- Chapter 5The Forward Pass
- Chapter 6Output Layers
- Chapter 7Loss Functions for Regression
- Chapter 8Loss Functions for Classification
- Chapter 9Derivatives in One Page
- Chapter 10Gradients
- Chapter 11Backpropagation for One Neuron
- Chapter 12Backpropagation Through Layers
- Chapter 13Gradient Descent
- Chapter 14A Complete Network From Scratch
- Chapter 15Training and Test Data
- Chapter 16Initialisation
- Chapter 17Learning Rates
- Chapter 18Momentum
- Chapter 19Adaptive Optimisers: RMSProp and Adam
- Chapter 20Vanishing and Exploding Gradients
- Chapter 21Normalisation Layers
- Chapter 22Regularisation
- Chapter 23Dropout
- Chapter 24Early Stopping and Checkpoints
- Chapter 25Debugging Training
- Chapter 26From Dense Layers to Convolutions
- Chapter 27Sequences and Recurrent Networks
- Chapter 28Attention
- Chapter 29The Transformer
- Chapter 30A Two-Layer Backward Pass, by Hand
- Chapter 31A Multi-Class Training Loop
- Chapter 32Evaluating a Classifier
- Chapter 33Classifying Text With Embeddings
- Chapter 34Hyperparameter Search
- Chapter 35What Networks Learn
- Chapter 36Regression Networks in Practice
- Chapter 37Autoencoders
- Chapter 38Transfer Learning and Fine-Tuning
- Chapter 39Saving, Loading and Deploying a Model
- Chapter 40Using a Framework
- Chapter 41Practical Training Recipe
- Chapter 42Where to Go Next
- AppendixExercises
- GlossaryGlossary
Who this book is for
Developers and students with basic Python and algebra who want to understand deep learning from the ground up.
Details
| Format | PDF + web edition |
|---|---|
| Length | 80 pages (approx.) |
| Chapters | 42 |
| Level | Intermediate |
| Language | English |
| Topic | Deep Learning |
| Author | Latent Library Editorial |
Keep reading