Neural network playground

Network

784 inputs64 ReLU10 softmax

Hidden units64

Initialization

Trainable parameters: 50,890

Changing the layer size or the initialization rebuilds the network and repeats the warm start on 150 synthetic digits.

Mini-batch SGD

Learning rate0.1

Batch size32

Epochs8

Shuffle every epoch Include the 150 synthetic digits

Train re-initializes the weights, then runs cross-entropy backpropagation over the training set. Without the synthetic digits you need at least 5 drawings of your own.

Teach

Steps per teach3

Auto-correct a wrong prediction

A teach step is one SGD update on the current drawing alone. With auto-correct on, Predict teaches the network whenever its answer disagrees with the label you picked.

Your training set

No drawings yet. Draw a digit, pick its label and press Add to training set.

How it works

Forward. The drawing is shrunk to 28 × 28 = 784 input values, passed through one ReLU hidden layer and turned into 10 class probabilities by a softmax.

Backward. Cross-entropy loss is differentiated by hand: backpropagation yields ∂L/∂W2 and ∂L/∂W1, and mini-batch SGD moves every weight against its gradient.

Start. Weights begin as He or Xavier noise, then a short warm start on font-rendered digits gives the network a rough idea of each class.

Draw & predict

Draw a digit here

784 inputs

Brush size18

No prediction yet

Draw a digit, then press Predict.

This drawing is a

Draw a digit, then press Predict. Pick its label to add it to the training set or to teach the network.

Training

Loss–cross-entropy

Accuracy–last epoch

Samples0yours, plus 150 synthetic

Epochs0 / 8warm start

The loss curve appears as soon as the first epoch finishes.

Preparing the warm start on synthetic digits.

Draw & predict

Training

Neurons, weights & gradients