granted07@kolkata:~/NeuralNetworkInC$ cmake --build build
neural-network-in-c
A neural network in C, built from the matrix up. The one behind this text runs the same arithmetic in your browser. By the end of the page you will have taught it something.
scroll
Everything in the library is one struct.
typedef struct {
int rows;
int cols;
double **matrix;
} Matrix;Inputs, weights, biases and gradients are all matrices. Add, subtract, multiply, transpose, scale, apply a function to every entry. Around a dozen operations, and nothing underneath them.
A layer is one multiplication and one function.
z = W x + b a = activation(z)
Move the inputs below. Each row of W scales the inputs, adds a bias and produces one number. The activation then bends it.
0.80 × 1.20 + −0.60 × 0.50 + 0.10
z = 0.76→a = 0.68
−0.40 × 1.20 + 0.90 × 0.50 + −0.20
z = −0.23→a = 0.44
0.50 × 1.20 + 0.70 × 0.50 + 0.05
z = 1.00→a = 0.73
Each dot is one row. Its position along the bottom is z, its height is what the activation turns z into. Move the sliders and watch them slide along the curve.
The activation is a function pointer.
double (*activation)(double); double (*activation_derivative)(double);
Every layer carries its own pair. Switching from sigmoid to relu a moment ago changed one pointer and no line of the layer.
Stack layers and you have a network. The sizes array is the whole blueprint.
int sizes[] = {2, 4, 1};
network_create(sizes, 2, activations, derivatives);Two numbers in, four in the middle, one out. That is two layers, and the picture at the top of the page.
now teach it
XOR is 1 when exactly one input is 1. Nobody writes that rule into the network. It gets four examples, guesses, is told how wrong it was, and every weight moves a little. Press train.
Green lines are positive weights, red are negative. Light moving along a line is input × weight. Drag to rotate.
Every possible input. Green means it answers 1, red means 0. White dots should be green, black dots red. Drag here to feed it your own input.
How wrong it is, per epoch.
epoch 0 loss — 0 xor 0 → 0.000 0 xor 1 → 0.000 1 xor 0 → 0.000 1 xor 1 → 0.000
This runs a TypeScript copy of the same arithmetic, one example at a time, like the C training loop.
That was network_forward then network_backward, four examples at a time. This is what one backward call does, layer by layer.
dMSE/da = 2/n · (prediction − target)
How far off each output was, and in which direction.
the rules it keeps
- Every matrix is heap-allocated. Whoever creates one frees it.
- network_forward returns a matrix the last layer owns. Copy it if you want to keep it.
- -Wall -Wextra -Werror: a warning is a failed build.
- Tests for matrices, activations, layer forward and backward, network forward and loss, run with ctest.
- The library builds as nncore. Programs in apps/ only link against it.
free_matrix(m); free_layer(layer); network_free(network);
still open
?One example at a time. Mini-batches mean teaching the matrices to hold many columns.
?Plain gradient descent. Momentum and Adam need state stored next to each layer.
?Weights start uniform in [−1, 1]. Larger networks want Xavier or He.
?Matrices are arrays of row pointers. One contiguous block would be faster.
?No saving or loading a trained network yet.