HOW A NEURAL NETWORK WORKS
A NEURAL NETWORK is a sequence of computations arranged like a network of neurons, where values are stored and manipulated as they propagate through the network.
For example, take a simple neural network designed to classify a galaxy in a 8x8-pixel monochrome image as either spiral or elliptical. The brightness value of each pixel is fed to a node in the input layer, 91 in all in our case. If that value exceeds a certain threshold, that neuron “fires” and feeds its value to neurons in the next layer, the first of multiple socalled hidden layers.
Each neuron in that layer performs a computation on the values it is fed: First, the values are multiplied by a number specific to the connection from which they came, called a weight. Then they are summed and added to another number specific to that node, called a bias. Weights effectively act as a measure of the strength of that connection in the network, and the biases indicate how sensitive the neuron is to firing.
The strength of a neuron’s signal is determined by the weights, biases, and the signal received — modified by a mathematical function called an activation function — which is then sent to the neurons in the next hidden layer. This process repeats, triggering patterns of neurons, until the values arrive at the final layer, the output layer. The output neurons are like options on a multiplechoice question: one for an elliptical galaxy and one for a spiral galaxy. The neuron with the highest value is the network’s choice for that image.
In the beginning, the weights and biases for each connection and neuron are set to random values, and the network’s choice is no better than a random guess. To train the network, the actual galaxy type — elliptical or spiral, as determined by a human — is propagated backward through the network, and the weights and biases are adjusted to improve the algorithm’s performance. This process can be repeated thousands or millions of times. In more complex neural networks, additional mathematical operations can be performed at the nodes of each layer; this may allow the network to learn to detect edges or textures in the image. The result is a network of nodes with weights and biases tuned to act on fresh input data and make the decision it was intended to make. — Mark Zastrow