Nov 16, 2026 | 484 words | 5 min read
7.1.7. PCMs: Neural Networks#
These pre-class materials introduce neural networks, how they process inputs through hidden layers, and how they are trained to minimize error.
In engineering, understanding neural networks helps you design algorithms for automation, computer vision, and predictive modeling. In this module, you will learn the structure of a neural network, how to calculate neuron activations using weights and biases, and how to use backpropagation to train a model.
Introduction to Neural Networks#
Structure of a Neural Network#
Neural networks are used to automate engineering tasks ranging from self-driving cars to large language models. These networks process data through three primary stages:
Input Layer: Raw data is fed into the network. For example, a computer vision model reading a handwritten digit translates the image into a grid of pixels where each pixel has a numeric value between 0 and 1.
Hidden Layers: These middle layers contain neurons that process the inputs to predict specific features. The number of neurons in these layers must be manually adjusted to optimize performance.
Output Layer: The final predicted value, such as determining which specific digit was written.
Calculating Neuron Activations#
To determine the value of any specific neuron in a hidden layer, the algorithm must calculate a weighted sum.
The system multiplies every input by a specific weight parameter and adds a separate bias parameter.
Because this weighted sum can range from negative infinity to positive infinity, engineers pass the value through an activation function (like the sigmoid or logistic function). This function scales the final prediction to a usable value between 0 and 1.
As networks grow to include multiple layers and neurons, these equations are written and processed efficiently using matrix notation.
Evaluating Model Performance#
A newly built neural network relies on estimated or random parameters and is generally inaccurate. Engineers evaluate the network’s accuracy using a cost function.
The cost is calculated by taking the squared difference between the neural network’s predicted output and the actual expected output. A high cost indicates the network is performing poorly and the parameters must be recalibrated.
Training Neural Networks#
Training and Backpropagation#
The goal of training a neural network is to tune the parameter weights and biases until the cost function is minimized. The process of recalibrating these parameters is called backpropagation.
Computing the Gradient: Because complex networks have thousands of parameters, engineers compute the gradient to determine the exact direction each parameter must be adjusted to ensure the fastest decrease in the cost function.
The Chain Rule: To find how much a specific parameter affects the total cost, the system uses the chain rule. It multiplies the effect of the weight on the weighted sum, the effect of the weighted sum on the activation, and the effect of the activation on the cost.
Applying the Learning Rate: To calculate the required change for a parameter, the system takes the average of these derivatives and multiplies it by a specific learning rate.