Perceptron Notebook Reflections

I worked through three neural network notebooks: a simple perceptron, a perceptron trained for the AND operator, and a multilayer perceptron for a more complex non-linear problem.

Simple perceptron

Google Colab notebook: Open in Google Colab · Raw Notebook: Download .ipynb

The first notebook introduces the basic structure of a perceptron using NumPy arrays. One array holds the inputs, another holds the weights, and their dot product gives the weighted sum.

The main process is:

1. Define input values.
2. Define weights.
3. Multiply inputs by weights using a dot product.
4. Pass the result into a step function.
5. Return either 1 or 0 depending on whether the weighted sum reaches the threshold.

At its simplest, a perceptron converts numeric inputs into a binary decision. In the notebook, adjusting the weights changes the final output, making their effect on the model's behaviour easy to see.

This simple example uses only one input vector. That makes the calculation easy to follow, but also highlights a limitation: a single perceptron calculation is only one small part of training a model for a real use case.

Perceptron AND operator

Google Colab notebook: Open in Google Colab · Raw Notebook: Download .ipynb

The second notebook builds on this idea by using the full truth table for the AND operator:

Input 1	Input 2	Output
0	0	0
0	1	0
1	0	0
1	1	1

Here, the input rows form a matrix, the targets are held in an array, and the weights start at zero. The notebook then defines a training loop. For each row, the perceptron makes a prediction, compares it with the expected output, calculates the error, and updates the weights when the prediction is wrong.

This is closer to a machine learning task because the model is not simply given fixed weights. Instead, it adjusts them over repeated passes until the total error reaches zero. The final weights then correctly classify all four AND cases.

AND is linearly separable, so a single perceptron can solve it: one straight decision boundary separates the positive case (1, 1) from the other three cases. This dataset is useful for learning, but it is also very small and artificial. It does not include noise, ambiguity, missing values, or overlapping classes.

Multilayer perceptron

Google Colab notebook: Open in Google Colab · Raw Notebook: Download .ipynb

The third notebook introduces a multilayer perceptron for a more complex target pattern. Its outputs match the XOR operator: (0, 1) and (1, 0) should return 1, while (0, 0) and (1, 1) should return 0.

A single perceptron cannot solve XOR because the classes are not linearly separable. The notebook addresses this by adding a hidden layer between the input layer and the output layer. It also replaces the step function with the sigmoid activation function, which produces values between 0 and 1.

The training process is more involved:

1. Send the inputs through the first set of weights into the hidden layer.
2. Apply the sigmoid function to the hidden-layer values.
3. Send the hidden-layer outputs through a second set of weights into the output layer.
4. Compare the predicted outputs with the target outputs.
5. Use the sigmoid derivative to calculate error signals.
6. Update both sets of weights through backpropagation.
7. Repeat this process for many epochs until the error becomes very small.

The hidden layer allows the network to learn an internal representation of the data rather than relying on a single straight boundary. This extra capacity comes at a cost: training is more computationally expensive and more sensitive to choices such as random weight initialisation, learning rate, number of epochs, and activation function.

Learning reflection

A simple weighted sum demonstrates the mechanics of a perceptron, but it does not learn. A single perceptron can learn the AND dataset because the classes are linearly separable, whereas the XOR-style dataset needs a multilayer network because one linear boundary cannot separate the pattern.

The contrast also highlights why results from simple datasets do not transfer neatly to real-world problems, where data may contain noise, missing values, non-linear relationships, class imbalance, and ambiguous boundaries. As the dataset becomes more complex, the model may need more capacity, more training time, and more careful tuning.