![Cover image for The Perceptron! [visually see an AI learn in real time! 👀]](https://media2.dev.to/dynamic/image/width=1000,height=420,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fcftzntahq9yxf2h4pctd.jpg)
I have read loads of articles on how Machine Learning (ML) models work. They all explain the same things in pretty much the same way.
One thing I haven't found though is the purest form of machine learning.
Machine learning that is so simple that I can visually understand what is happening. An example where I can see a model go from knowing nothing to understanding something, and actually follow along exactly how it works!
I mean, I (kinda) get all the weights, biases etc. I managed to build a neural network (NN) in vanilla JS (and even built a Neural network in CSS 🤯!), but I still couldn't wrap my head around what that actually looked like.
Then I read about "the perceptron" (project PARA), one of the earliest ML experiments from the late 1950s and early 1960s (yes, we have been doing machine learning for that long, in fact since the 1940s!)
This was what I was looking for as this was a machine learning model that used ANALOGUE systems to work, and so had to be simple(r).
And from that simplicity I could finally build what I wanted, a simple model where we could see it learning in real time and, more importantly, understand what is happening as it learns!
If you are inpatient you can skip to the demo, however I would recommend reading about how it works and the rules it uses to learn as that is important to understand what you are seeing!
Shoutout to Welch Labs!
Welch labs did an amazing explainer video on the perceptron, which inspired me to make this article. Some of the screenshots are from the physical perceptron machine they built!
A Perceptron model you can watch learn!
This is what I built (or should I say 03-mini built with a lot of prompting and some hand holding - FYI o3 mini produces some horrible code, but it is impressive as a prototyping tool! Still took about 4 hours to put this example together though!).
Let me explain what you are seeing in that screenshot and then you can have a play with the demo!
Input grid
So the first two boxes at the top are our inputs.
They are two 5 by 5 boxes where you can flip "switches".
A checked box represents a positive voltage, an unchecked checkbox represents a negative voltage.
For our model to demonstrate learning we need a "good" shape and a "bad" shape.
These are our code representation of an input grid of switches, like in this example here where you would manually flip switches each time for a "good" or "bad" shape:
Dials / weights
Below the input grid we have the dials / weights.
In the real world analogue version of this they were physical dials that multiplied the input voltage by the amount showed on the dial.
In our example they are represented by numbers in each box, showing either a positive or negative multiplier.
You will also notice in the screenshot there is a colour on the cell (for a positive we have green, for a negative we have a red) for easier visual distinction (for those who aren't colourblind).
And finally you will notice there is a "+" or a "-" in each box, yet again to show if they are making that inputs signal stronger or weaker.
This is effectively our Neural Network. These are the "weights / biases" you may have heard about in modern neural networks.
If a grid square is positive / green then it is like turning a dial up, and a negative / red grid square is like turning a dial down.
They are a representation of a set of dials that look like this:
Each perceptron's output voltage visualiser
This is just to help us understand what is going on in the model, it is the output of an input multiplied by a "dial".
To explain further let's assume the first dial is set to "+18"
Then let's assume we feed the perceptron a positive voltage (and all inputs are either +10 volts if they are on, or -10 volts if they are off).
That means that this dial would output +180 volts.
Now let's assume we changed the switch (the first checkbox) to "off". Now we would get an output of -180v.
The important part to note is that if this "dial" is negative (say it is set to "-12") then the opposite is true.
If the switch is "on" then we would get -120v output (+10 volts multiplied by -12) and if the switch is "off" then we would get +120v.
Then the way the model works is by adding up all the voltages.
Oh and one last tiny note here, if a dial is "0" then we just output the voltage it received.
The voltage output meter
The voltage output meter is our final output.
It is the result of multiplying the input voltages for each switch (our checkboxes, either +10v or -10v) by each corresponding gain switch and then adding all the results together.
In our perceptron a positive value is meant to be "good", and a negative value "bad". The amount of the voltage does not matter, just whether it is above 0 or below 0.
This represents a voltage meter in the real model:
How the model learns.
So this perceptron model follows some simple rules.
It always starts by feeding it a shape, in this instance our "good" shape, with all the dials / weights set at 0.
Remember that any checked box outputs +10 volts and any unchecked box will output -10 volts.
So if we draw a shape with 4 checkboxes we will get +40 volts from those, but also get -450 volts from all the others (bear in mind that the actual board is 7 by 7 so 49 total on / off inputs.)
So as we have no weightings / adjustments we get a total negative output voltage.
That is the key as then we apply rules on what to do according to that output:
- If the correct output voltage should be positive (a good shape) and we get a negative voltage then we turn all of the cells which are "on" up and all of the other cells down.
- If the correct output voltage should be positive and we get a positive voltage output - we do nothing.
- If the correct output voltage should be negative (a bad shape) and we get a positive voltage then we turn all of the cells which are "on" down and all of the other cells up.
- If the correct output voltage should be negative and we get a negative voltage output - we do nothing.
In our model we jump in multipliers of 3. So after one training session we get an output like this:
Why the 49 boxes when our input is only 25 boxes?
This is where the magic happens.
You see if our shape was the same size as our output grid we could argue we are just storing that shape in memory (effectively). The model hasn't learned anything in reality.
So the next thing we do is to move the shape one square to the right and follow the same procedure.
This time (due to our previous step) the model will output a different voltage, because now we have some of the off switches being multiplied by a negative number and giving a positive voltage output, and some of our on switches will have moved onto a negative multiplier and become negative.
So we follow the same rules this time depending on the output voltage.
Then finally, once we have tried all 9 positions for our "good" shape and adjusted the dials for each one we then feed in the 9 positions for the "bad" shape.
At the end of this training the model has learned how to identify a shape. It may not always get it right after one learning pass, but it will eventually get it right after many passes (well, nearly always, there are a couple of shape combinations that will never "settle" on a correct dial pattern).
Ok, so that was a lot of words, not we get to play!
The demo
With all of the above in mind, the demo is below.
BEFORE YOU DO play with it, read the following instructions.
Using the model properly is crucial to making it easy to understand what is going on!
Instructions
Enter a shape in the "good" box and enter a different shape in the "bad" box by clicking on the checkboxes.
Once you have your shapes it is time to train!
Click on the first button "start training".
You will see the system put the good shape in the "dials and weights" box in the top left most position and instantly adjust the voltages according to the rules.
Then it will move the shape to the right one position and follow the same rules (keep an eye on the voltage meter as it moves so you can see if the voltage is positive or negative and how that changes the behaviour).
It will repeat this for 3 column and 3 row combinations so that all possible positions where the shape could be entered are trained upon.
It will then switch the to "bad" shape and follow the same rules (bearing in mind that this time we want a negative output).
When training is complete you will see an alert:
testing the model!
Now you have two options:
Option 1: Click one of the 9 boxes in the "Select Top-Left Position for Testing" box. You will see a modal asking you for which shape to test ("A" is "good" and "B" is "bad").
You will get an alert with the perceptron's guess.
And then you will see the shape rendered on the "Select Top-Left Position for Testing" box, along with the resulting voltage outputs for each dial / switch combination.
Click a few more boxes, choose "A" or "B" to test good and bad shapes and see how often the model gets the guess right or wrong!
This is the best way to understand how the model is working, so do this a couple of times and check the outputs.
Option 2: Let the system test until it find a failure!
If you just want to see how well the model currently does click "Test All".
This will run through all of the shape positions for both shapes and will stop with an error if it guesses wrong (outputs a negative voltage for a good shape for example) and let you know which position it failed at.
Or if you are really lucky it may complete all the checks and you will get a success message!
If it passes then it will correctly identify the shape (Good / A or Bad / B) no matter which valid position you place it in.
It failed
Not surprising, you did one training pass!
Now you could click "start training" again, but that is deliberately slow.
So to speed things up you can click "Self train and test".
This runs through all training positions at 10x speed and then runs tests until it fails on a shape, repeating this process 50 times or until it passes every test!
Once you understand the principles and have played enough with the slow version, click this button and watch how the voltages for each tile change and how it gradually gets further through the tests until it (hopefully) finally passes!
If it doesn't pass try a different shape or press "self train and test" again to run another 50 cycles!
Then once it passes everything try clicking on a few on the "Select Top-Left Position for Testing" squares and trying an "A" or "B" shape.
Pay special attention to the voltages in each box and the dial settings. Notice where the strongest and weakest cells are etc.
I find it super interesting how "easy" it is to understand what is going on!
Codepen Demo
using the button at the bottom of the CodePen so it is easier to use
Trying new shapes
If you want to try a new shape or just want to watch it learn from scratch again, just press "reset dials" and all of the dials will return to 0.
That's a wrap
I know that was a lot of words, but I am hoping you find it useful for understanding how a model learns, in it's simplest form.
I found this a lot easier to understand than matrix multiplication and graphs and charts and although it isn't how modern LLMS / ML models work, it is a great way of getting the principles.
Let me know in the comments if you found it useful / helped you understand a little better!
Top comments (4)
I hope you enjoyed this article, I know it was long but it has lots of interesting things that really help with true foundational understanding of how a ML model can learn.
Bear in mind this is not how modern NN learn, for that you may want to check out my neural network in vanilla JS as that is a lot closer to modern ML (although more complex). 💗
Is there a CodePen link for the demo?
Very cool article, I will read it more thoroughly later 🔥
OMG thank you, I wrote all that and while editing deleted the actual codepen link section! hahahaha
Here it is so you don't have to sift through the article for it: codepen.io/GrahamTheDev/pen/XJrQrGY
Thank you! 👍