JavaScript – Machine Learning with a Perceptron is Limited

Git Hub Code

In my post on Machine Learning (ML) (see below), I was ‘looking’ for the best way to update my speech recognition program to recognize certain words.

Post – https://erichelin.wordpress.com/2017/07/03/javascript-kmeans-is-for-unsupervised-machine-learning-and-i-need-supervised-machine-learning-doh/

To that end, I focused on supervised learning and read Adam Geitgey’s blog series (see reference #3).  Using his house example in the first part, I created a JavaScript version of his algorithm and tried running with it.  About half way in, it dawned on me that he hadn’t provided a complete example (you apparently can get the step by step by subscribing to his class (see reference #4)). I tried completing it without reviewing the video to see how I did.  Didn’t get very far.  I will be signing up for his class shortly 😉

So, I then decided to try the more basic perceptron example I have seen in many forms across the web.  I chose a version I found on coding vision (see reference #2).  I was able to quickly replicate that author’s success.

My application structure looks like this:

Screen Shot 2017-08-17 at 9.18.47 PM

To view:

  • Make a GET call

Screen Shot 2017-08-17 at 8.51.15 PM

  • View The Results

Screen Shot 2017-08-17 at 8.51.33 PM

This JavaScript version of the author’s code consists of a training and run method.  As described on the author’s site, the weights are ‘trained’ on an expected range of input. Then, when presented to the run method with the weights, the specified elements are picked as ‘recognized’ elements from the others.

Screen Shot 2017-08-17 at 9.04.23 PM

After completing that part, I wanted to see if it could be applied the house example and this answer is…well, kinda 🙂  Let me explain.

The house example had three parameters – bedroom count, square footage and the city it is located in.  I plugged this directly into the same perceptron code to see if it worked.  However, instead of 2 inputs, there were 3.  Perceptrons seem to work by dividing two groups of points that are linearly separable (as the reference #2 author pointed out).  3 inputs would imply a 3 dimensional graph with an x, y and z axis.

I never got it to work 😦

So, I removed the location city from the list of variables to make it essentially the same problem as reference #2’s author presented.  It works…kinda and only sometimes.

  • First Issue – If you look at my training data, you will notice that the ‘recognized’ element is last (i.e. ‘expectedOutput: 1’).  If I placed it at the start or middle of the training set, I get false positives.

Screen Shot 2017-08-17 at 9.25.54 PM

  • Second Issue – If you look at my run data, you will notice that the number of bedrooms and the square footage size are not greater than the ‘recognized’ element.  If I added either to a non-recognized element, I got false positives.  It doesn’t seem to matter whether I include data that is larger in the run or training set.

Screen Shot 2017-08-17 at 9.29.12 PM

As long as I stayed within these caveats (ish), it seems to work 🙂

Screen Shot 2017-08-17 at 9.36.56 PM

However, it just kinda reaffirms to me that the Perceptron is good for small, academic and limited issues.  I have played with same algorithm in the past.  For example, I tried to get it to recognize the color blue.  More specifically and following another blogger’s post (he couldn’t get it to work either), I created a 3 input Perceptron to process various RGB colors.  Each input had a value of 0-255/255.   Never worked for me.

For an algorithm to be able to solve the house problem with more than two inputs reliably, it must be a lot more robust than this effort.  Or, my knowledge of how to correctly model my data needs to be improved.  I think both are true 🙂

So, my next post in this series will (hopefully) be with a robust, supervised machine learning algorithm that can complete the house problem.

Stay tuned!

References

  1. http://aass.oru.se/~lilien/ml/seminars/2007_02_01b-Janecek-Perceptron.pdf
  2. http://www.codingvision.net/miscellaneous/c-perceptron-tutorial
  3. https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec3c471
  4. https://www.lynda.com/Data-Science-tutorials/Machine-Learning-Essential-Training-Value-Estimations/548594-2.html?lpk35=9149&utm_medium=ldc-partner&utm_source=CMPRC&utm_content=524&utm_campaign=CD20575&bid=524&aid=CD20575
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s