JavaScript – Machine Learning – K-Means Clustering

GitHub Code


As I have stated earlier, I have many software related interests and time is short, so I tend to ‘explore’ a variety of topics all at once.   At times, I will put down a topic until I have time and/or interest in it again.  This time, I have resurrected my speech recognition program (see earlier blog post here – https://erichelin.wordpress.com/2016/10/30/speech-recognition-part-1/ and git code here – https://github.com/ehelin/SpeechRecognition).

I ‘resurrected’ this project because I really think as a software engineer, Artificial Intelligence (AI) related skills are what will allow me to keep working in the future.  My own opinion is that the coding I do now (i.e. writing applications with traditional loops, classes and other Object Oriented (OO) or Functional based techniques) will become redundant in the future.  At some point, AI agents will be creating software without the intervention of the humans.  When this will happen? I am not exactly sure.  But I think it will happen in the next 5-10 years if not sooner.

A good software engineer tries to stay ahead or up to par with whatever technical trends are going on in his or her arena.  For me, this has been .NET.  Recently (last couple of years), my arena has expanded to include JavaScript and Amazon/Azure cloud technologies.  Always onward 🙂

Back to the point. I started the speech recognition application in a Windows 10 Store Application wrapper to learn the related technology (i.e. store applications and speech AI). To this day, I still don’t completely understand while Microsoft had to completely write new API’s to support it, but they did.  Using the windows store API, Visual Studio and the many blog posts available on Windows 10 Store Applications, I was able to create a simple store application that recorded audio, split it into samples, displayed it and played it back.  The version used in the previous blog posts has a number of issues that I have fixed.  However, when I added the AI learning part, I didn’t get the results I expected.  Not even close :(  I will blog about the speech recognition and share code once I get it to work.  A step towards that goal is to take a deep dive into machine learning.

This post will deal with one (of many) machine learning algorithms – K means clustering.

To run:

  • Make GET call

Screen Shot 2017-05-29 at 11.31.01 AM

  • View Results

NOTE: This results set is based on these values:

Screen Shot 2017-05-29 at 10.54.41 AM

Screen Shot 2017-05-29 at 10.55.08 AM

Reference #2 pointed me towards K-Means clustering when I was looking for a solution to my speech recognition problem.  A specific example is provided by reference #1.  The ‘algorithm process’ (as I understand it) is as follows:

  • Take a data set and figure out the two elements that are the furthest from each other
    • Each element has at least two data points to be compared
  • Create two clusters
    • One cluster takes one element
      • Calculate the cluster mean (same as element’s mean at this point)
    • One cluster takes the other element (i.e. one furthest from the first one)
      • Calculate the cluster mean (same as element’s mean at this point)
  • Pass 1
    • In sequence
      • Take the current element’s  mean (i.e. average of the difference between to its two data points) and compare with each cluster’s mean.
      • Assign this element to which ever cluster’s mean is closer
      • Recalculate the assigned cluster’s mean
  • Pass 2
    • Re-compare each element’s mean against its cluster and the other cluster.
    • Re-assign element to the other cluster if its mean is closer than the current cluster.

Using the above process, I implemented a JavaScript version of the K-Means algorithm.

Screen Shot 2017-05-29 at 11.38.43 AM

I added a utility module to keep the main implementation as clean as possible.

Screen Shot 2017-05-29 at 11.38.58 AM

My data set is structurally similar to the one from reference #1.  In it, I created six egg manufacturers and took prices from two different days (totally arbitrary):

egg manufacturer          Day 1          Day 2
egg manufacturer 1          3.50              3.55
egg manufacturer 2          2.99              2.85
egg manufacturer 3          3.10              3.12
egg manufacturer 4          3.05              3.03
egg manufacturer 5          2.20              2.19
egg manufacturer 6          2.80              2.56

In reviewing the results, I too was able to create two clusters which each element belonged too.  To see variety, I also changed up the price data set.

  • Run Two

Prices:

Screen Shot 2017-05-29 at 10.55.46 AM

Screen Shot 2017-05-29 at 10.56.03 AM

  • Run Three

Prices:

Screen Shot 2017-05-29 at 10.56.24 AM

Screen Shot 2017-05-29 at 11.53.36 AM

The next question is how does this help me with my speech recognition problem?  Not exactly sure.  I see parts of a solution.   I am using a perceptron algorithm in the speech recognition application.  A perceptron is an artificial neuron.  It it, inputs and weights are compared from a training set to a specific input.  If the input matches, the perceptron ‘fires’ and the pattern is recognized.  I have been able to use this algorithm for [00,10,01,11] type combinations, but I haven’t been able to get it to work on anything more complicated.

My understanding is that if the training sets fall into two linear separable groups, inputs will be correctly recognized.  Using K-Means clustering, I can see the two linearly separate groups.  It all comes down to if you have structured your data correctly.  The next question will be how to apply the algorithm into assigning new elements to the correct cluster.

Stay tuned!

References:

  1. http://mnemstudio.org/clustering-k-means-example-1.htm
  2. https://blog.intercom.com/machine-learning-way-easier-than-it-looks/
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s