Neural Network in ml5.js

12/27/2020 1 Comment

Every time I go back to ml5.js, I am amazed at the simplicity of this great Javascript ML library. I found an old classification code using this library while rummaging through old codes. This library provides additional helper modules that can be used to do miscellaneous ML function. We can use a helper module present in this library to train classifiers. This creates a neural network model based on what kind of task you intend to do. Currently it supports classification, regression and imageClassification. Let’s get a refresher on each of these tasks in the next section.

Classification vs Regression

Classification and Regression are two types that fall under the heading of supervised learning. The key difference between these is that classification outputs discrete values i.e. defining classes while regression will provide a continuous value. An example of classification is segregating animals by type. For example, model should be able to identify between a fish, duck, seal or a deer. When we send an image of a duck to the algorithm, it should return me a classname of Duck. We can also classify a Yes/ No answer; Low, Medium or High severity etc. Image Classification is a special case of classification dealing with classifying images.

On the other hand, regression does not work on classifying objects. A classical example of regression algorithm is to predict housing prices. For example, a campground rental price may depend on following:

  • location
  • area provided
  • time of year/ season
  • amenities
  • access to facilities
  • type of vehicle
  • duration of stay

Given these variables, regression algorithm will be able to predict what a campground rental would be. This pricing will fit somewhere in the model as shown in diagram above.

We assess classification algorithm by accuracy. The more accurate results are, the better this model is. For example out of 100 samples, if prediction is correct for 95, we get a 95% accuracy.

On the other hand, we normally calculate regression accuracy using root mean square error (RMSE). This is the root of squares for sum of all deviations from correct result averaged over number of samples. The smaller the RMSE, better our model is.

RMSE = \sqrt{\left\frac{\sum_{i=1}^{N}(f-o)^2}{N}\right}f: forecasted value
o: observed value
N: number of samples
RMSE equation

What we plan to do

One of the most frequently used dataset for classification problem testing is the iris dataset. You can get this dataset by searching the repository here. Iris dataset was initially created by biologist Ronald Fischer. This dataset has 4 different variables petal_length, petal_width, sepal_length and sepal_width that define the final class of flower. There are three different irises that can be classified viz. setosa, virginica and versicolor. We will use this dataset to test ml5 neural network as a classifier.

I downloaded the dataset, and extracted out 3 records (one each per class) for test. So I am left with 147 records for training. I am putting some sample data below. This data contains Sepal Length (cm), Sepal Width (cm), Petal Length (cm), Petal Width (cm) and Iris Class in that order.

5.1,3.5,1.4,0.2,Iris-setosa
4.9,3.0,1.4,0.2,Iris-setosa
5.0,2.0,3.5,1.0,Iris-versicolor
5.9,3.0,4.2,1.5,Iris-versicolor
7.2,3.0,5.8,1.6,Iris-virginica
7.4,2.8,6.1,1.9,Iris-virginica

Start Coding

Let’s start by creating base html that we are going to use. We will create a file loader as the intent is to load iris data. Also we will keep options to test the samples that we separated out. We will not use any special libraries and rely on only HTML code. This is so that we can keep the page simple and just worry about the functionality. 

<table>
    <tr>
        <td><input id="fin" class="file-input" type="file"></input></td>
        <td><button id="ldset" onclick="loadFile()">Load DataSet</button></td>
    </tr>
    <tr>
        <td colspan="2"><textarea id="rtext" rows="25" cols="50"></textarea></td>
    </tr>
    <tr>
        <td colspan="2" style="text-align:center">
            <button id="trng" onclick="startTraining()">((( Start Training )))</button>
        </td>
    </tr>
</table>

Here the first thing we do is to create a file loader and a text area to show the uploaded file. We also have a button to initiate training.

<table>
    <tr>
        <td><button id="test1" onclick="irisTest1()">Setosa Test</button></td>
        <td><button id="test2" onclick="irisTest2()">Versicolor Test</button></td>
        <td><button id="test3" onclick="irisTest3()">Verginica Test</button></td>
        <td><button id="test4" onclick="randomTest()">Random Test</button></td>
    </tr>
</table>

This set just creates a few buttons that are used for testing classification post testing. In this case, the first three buttons will test the three records we had separated out. Next set of code is just to display the classification data that is returned by the model.

<table border="1">
    <tr>
        <th>Sepal Length</th>
        <th>Sepal Width</th>
        <th>Petal Length</th>
        <th>Petal Width</th>
    </tr>
    <tr>
        <td><label id="sl"></label></td>
        <td><label id="sw"></label></td>
        <td><label id="pl"></label></td>
        <td><label id="pw"></label></td>
    </tr>
    <tr>
        <td colspan="4"><label id="ic1"></label></td>
    </tr>
    <tr>
        <td colspan="4"><label id="ic2"></label></td>
    </tr>
    <tr>
        <td colspan="4"><label id="ic3"></label></td>
    </tr>
</table>

Adding functionality

The first thing we will be doing is to load the iris datasource file and show it in the text area. We will use the file loader in HTML to do this.

function loadFile() {
  var elem = document.getElementById("fin");
  var file = fin.files[0];
  if (!file) {
    return;
  }

  var reader = new FileReader();
  reader.onload = function(e) {
    var txt = e.target.result;
    showText(txt);
  };
  reader.readAsText(file);
}

function showText(txt) {
  var elem = document.getElementById("rtext");
  elem.textContent = txt;
}

After the data is loaded, we will start training against this data. For this we will need to parse the data to extract each record. The first step that we do is to initialize the neural network function. Next we will normalize the data we loaded. Normalizing data makes sure that we have unbiased network during training. Following this, we start our model training. For this sample we will only train to epoch 128. By that time we should have fairly usable model.

function startTraining() {
  var sarr = [];
  // Load the data
  var elem = document.getElementById("rtext");
  var larr = elem.textContent.split(/\r\n|\n/);
  larr.forEach(x => {
    flds = x.split(',');
    fld = {
      sepal_length: parseFloat(flds[0]),
      sepal_width: parseFloat(flds[1]),
      petal_length: parseFloat(flds[2]),
      petal_width: parseFloat(flds[3]),
      iris_class: flds[4]
    };
    sarr.push(fld);
  });

  // Neural Network Options
  const options = {
    inputs: ['sepal_length', 'sepal_width', 'petal_length', 'petal_width'],
    outputs: ['iris_class'],
    task: "classification",
    debug: true
  };

  // Initialize
  nn = ml5.neuralNetwork(options);

  sarr.forEach(item => {
    const inputs = {
      sepal_length: item.sepal_length, 
      sepal_width: item.sepal_width, 
      petal_length: item.petal_length,
      petal_width: item.petal_width
    };
    const output = {
      iris_class: item.iris_class
    };
    nn.addData(inputs, output);
  });

  // Normalize
  nn.normalizeData();

  // Train
  const trainingOptions = {
    epochs: 128,
    batchSize: 10
  }
  nn.train(trainingOptions, finishedTraining);
}

function finishedTraining(){
  alert("Training Complete");
}

In the code above, we read loaded file from text area. We then extract each line, followed by extracting each comma separated value. We then create a fixed struct for this data and push it to an array. Next we define the input and output for model training. Finally we start the training on this array of value.

One interesting aspect is the debug option for this network. If we set debug to true, the library shows the epoch to loss summary as a graph during the entire training. This makes it very easy to visualize how the selected layers worked. ml5.js assumes some default layers for training based on the type selected. According to definition here, default layers for classification includes two layers viz. a hidden layer with ReLU activation followed by an output layer with sofmax that classifies the outputs. In case of regression model, the output layer is replaced with a sigmoid activation function.

If needed, it is very easy to define our own custom layers. Let us create one more layer in middle and check how it affects performance.

ReLU + ReLU vs. Sigmoid + Sigmoid

Here we have used two different custom layers. The first one used ReLU as activation functions, and the second used sigmoid. As can be seen the sigmoid function took more epochs to converge, but was more stable overall after converging.

Conclusion

There are a lot more things that can be done with this module including loading different types of files for datasets, saving and loading trained models etc. I will keep the codes checked in to my GitHub here. Ciao for now!

Featured Image Copyright: Image by StockUnlimited

1 thought on “Neural Network in ml5.js”

Comments are closed.