MacBook Pro M1 vs old MacBook Pro

Finally I got delivered my MacBook Pro with the M1 chip. It replaces my old 2016 MacBook Pro. The first impression of this MacBook is very positive. It came mostly charged so I could start using it immediately. Browsing was blazing fast, connectivity to my 5 GHz wifi network was very stable without any issues. Oh, and I love the battery life!

Then I encountered my first issue. Docker is not supported yet. However, I found that it is in the works and I can get a version if I register for developer previews. Maybe I will be doing that in the next few days.

However, I wanted to see how much better is this processor from my current version. It promises much faster ML trainings given that it has its own GPU and Neural Engine already embedded in its core. I wanted to put that to test so came up with a very simple swift code. Since tensorflow may not be optimized, I went with Xcode for both cases. I have a simple swift playground to do image classification. Both of the machines are running Big Sur 11.1. The new MacBook definitely has advantage on both CPU and Memory aspects, going up from 4 GB to all the way to 16GB on the new one. The old MacBook also has all my codes on a SD card, so I/O will be a limiting factor. As such the test is definitely not an apples to apples compare. Instead of comparing, it is better to just take the new MacBook test results on its face value.

The project

I downloaded Intel image classification dataset from Kaggle. This contains data of natural scenes in the world. There are six classes of data viz. buildings, forest, glacier, mountain, sea and street. There are around 14K images in this dataset each of 150×150 px dimensions. All images are split in test and training datasets and are in specific directories for each label. Now let’s get on with the swift code for classifier.

import CreateML
import Foundation

// Image Paths
let trnImgPath = URL(fileURLWithPath: "/Classification/train/")
let tstImgPath = URL(fileURLWithPath: "/Classification/test/")
print("Training Image Path: ", trnImgPath)
print("Test Image Path: ", tstImgPath)

// Start Training
var start = DispatchTime.now()
let model = try MLImageClassifier(trainingData: .labeledDirectories(at: trnImgPath))
var end = DispatchTime.now()
var elapsed = end.uptimeNanoseconds - start.uptimeNanoseconds
print("Elapsed Training (ms): ", Double(elapsed)/1000000.00)

// Start Testing
start = DispatchTime.now()
let evaluation = model.evaluation(on: .labeledDirectories(at: tstImgPath))
end = DispatchTime.now()
elapsed = end.uptimeNanoseconds - start.uptimeNanoseconds
print("Elapsed Test (ms): ", Double(elapsed)/1000000.00)

// Save Model
try model.write(to: URL(fileURLWithPath: "/Classification/model/intel.mlmodel"))

Here we are using the MLImageClassifier which is part of Apple Core ML function. We are also recording the total time taken – so that we can do a comparison.

The Result

The results were amazing. Let me add the results here. I have removed most of intermediate data to keep this short. First on the list is data from old computer.

Extracting image features from training data.
Analyzing and extracting image features.
+------------------+--------------+------------------+
| Images Processed | Elapsed Time | Percent Complete |
+------------------+--------------+------------------+
| 1                | 1.92s        | 0%               |
| 2                | 2.03s        | 0%               |
| 10600            | 19m 18s      | 99%              |
| 10650            | 19m 23s      | 99.5%            |
| 10687            | 19m 27s      | 100%             |
+------------------+--------------+------------------+
Extracting image features from user validation data.
Analyzing and extracting image features.
+------------------+--------------+------------------+
| Images Processed | Elapsed Time | Percent Complete |
+------------------+--------------+------------------+
| 1                | 120.495ms    | 0%               |
| 2                | 232.121ms    | 0.25%            |
| 500              | 54.09s       | 92.25%           |
| 541              | 58.51s       | 100%             |
+------------------+--------------+------------------+
Beginning model training on processed features. 
Calibrating solver; this may take some time.
+-----------+--------------+-------------------+---------------------+
| Iteration | Elapsed Time | Training Accuracy | Validation Accuracy |
+-----------+--------------+-------------------+---------------------+
| 0         | 1.292316     | 0.752503          | 0.768946            |
| 1         | 2.516076     | 0.803967          | 0.826248            |
| 2         | 4.733388     | 0.851876          | 0.855823            |
| 19        | 20.531309    | 0.931786          | 0.926063            |
| 24        | 26.077530    | 0.935061          | 0.937153            |
+-----------+--------------+-------------------+---------------------+
Elapsed Training (ms):  1271376.911218
Extracting image features from evaluation data.
Analyzing and extracting image features.
+------------------+--------------+------------------+
| Images Processed | Elapsed Time | Percent Complete |
+------------------+--------------+------------------+
| 1                | 128.457ms    | 0%               |
| 2                | 255.174ms    | 0%               |
| 2800             | 5m 18s       | 99.75%           |
| 2806             | 5m 18s       | 100%             |
+------------------+--------------+------------------+
Elapsed Test (ms):  323880.458201

MacBook Pro M1

Here is data from new M1 chip. As stated above, this has more memory (16GB) and the code is on native SSD.

Extracting image features from training data.
Analyzing and extracting image features.
+------------------+--------------+------------------+
| Images Processed | Elapsed Time | Percent Complete |
+------------------+--------------+------------------+
| 1                | 2.32s        | 0%               |
| 2                | 2.32s        | 0%               |
| 10000            | 34.41s       | 93.5%            |
| 10687            | 36.56s       | 100%             |
+------------------+--------------+------------------+
Extracting image features from user validation data.
Analyzing and extracting image features.
+------------------+--------------+------------------+
| Images Processed | Elapsed Time | Percent Complete |
+------------------+--------------+------------------+
| 1                | 5.915ms      | 0%               |
| 2                | 11.592ms     | 0.25%            |
| 500              | 1.64s        | 92.25%           |
| 541              | 1.77s        | 100%             |
+------------------+--------------+------------------+
Beginning model training on processed features. 
Calibrating solver; this may take some time.
+-----------+--------------+-------------------+---------------------+
| Iteration | Elapsed Time | Training Accuracy | Validation Accuracy |
+-----------+--------------+-------------------+---------------------+
| 0         | 0.251375     | 0.755497          | 0.768946            |
| 1         | 0.536501     | 0.835033          | 0.841035            |
| 2         | 1.066126     | 0.859362          | 0.870610            |
| 19        | 4.894475     | 0.931225          | 0.922366            |
| 24        | 5.963021     | 0.937213          | 0.935305            |
+-----------+--------------+-------------------+---------------------+
Elapsed Training (ms):  46906.3615
Extracting image features from evaluation data.
Analyzing and extracting image features.
+------------------+--------------+------------------+
| Images Processed | Elapsed Time | Percent Complete |
+------------------+--------------+------------------+
| 1                | 4.601ms      | 0%               |
| 2                | 8.02ms       | 0%               |
| 2000             | 6.39s        | 71.25%           |
| 2806             | 8.94s        | 100%             |
+------------------+--------------+------------------+
Elapsed Test (ms):  9824.890041

Head to Head

That made me feel like my old MacBook Pro as a low end machine. Coming to think about it, I never had performance problems with any of my regular developments on my old MacBook Pro, but then I was probably ready to wait. I remember waiting for OpenCV to build on that MacBook, and it felt like eternity. Now to wait and see when tensorflow starts supporting the new chip so that I can use this machine for training. Ciao for now!

Featured Image Copyright: Image by StockUnlimited