Hello there, welcome to the blog. In this blog we will continue with our modeling of chess pieces identification. If you have not read the previous please follow link to previous blog post. In this section we will take the images we had labeled in previous section, upload them to our Google drive and start training on Google Colaboratory. Final target is of course to create a model to identify chess pieces as shown above.
Recap
We will start by recapitulating what we have already achieved in the previous section.
- We had started with getting images from Google for various chess pieces
- After this, we have labeled each image using LabelImg and grouped them into six different classes
- We created configurations for training Yolo
- We defined how we can train using darknet
Finally we left off with darknet command to train the model.
$ ./darknet detector train ./cfg/trainyolo.data ./cfg/myyolov3.cfg \ ./cfg/darknet53.conv.74
For this session we will start with getting our data to the cloud and start training.
Prepping data
First you need to create any appropriate directory structure for the project. I am giving a diagram of how I had put all my directories.
- darknet: This directory will later have a pre-built version of darknet, so we do not have to build it every time we start
- backup: Empty directory, will contain intermediate weights generated during training
- cfg: Yolo configuration files for darknet. It contains the following files we discussed during our last session
- test.txt
- train.txt
- trainyolo.data
- myyolov3.cfg
- darknet53.conv.74
- scaleimg: This contains all images along with the bounding box data. Please refer to previous blog for details
Upload all relevant images to scaleimg directory and all configurations to cfg directory. We will have rest of the directories empty.
Build darknet for GPU
For the previous session we had taken the darknet version from pjreddie. However, in this case we will use a different clone from AlexeyAB.
Open Google Colaboratory by going to this link on the browser. Login using your user ID and you will get a new Python Jupyter notebook. Rest of all our commands we will execute on this environment.
When opening Google Colab and getting a machine make sure to enable GPU. You can do this be selecting Change Runtime Type from Runtime menu. See adjoining image. If GPU is not enabled, we will not be able to use Nvidia GPU compiler.
Let us first get access to our Google Drive from this environment. We will use a library provide by Google to mount this as a drive. After mounting, drive will be available under /content/gdrive/My Drive/.
from google.colab import drive
drive.mount('/content/gdrive')
!ls "/content/gdrive/My Drive/yolotrain/"Check if we have GPU. You should also be able to query version for CUDA compiler.
!/usr/local/cuda/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2019 NVIDIA Corporation Built on Sun_Jul_28_19:07:16_PDT_2019 Cuda compilation tools, release 10.1, V10.1.243
Following this, we will do two things. First, we will install the OpenCV libraries on this machine. Next we will get the darknet source from GitHub.
!apt install libopencv-dev python-opencv !git clone https://github.com/AlexeyAB/darknet/ %cd darknet/
The first command will download opencv and install. Second command will clone the darknet repository into ./darknet directory. We finally cd into this directory.
!sed -i 's/OPENCV=0/OPENCV=1/g' Makefile !sed -i 's/GPU=0/GPU=1/g' Makefile !sed -i 's/i%10000/i%100/g' src/detector.c
Here we are changing some configurations. We enable OPENCV and GPU in this build. Also, by default, darknet saves the model every 100 iterations till 900, but then saves every 1000 iterations. We want to make sure to save it every 100 iterations beyond 900. Third line on top makes relevant changes to detector.c file.
Build Darknet
This just takes one command, make!
!make
Now that we have changed all configurations that were needed to be changed, the final step is to run a build. You should see a lot of warnings, but finally build will complete and you should have a file called darknet in current directory.
!cp ./darknet "/content/gdrive/My Drive/yolotrain/darknet/darknet"
We will keep this binary file backed up in Google drive under ./darknet directory so that we can copy it next time – instead of building it again.
Copy your files
We have darknet built now. Next step is to get the files we stored in Google drive to this machine.
!cp -R "/content/gdrive/My Drive/yolotrain/resources/" . %cd resources/ !cp "/content/gdrive/My Drive/yolotrain/darknet/darknet" . !chmod u=rx darknet
Let’s analyze the commands above. Here we copy the files we had in resources to this machine. If you remember from above, it contains all our images with bounding box and Yolo configurations for darknet build. We then cd into the directory created, copy the built darknet file here and change it to be executable.
Finally, after all of these pre-steps, we are ready to train.
Train
!./darknet detector train -dont_show ./cfg/trainyolo.data \ ./cfg/myyolov3.cfg ./cfg/darknet53.conv.74
This should capture some GPU instances and start training. You will see iterations running for the training and loss for that iteration displayed. Training should be allowed continue till the loss between consecutive iteration doesn’t seem to change by too much.
This will also keep a backup of the generated weights file every 100 iterations. Make sure to copy this file over to your Google drive if the process terminates.
!cp backup/myyolov3_last.weights \ "/content/gdrive/My Drive/yolotrain/resources/backup/myyolov3_last.weights"
If you want to resume training from where it stopped, just feed the last Yolo weights file that was saved as the last parameter.
!./darknet detector train -dont_show ./cfg/trainyolo.data \ ./cfg/myyolov3.cfg ./backup/myyolov3_last.weights
Be prepared for a long time of training. My model started detecting objects with some amount of confidence only after 4,000 iteration. The loss at that point was about 0.2. However, I did not want to continue anymore, as I could deem my experiment as successful at that point.
Testing the Model
We will write a small Python module to test the model. First, we will import required libraries.
import cv2 import numpy as np
We will take all file locations as parameters to this class. This way the class itself does not need to know about any of the locations.
class DemoRun:
def __init__(self, lbf, cfg, wgt):
self.label_file = lbf
self.config_file = cfg
self.weights = wgt
self.color = (0, 0, 255)Next we will load the Yolo model and class names.
def start_test(self, arrimg):
# Labels
labels = open(self.label_file).read().strip().split("\n")
print("Labels: {}".format(labels))
# Darknet
net = cv2.dnn.readNetFromDarknet(self.config_file, self.weights)
ln = net.getLayerNames()
print("Layers: {}".format(ln))
ln = [ln[i[0] - 1] for i in net.getUnconnectedOutLayers()]
print("Unconnected Layers: {}".format(ln))Finally, we will start our processing for the sent in images. I am also adding the class name to identified image – so we know what objects were identified.
# Now identify images
for imgt in arrimg:
print("Processing Image: {}".format(imgt))
img = cv2.imread(imgt)
imgh, imgw, _ = img.shape
blob = cv2.dnn.blobFromImage(img, 1 / 255.0, (608, 608), swapRB=True, crop=False)
net.setInput(blob)
layers = net.forward(ln)
allclz = {}
for output in layers:
for detection in output:
scores = detection[5:]
classID = np.argmax(scores)
confidence = round(100*scores[classID], 2)
if confidence > 60:
if classID in allclz:
if allclz[classID]['confidence'] < confidence:
allclz[classID] = {'confidence': confidence, 'center': detection[0:2]}
else:
allclz[classID] = {'confidence': confidence, 'center': detection[0:2]}
for key in allclz:
print("\tClassID: {}, confidence: {}%, center: {}, {}"
.format(labels[key], allclz[key]['confidence'],
allclz[key]['center'][0]*imgw, allclz[key]['center'][1]*imgh))
ccoord = (int(allclz[key]['center'][0]*imgw),
int(allclz[key]['center'][1]*imgh))
cv2.putText(img, labels[key], ccoord, cv2.FONT_HERSHEY_SIMPLEX,
0.5, self.color, 2)
cv2.imshow("Image", img)
cv2.waitKey(0)
cv2.destroyAllWindows()Main module calls this function by initializing all parameters.
if __name__ == "__main__":
dr = DemoRun(
"../resources/yolofiles/classes.txt",
"../resources/yolofiles/myyolov3.cfg",
"../resources/yolofiles/myyolov3_last4300.weights"
)
dr.start_test(
["../resources/testimg/test01.jpg",
"../resources/testimg/test02.jpg",
"../resources/testimg/test03.jpg"
]
)Conclusion
The image that we started with is an image identified by the program. Remember, we only had run 4,300 iterations. Let us try one more image to finish it off.
Ciao for now! I will put the code in my GitHub directory for reference. Hope you found this useful.
All images are copyright their respective owners. I do not own the images, they are used for demonstration only.