Following my last post, where I build a CNN model to detect 11 different mediterranean butterflies. Here I will explain how ended up making an APK for android with a built in tensorflow model.
After several tries trying to optimize the model, finally I got a pretty accurate model, capable of recognizing most of the 11 butterflies. It is still not perfect, and certainly I would need more time to make it 100% accurate.
I collected 200 images of each of the following butterflies:
- Vanessa Atalanta
- Melanargia Ines
- Maniola Jurtina
- Lysandra Bellargus
- Issoria Lathonia
- Iphiclides Feisthamelii
- Gonepterix Cleopatra
- Colias Croceus
- Coenonympha Pamphilus
- Aricia Cramera
- Anthocharis Cardamines
And 200 random images, so the model can label as "Unknown", everything that is not one of these 11 butterflies.
I did some data augmentation, so the model can train the images in different positions and shapes in each epoch, and I used tensor flow to build the model, as explained in my later post, I used VGG16 for transfer learning.
Before getting to the android part, I had to build a Graphic User Interface (GUI), using Kivy. And I used my computer's camera to take photos of butterflies, which were displayed using my smartphone.
Once the script was done, and well structured with the right directories, I had to compile it for android. To perform this task I used buildozer.
Now comes the tricky part... Every time I compiled my script to an APK, I was having issues when opening it on my smartphone. Of course, I was not able to see the errors, so I decided to use Android Studio, which works as a great debugging tool for APKs.
The errors were endless, and I spent a lot of time debugging the APK. I got stuck on a very particular error, which in the end I found out that it was related to tensorflow. In fact, android doesn't read tensorflow models. The way I found to bypass this issue was by converting the TF model into TF lite, basically it packages the model's directory in one single .tflite file.
After some finally adjustments on the script and the .spec file (file needed to compile using buildozer), the APK was finally working. See below some images of it working on Samsung A20e:
For the remark, using the smartphone to catch butterflies isn't indeed the best option, it gets quite difficult (also the capturing feature of the app isn't great). A better approach would be to upload pictures of butterflies and make the prediction.