Microsoft Project15 & University College London Red Panda Project

Published Apr 21 2021 09:37 AM 1,757 Views
Microsoft

Guest post by Farid El-Aouadi Computer Science Student, Microsoft Project 15 Capstone Project with Farid El-Aouadi at University College London 


pic 

http://faridelaouadi.tech/


Introduction

 

Founded in 2007, Red Panda Network has become a world leader in efforts to protect Red Pandas and their habitat. Using a variety of techniques such as community based conservation and developing anti poaching networks, they are committed to saving every last panda that is currently living in the Himalayan mountains.

Since my first meeting with Sonam Lama, Red Panda Network it was evident that their current workflow lacked any sophistication and revolved heavily around human labour. From manually classifying and retrieving images to making in-field observations using pen and paper, it was clear that the workflow needed a flair of innovation to allow the organisation to work smarter and not harder. 

 

The Red Panda Network’s current workflow to track red panda’s in the mountains is illustrated in the diagram below. The process can be decomposed into two main sub processes : image retrieval from camera traps + classification and data entry + analysis. 

 

 

RedPanda.png


Process Analysis

Image retrieval and classification

Currently, the organisation has several camera traps in the mountains. These motion activated cameras take pictures when motion is detected in their vicinity and saves the images to the local SD card. As you can imagine, these camera traps can capture several false positive images ( when the motion is thought to be a red panda when it is not ) . Periodically, a member of the red panda network team has to go to the mountains, download the SD card data then classify these images as being red panda or not.

 

To put things into perspective, the Red Panda Network currently has around 20 cameras that took a total of 55,000 images over approximately 2 months - that is a lot of manual classifying! The images then get sent to the red panda headquarters where a spreadsheet is updated for further analysis later on.

 

Data Entry

The Red Panda network has a team of volunteers called the forest guardians. Forest Guardians are local people who are paid to monitor and protect red panda habitat, as well as educate communities. They currently do all their data gathering on paper and manually relay the information to the RPN HQ. The data entry of the manual observations is from pencil and paper which is prone to errors, inaccuracies and can unnecessarily take a long time. 

 

As you can probably see, both processes can benefit greatly from a technological solution. I will primarily be focusing my efforts on the first problem for two main reasons. The first is that this problem can make use of an elegant solution that incorporates AI, IoT as well as cloud computing all using the azure platform. The second reason is that by tackling this problem, I will be saving the employees a lot of time which would allow them to focus their time in other conservation efforts that the red panda network is involved in. 


Proposed solution and workflow

 

 

RedPandaArchitecture.jpg

 

My proposed solution leverages the following Azure services:

 

In the proposed workflow, users will first be authenticated using Active directory. Once authenticated, users will be able to upload images associated with a specific camera trap and view them on an azure map. The web app will be responsive which will also allow for forest guardians to upload images straight from their phone. Once they have uploaded the images, they will be stored in azure blob storage with the relevant metadata stored in an azure table. When the user uploads an image, they will be classified as being “panda” or “not panda” using a pre-trained classifier and display them on the azure map to reflect their classification. 


Tech stack

 

 

 

RedPandaTechStack.jpg

 

From an application development perspective, I will be creating the server using python flask and writing any frontend logic using Javascript. Flask is a lightweight WSGI web application framework and chosen for this project as I have experience using it for smaller projects and found it to be well documented and supported. Javascript was chosen ( as opposed to a Javascript library like React or Angular ) as I felt that I was comfortable enough using Javascript and that learning a library would be time consuming and not provide any extra value to the end user. Jinja is a web template engine for the Python programming language that I will be using to build HTML pages on the server. This allows me to reuse HTML code to make the templates more readable for future developers.

 

For ease of development and deployment, I will be using a CI/CD pipeline. Using Azure pipelines, it will be extremely easy for me to make changes to the source code, push it to my repository on GitHub then deploy it so that users can see these changes within a few minutes. I can also define tests that need to pass before the production code gets changed which will uphold code correctness and platform availability. 

 

To ensure the design of the app adheres to Don Norman’s “10 usability heuristics for user interface design”, I will be using bootstrap for the styling of all my components. This will ensure the UI is consistent, learnable and promotes recognition rather than recall. 

 


Final platform

Main dashboard

RedPandaMainDash.jpg

 

 

Add a camera

RedPandaCamera.jpg

 

View Individual panda (Click on panda icon)

 

RedPandaCamera1.jpg

 

View images from a camera trap

RedPandaCameraTrap.jpg

 

Upload New Images

 

RedPandaUploadNew.jpg

 

Classification report after model inference

RedPandaClassification.jpg

 

Full Video demo 

 

Source Code

faridelaouadi/RedPandaNetwork: A web app created for the red panda network in collab with Microsoft'...


Technical challenges


Online classifier limit 

Initially, to classify user uploaded images I was using the customvision.ai API to make HTTP requests that returned the classification. This worked fine for a few images however during my testing, I decided to upload a batch of images that consisted of 15 images. Each image took around 3 seconds to get classified which meant that the user would be stuck on the loading screen for far too long. The logical next step would be to make asynchronous calls to this API to drastically reduce the wait time. The wait time was indeed reduced however this optimization exposed another problem about the API and its rate limit - users could make a maximum of 10 calls per second. To try and work around this, I simulated a pause in my code after every 10 asynchronous calls. This seemed like a crude solution whose runtime was still linear. 

 

To improve upon this, I decided that the best way to tackle this problem would be to download the model locally and classify the images on the server. However this brought about its own challenges as I now had to do image preprocessing to ensure that the user uploaded images were compatible with the trained model. To preprocess the images I made use of the “OpenCV” and “Numpy” modules in python which provided a simple API to scale, crop, resize and update orientation of images of any format. I then made asynchronous calls to this local model which yielded excellent results! 

 

Azure maps 

Azure maps provided a steep learning curve especially when trying to add custom features such as filtering and onclick functionality to pandas and cameras that acted differently. When tackling the filtering, I had to add unique attributes to each of the cameras that were being displayed on the map. I then created a filter list that was then set to act on the camera symbol layer of the map. For the onclick functionality, I wanted to constantly reuse the same bootstrap modal for each of the cameras. To do this, I had to once again add further attributes to the camera features on the map and run a  Javascript function that built the modal on the fly by extracting the attributes from the selected camera to hence make an ajax request to the server to retrieve the relevant data. 

 

 

Future plan

 

Online learning

 

The next challenge I am working on is making the classifier improve itself as the user uploads more images ( and corrects misclassifications ) using the system. To do this I will make use of Azure functions that would get triggered when new images have been uploaded to the blob storage. This azure function would then retrain the classifier with these new images as additional training data.
There will also be logic in the azure function that prevents the classifier being re-trained for every new image as this would get costly and tedious. There would also need to be logic to prevent overfitting so if your interested in learning more about the project join me at the following webinar.

 

Webinar 22nd April 2021

 

Project 15: Empowering all to preserve endangered species & habitats.

Date: April 22, 2021

Time: 08:00 AM - 09:00 AM (Pacific)

Format: Livestream

Topic: DevOps and Developer Tools


Join Microsoft Project 15 and students from University College London live on Earth Day (April 22) at 8 am PT to explore some of the ways the tech community is using IoT solutions to further preservation efforts. Get involved – register for the event now.  

 

 

Co-Authors
Version history
Last update:
‎May 11 2021 11:56 AM
Updated by: