Welcome to the second part of the learning series “Serverless image classification with Azure Functions and Custom Vision”!
About me
Hi, I am Foteini Savvidou, a Gold Microsoft Learn Student Ambassador!
I am an undergraduate Electrical and Computer Engineering student at Aristotle University of Thessaloniki (Greece) interested in wireless communications, IoT, AI, cloud technologies and biomedical engineering. Always passionate about teaching and learning new things, I love helping people expand their technical skills through organizing workshops and sharing articles on my blog.
Introduction
In the previous article, you built an image classification model that predicts whether a photo contains a dog or a cat using the Azure Custom Vision service and the Python SDK.
In this article, you are going to export your image classifier using the Custom Vision SDK for Python and use the model locally to classify images. You will learn how to:
- Export a Custom Vision model using the Python client library and Visual Studio Code.
- Run the exported TensorFlow model locally to classify images.
To complete the exercise, you will need:
- An Azure subscription. If you don’t have one, you can sign up for an Azure for Student or an Azure free account.
- A Custom Vision resource.
- A trained image classification model in Azure Custom Vision.
You will also need to install Python 3 and Visual Studio Code or another code editor.
Note: To export a Custom Vision model, you should use a Compact domain. Compact domains are optimized for real-time classification and object detection on edge devices.
Set up your application
Create a configuration file
Create a configuration file (.env) and save the key and the endpoint of your training resource and the id of your project and the trained iteration you wish to download.
Create a new Python application
Want to view the whole code at once? You can find it on GitHub.
- Create a new Python file (export_model.py) and import the following libraries:
from azure.cognitiveservices.vision.customvision.training import CustomVisionTrainingClient from msrest.authentication import ApiKeyCredentials from dotenv import load_dotenv import os, time, requests, zipfile - Add the following code to load the values from the configuration file.
load_dotenv() training_endpoint = os.getenv('TRAINING_ENDPOINT') training_key = os.getenv('TRAINING_KEY') project_id = os.getenv('PROJECT_ID') iteration_id = os.getenv('ITERATION_ID')
Export an image classifier
Create a training client
Use the following code to create a CustomVisionTrainingClient object. You will use the trainer object to export your model in one of the available formats.
credentials = ApiKeyCredentials(in_headers={"Training-key": training_key})
trainer = CustomVisionTrainingClient(training_endpoint, credentials)
Export your model
Add the following code to export the trained iteration to a TensorFlow file and download the exported model. For more information about the export_iteration method, see the Custom Vision SDK for Python documentation.
platform = "TensorFlow"
flavor = "TensorFlow"
export = trainer.export_iteration(project_id, iteration_id, platform, flavor, raw=False)
while (export.status == "Exporting"):
print("Waiting 10 seconds...")
time.sleep(10)
exports = trainer.get_exports(project_id, iteration_id)
# Find the export for this iteration
for e in exports:
if e.platform == export.platform and e.flavor == export.flavor:
export = e
break
print("Export status is: ", export.status)
if export.status == "Done":
# Download the model
export_file = requests.get(export.download_uri)
with open("export.zip", "wb") as file:
file.write(export_file.content)
Note: If you've already exported a trained iteration in a certain format, you cannot call the export_iteration method again. Instead, use the get_exports method to download the existing exported model.
Then, use the following code to unzip the downloaded file.
if not os.path.exists("./model"):
os.mkdir("./model");
zip_ref = zipfile.ZipFile("export.zip", 'r')
zip_ref.extractall("./model")
zip_ref.close()
print("Data extracted in: ./model")
Use the model to classify images
The downloaded .zip file contains a model.pb and a labels.txt file. These files represent the trained model and the classification labels respectively. The following code loads the model and the labels into your project and performs image manipulation to prepare an image for prediction. The code for image preprocessing was derived from Microsoft Docs.
import tensorflow as tf
from PIL import Image
from urllib.request import urlopen
import numpy as np
import os, cv2
graph_def = tf.compat.v1.GraphDef()
labels = []
network_input_size = 0
output_layer = 'loss:0'
input_node = 'Placeholder:0'
basedir = os.path.dirname(__file__)
filename = os.path.join(basedir, "model", "model.pb")
labels_filename = os.path.join(basedir, "model", "labels.txt")
def convert_to_opencv(image):
# RGB -> BGR conversion is performed as well.
image = image.convert('RGB')
r,g,b = np.array(image).T
opencv_image = np.array([b,g,r]).transpose()
return opencv_image
def crop_center(img,cropx,cropy):
h, w = img.shape[:2]
startx = w//2-(cropx//2)
starty = h//2-(cropy//2)
return img[starty:starty+cropy, startx:startx+cropx]
def resize_down_to_1600_max_dim(image):
h, w = image.shape[:2]
if (h < 1600 and w < 1600):
return image
new_size = (1600 * w // h, 1600) if (h > w) else (1600, 1600 * h // w)
return cv2.resize(image, new_size, interpolation = cv2.INTER_LINEAR)
def resize_to_256_square(image):
h, w = image.shape[:2]
return cv2.resize(image, (256, 256), interpolation = cv2.INTER_LINEAR)
def update_orientation(image):
exif_orientation_tag = 0x0112
if hasattr(image, '_getexif'):
exif = image._getexif()
if (exif != None and exif_orientation_tag in exif):
orientation = exif.get(exif_orientation_tag, 1)
# orientation is 1 based, shift to zero based and flip/transpose based on 0-based values
orientation -= 1
if orientation >= 4:
image = image.transpose(Image.TRANSPOSE)
if orientation == 2 or orientation == 3 or orientation == 6 or orientation == 7:
image = image.transpose(Image.FLIP_TOP_BOTTOM)
if orientation == 1 or orientation == 2 or orientation == 5 or orientation == 6:
image = image.transpose(Image.FLIP_LEFT_RIGHT)
return image
def initialize():
global labels, network_input_size
# Import the TF graph
with tf.io.gfile.GFile(filename, 'rb') as f:
graph_def.ParseFromString(f.read())
tf.import_graph_def(graph_def, name='')
# Create a list of labels.
with open(labels_filename, 'rt') as lf:
labels = [l.strip() for l in lf.readlines()]
# Get the input size of the model
with tf.compat.v1.Session() as sess:
input_tensor_shape = sess.graph.get_tensor_by_name('Placeholder:0').shape.as_list()
network_input_size = input_tensor_shape[1]
def predict_image(image):
initialize()
# Update orientation based on EXIF tags, if the file has orientation info.
image = update_orientation(image)
# Convert to OpenCV format
image = convert_to_opencv(image)
# If the image has either w or h greater than 1600 we resize it down respecting
# aspect ratio such that the largest dimension is 1600
image = resize_down_to_1600_max_dim(image)
# We next get the largest center square
h, w = image.shape[:2]
min_dim = min(w,h)
max_square_image = crop_center(image, min_dim, min_dim)
# Resize that square down to 256x256
augmented_image = resize_to_256_square(max_square_image)
# Crop the center for the specified network_input_Size
augmented_image = crop_center(augmented_image, network_input_size, network_input_size)
with tf.compat.v1.Session() as sess:
try:
prob_tensor = sess.graph.get_tensor_by_name(output_layer)
predictions = sess.run(prob_tensor, {input_node: [augmented_image] })
except KeyError:
print ("Couldn't find classification output layer: " + output_layer + ".")
exit(-1)
highest_probability_index = np.argmax(predictions)
return {"label": labels[highest_probability_index], "probability": predictions[0][highest_probability_index]}
def predict_image_from_url(image_url):
with urlopen(image_url) as imageFile:
image = Image.open(imageFile)
return predict_image(image)
def main():
# Load from a file
print('Predicting from local file...')
imageFile = os.path.join(basedir, "images", "4.jpg")
image = Image.open(imageFile)
prediction = predict_image(image)
print(f"Classified as: {prediction.get('label')}")
print(f"Probability: {prediction.get('probability')*100 :.3f}%")
print('Predicting from url...')
image_url = "<IMAGE_URL>"
prediction = predict_image_from_url(image_url)
print(f"Classified as: {prediction.get('label')}")
print(f"Probability: {prediction.get('probability')*100 :.3f}%")
if __name__ == '__main__':
main()
- The
predict_image()function calls theinitialize()function to load the model and the labels, performs image preprocessing to prepare the image for prediction, runs the image through the model, and returns the label with the highest confidence score and its probability. - The
predict_image_from_url()function opens an image from the specified URL and calls thepredict_image()function to get the predicted label.
Summary and next steps
In this article, you learned how to export a Custom Vision model using the client library for Python and run a test image through the model. In the next part of this series, you will learn how to import the exported TensorFlow model into an Azure Function and build a serverless HTTP API for classifying images.
If you are interested in integrating your exported model into an application, you may check out the following resources:
- Use a TensorFlow model exported from Azure Custom Vision
- Sample scripts for exported models from Custom Vision Service
- Custom Vision Android sample applications
Clean-up
If you want to delete this project, navigate to the Custom Vision project gallery page and select the trash icon under the project.
If you have finished learning, you can delete the resource group from your Azure subscription:
- In the Azure portal, select Resource groups on the right menu and then select the resource group that you have created.
- Click Delete resource group.