Troubleshooting blank frames on Azure Percept Device Kit

Introduction

 

I recently bought an Azure Percept Development Kit. The Out-of-Box experience is easy; from setup, connecting to Azure cloud and getting AI running on video frames, it’s seamless. I followed the Quickstart guide and Setup experience and I’m running in no time. Azure Percept Studio monitors and manages edge devices remotely. Everything was working fine ok until one day, I checked the web stream on Azure Percept Studio and it was receiving blank frames. I am happy that there are resources to help troubleshoot Azure Percept Device Kit.

Here’s my recent experience troubleshooting the dev kit and what I did to resolve an issue.

 

 

Troubleshooting

 

Once logged into Azure Percept Studio, under the Devices tab, It listed all registered Percept devices. Here is where I checked the status of the device. If the device status is connected, it means Azure can send and receive messages from the device. If the device is off or disconnected from the internet, the device status will be disconnected.

 

 

 

I can click on the device name, in this case percept-device-01 and get more information about the device. Go to the Vision tab to view the device stream. After a few seconds, it will prompt the link to view the video stream. Note: your laptop has to be in the same network as the device so it can view the video stream.

 

 

The device stream typically shows the camera feed. This time though, the image is blank and not receiving any video frames. The camera is pointed at something, but the stream is blank. Uh-oh, at this point I’m not sure what happened.

 

 

 

I clicked the “View Live Telemetry” button and checked if Azure is receiving telemetry results from the device. I waited for a few minutes, it’s still empty.

 

Then I clicked on the “Open device in IoT Hub” button.

 

 

It opened up the IoT Hub page and then I saw the azureeyemodule runtime status is getting an error. I read through the troubleshooting guide for Vision Solution and started to research about azureeyemodule; it’s the IoT module responsible for running the AI workload on the Percept DK. It is responsible for sending those live telemetry data.  

Open source on github at https://github.com/microsoft/azure-percept-advanced-development/

 

Why is it having an error?  There’s a “Troubleshoot” button so I can see logs from the IoT module. I selected azureeyemodule and clicked on the “Restart azureeyemodule”. Clicked the “Refresh” button to view the logs. It gave me a little clue on what’s happening on the device. It might be the device itself.

 

 

 

I read through the Azure Percept DK troubleshooting guide and found out that I can SSH into it. I followed this guide: Connect to your Azure Percept DK over SSH. I know the ip address of the dev kit because of the webstream viewer. During the Azure Percept Dev Kit setup, it asked to assign username and password. That’s what I used to login.

 

Interesting to learn that it’s using CBL-Mariner linux distribution. I ran the “iotedge list” command, this would list status of each IoT Edge module.

 

azureeyemodule has status of failed, everything else is running.

 

I checked the usb cable if it’s properly connected to the camera. It’s plugged in properly.

I verified if the operating system can see the camera. I used the ‘usb-devices’ command that gave information about Azure Eye SoM Controller.

This means the usb camera is connected properly.

 

At this point, I was stuck. Not sure how to solve the issue. I took a break.

 

The Solution

 

After a few days, I checked for other cables that connect to the camera itself. I wiggled it a little bit and it’s loose! The camera flex ribbon cable was disconnected. It doesn’t look obvious because the cable is under a massive heatsink.

 

 

So I unscrewed the plate covers, 4 screws. I’m really happy that the Azure percept DK provides the hex tool.  I plugged the camera to the right side of the board. I noticed there’s another plug on the left side, I wonder if it can actually process 2 cameras.

 

 

After assembling it all back. I rebooted the machine. I got back to the Azure Percept Studio and checked device status. Then checked the device in the IoT Hub Portal, azureeyemodule is running again.

 

I verified the webstream video is receiving video frames, and also live telemetry data.

 

Conclusion

 

As a cloud engineer, my troubleshooting process is a little bit different. I started from the Azure cloud side, checked data communications between cloud and device first in Azure Percept Studio and IoT Hub; I logged into the edge device and checked the software module if it is working and then checked camera drivers. Then I checked the camera driver, then the camera itself. If I was a hardware engineer, most likely I would have checked the camera first and would have solved it sooner.

 

Resources

• Learn more about Azure Percept development: http://aka.ms/getazurepercept.
• Quick Start Guide: https://docs.microsoft.com/en-us/azure/azure-percept/quickstart-percept-dk-unboxing
• Setup Guide: https://docs.microsoft.com/en-us/azure/azure-percept/quickstart-percept-dk-set-up 
• Azure Percept DK setup experience troubleshooting guide: https://docs.microsoft.com/en-us/azure/azure-percept/how-to-troubleshoot-setup
• General Azure Percept DK Troubleshooting guide: https://docs.microsoft.com/en-us/azure/azure-percept/troubleshoot-dev-kit
• Vision solution troubleshooting: https://docs.microsoft.com/en-us/azure/azure-percept/vision-solution-troubleshooting
  
  

Ron Dagdag

Lead Software Engineer

During the day, Ron Dagdag is a Lead Software Engineer with 20+ years of experience working on a number of business applications using a diverse set of frameworks and languages. He currently support developers at Spacee with their IoT, Cloud and ML development. On the side, Ron Dagdag is active participant in the community as a Microsoft MVP, speaker, maker and blogger. He is passionate about Augmented Intelligence, studying the convergence of Augmented Reality/Virtual Reality, Machine Learning and the Internet of Things.
rondagdag