Sending HoloLens 2 video stream through Roboflow API

Copper Contributor

Hello everyone,

 

I’m currently working on a mixed reality project for the HoloLens 2, which involves the use of a computer vision model for the recognition of some shop-floor equipment and parts. My team has trained this model in Roboflow, using the data we’ve collected in a factory.

 

However, we’re not sure how to integrate this model to the HoloLens. We thought it could be a good idea to access the HoloLens’ video stream in order to send it to the CV model through the Roboflow API, and get the result back, in runtime. So we’re looking at Microsoft’s CV samples for the HoloLens 2, which includes a “StreamRecorder” sample. However, this sample saves the stream in a raw format, which would need to be converted before sending it to the CV model.

 

We’re actually not even sure if using this sample as a basis is the best way to do what we want, so we’re a bit lost at the moment. Has anyone here tried accessing the Roboflow API from a HoloLens 2? How would be the best way to go about it?

 

Thanks in advance.

1 Reply