I successfully saved and displayed the spatial mesh after scene understanding room scanning process. Now I am struggling about how to apply hololens camera live texture to mesh. Any Solution?

Just FYI, this is a pretty tricky problem to solve! I've seen it done, but it may require a fair bit of work.If you're interested, there was a brief discussion on this topic a while ago that does go into a few details:https://techcommunity.microsoft.com/t5/mixed-reality-feedback-and-help/texturing-spatial-mesh/m-p/2149245It's also quite helpful for us to know what you're hoping to achieve with this as well. Can you tell us anything about your use-case?

Thank you for your reply. My use case is display the current Room with real color texture at runtime in front of the hololens camera. It is like a 3D minimap. I am using unity3d(2019.4.23 version) with c#. I have imported the scene understanding plugin for Room scanning. Room scanning and displayed the room mesh functionalities are completed. Now I want apply the real world room texture to the displayed mesh.I have attached my sample output.

Hi I am interested in the same thing :), I also just managed to get the scene on .obj and would like to have the texture from the camera over the object, even if in post processing it wouldn't be a problem for me. Did you manage to advance over this problem?By the way, I guess the code used on the paper mentioned on the other post can be found at https://github.com/samueldong-us/Realtime-Hololens-Retexturing.

I'm looking into the same topic. Is there a way to work together on this subject?As far as I know by now, it's seams to be a very tricky subject. Some have solved it but I cannot find out how they did it

Hi,I searched the holo texture functions in github. Someone developer has successfully completed.check the link:https://github.com/tarukosu/HoloCameraTextureMappingIt is a unity 5.6.7 version project and you can take the build without any error. It is properly working in hololens 1. But currently I am working with Hololens2 device. So I tried this project open with latest version(2019.4.4). But lot of errors coming, I could not solve the errors.Can you help me for the solution?

HELP: Hololens2 apply camera capture texture for Spatial Mesh

koujaku
Microsoft
Jul 12, 2021
Just FYI, this is a pretty tricky problem to solve! I've seen it done, but it may require a fair bit of work.

If you're interested, there was a brief discussion on this topic a while ago that does go into a few details:
https://techcommunity.microsoft.com/t5/mixed-reality-feedback-and-help/texturing-spatial-mesh/m-p/2149245

It's also quite helpful for us to know what you're hoping to achieve with this as well. Can you tell us anything about your use-case?
- senthilvelGS
  Copper Contributor
  Jul 14, 2021
  Thank you for your reply. My use case is display the current Room with real color texture at runtime in front of the hololens camera. It is like a 3D minimap. I am using unity3d(2019.4.23 version) with c#. I have imported the scene understanding plugin for Room scanning. Room scanning and displayed the room mesh functionalities are completed. Now I want apply the real world room texture to the displayed mesh.
  I have attached my sample output.
  - daniel_camara
    Copper Contributor
    Aug 13, 2021
    Hi I am interested in the same thing :), I also just managed to get the scene on .obj and would like to have the texture from the camera over the object, even if in post processing it wouldn't be a problem for me. Did you manage to advance over this problem?
    
    By the way, I guess the code used on the paper mentioned on the other post can be found at https://github.com/samueldong-us/Realtime-Hololens-Retexturing.
Epic-Max
Copper Contributor
Aug 14, 2021
I'm looking into the same topic. Is there a way to work together on this subject?
As far as I know by now, it's seams to be a very tricky subject. Some have solved it but I cannot find out how they did it
- senthilvelGS
  Copper Contributor
  Aug 16, 2021
  Hi,
  
  I searched the holo texture functions in github. Someone developer has successfully completed.
  
  check the link:
  https://github.com/tarukosu/HoloCameraTextureMapping
  
  It is a unity 5.6.7 version project and you can take the build without any error. It is properly working in hololens 1. But currently I am working with Hololens2 device. So I tried this project open with latest version(2019.4.4). But lot of errors coming, I could not solve the errors.
  
  Can you help me for the solution?
  - daniel_camara
    Copper Contributor
    Aug 17, 2021
    senthilvelGS
    A person that worked with me diged up on that code and managed to "convert" it to a newer version, supposedly compatible with Hololens 2. You can find a version over https://github.com/danielcamara/Holo2CameraTextureMapping.
    
    Unfortunately it does not work, it gives no compilation errors, but does not work. It can be a better starting point for someone that want to adapt it to Hololens 2. My skils are shalow, I am not able to fix it... but maybe someone else is 🙂

Forum Discussion

HELP: Hololens2 apply camera capture texture for Spatial Mesh