Texturing spatial mesh

%3CLINGO-SUB%20id%3D%22lingo-sub-2149245%22%20slang%3D%22en-US%22%3ETexturing%20spatial%20mesh%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2149245%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20everyone%2C%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20try%20to%20make%20an%20application%20that%20can%20export%20an%20obj%20file%20from%20the%20spatial%20mesh%20to%20show%20it%20in%20another%20software.%3C%2FP%3E%3CP%3E%3CBR%20%2F%3ETo%20add%20realism%20for%20my%20object%2C%20I%20want%20to%20add%20real%20world%20colors%20to%20this%20mesh.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ESo%2C%20here%20is%20my%20question%20%3A%3C%2FP%3E%3CP%3EHow%20can%20I%20colorize%20my%20mesh%20to%20match%20the%20real%20room%20I%20scan%20%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2149245%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EHoloLens%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EMRTK%20Unity%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EUnity%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2150162%22%20slang%3D%22en-US%22%3ERe%3A%20Texturing%20spatial%20mesh%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2150162%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F972274%22%20target%3D%22_blank%22%3E%40Layra_Alixis%3C%2FA%3E%26nbsp%3BI've%20seen%20people%20do%20this%20before%2C%20the%20idea%20is%20fairly%20simple%2C%20but%20the%20implementation%20details%20are%20definitely%20a%20bit%20tricky!%20I%20don't%20know%20of%20any%20off-the-shelf%20solutions%20for%20this%2C%20but%20I%20haven't%20been%20looking%20recently.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThe%20core%20idea%20of%20this%20is%20to%20take%20color%20pictures%20as%20you%20wander%20around%2C%20and%20keep%20the%20camera%20position%2C%20orientation%2C%20and%20other%20details%20for%20that%20photo.%20Once%20you%20have%20a%20mesh%20of%20the%20room%2C%20that%20mesh%20needs%20UV%20coordinates%20that%20don't%20overlap!%20Create%20a%20blank%20texture%2C%20and%20then%20you%20can%20stamp%2C%20or%20project%20the%20photo%20onto%20that%20texture.%20This%20type%20of%20unwrapping%20and%20projecting%20may%20not%20be%20too%20tricky%20to%20figure%20out%20in%20a%203D%20modeling%20tool%2C%20but%20if%20you're%20doing%20this%20through%20code%2C%20it%20would%20definitely%20be%20a%20project!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2155443%22%20slang%3D%22en-US%22%3ERe%3A%20Texturing%20spatial%20mesh%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2155443%22%20slang%3D%22en-US%22%3E%3CP%3EI%20have%20to%20do%20it%20through%20the%20code%20so%20that%20the%20user%20can%20see%20in%20real%20time%20the%20virtual%20environment%20surrounding%20them.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20use%20Unity%202018.4.20%20and%20MRTK%202.4%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2157197%22%20slang%3D%22en-US%22%3ERe%3A%20Texturing%20spatial%20mesh%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2157197%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F972274%22%20target%3D%22_blank%22%3E%40Layra_Alixis%3C%2FA%3E%20Funny%20enough%2C%20while%20looking%20around%20for%20a%20good%20example%2C%20I%20spotted%20your%20StackOverflow%20question%2C%20and%20someone%20already%20linked%20you%20to%20the%20exact%20%3CA%20href%3D%22https%3A%2F%2Fsites.cs.ucsb.edu%2F~holl%2Fpubs%2FDong-2018-IEEEVR.pdf%22%20target%3D%22_self%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3Esame%20PDF%20I%20had%20spotted%3C%2FA%3E!%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EFrom%20a%20cursory%20glance%20around%2C%20I%20can't%20really%20find%20any%20pre-baked%20solutions%20for%20this%20problem%2C%20so%20you're%20either%20going%20to%20have%20to%20do%20a%20lot%20more%20digging%2C%20or%20write%20this%20stuff%20from%20scratch.%20If%20you%20are%20going%20to%20write%20it%20from%20scratch%2C%20that%20paper%20may%20provide%20you%20with%20some%20great%20insight!%20From%20a%20quick%20skim%2C%20it%20looks%20like%20the%20right%20approach%2C%20and%20is%20more%20or%20less%20what%20I%20described%20in%20my%20previous%20post%2C%20except%20in%20much%20greater%20detail.%3C%2FP%3E%3C%2FLINGO-BODY%3E
New Contributor

Hi everyone,

 

I try to make an application that can export an obj file from the spatial mesh to show it in another software.


To add realism for my object, I want to add real world colors to this mesh.

 

So, here is my question :

How can I colorize my mesh to match the real room I scan ?

 

Thanks

3 Replies

@Layra_Alixis I've seen people do this before, the idea is fairly simple, but the implementation details are definitely a bit tricky! I don't know of any off-the-shelf solutions for this, but I haven't been looking recently.

 

The core idea of this is to take color pictures as you wander around, and keep the camera position, orientation, and other details for that photo. Once you have a mesh of the room, that mesh needs UV coordinates that don't overlap! Create a blank texture, and then you can stamp, or project the photo onto that texture. This type of unwrapping and projecting may not be too tricky to figure out in a 3D modeling tool, but if you're doing this through code, it would definitely be a project!

I have to do it through the code so that the user can see in real time the virtual environment surrounding them.

 

I use Unity 2018.4.20 and MRTK 2.4

@Layra_Alixis Funny enough, while looking around for a good example, I spotted your StackOverflow question, and someone already linked you to the exact same PDF I had spotted!

 

From a cursory glance around, I can't really find any pre-baked solutions for this problem, so you're either going to have to do a lot more digging, or write this stuff from scratch. If you are going to write it from scratch, that paper may provide you with some great insight! From a quick skim, it looks like the right approach, and is more or less what I described in my previous post, except in much greater detail.