Forum Widgets
Latest Discussions
How to unproject a 2D point in an captured image back to its accurate location in the physical world
Hi all, I'm creating HoloLens2 software that detects corners in an image and then shows their actual location in the world. However, there may be slight deviations between the projected points and their actual location, which increase with distance from the user. As following figure shows. Here's my code snippet for projecting the detected corners: I convert the corners from image to screen coordinates, then to NDC for unprojection. Next, I use the inverse of the projection matrix to convert the points from NDC to Camera space and then from Camera space to World space using the camera-to-world matrix. Finally, I instantiate the detected points using a Ray and RaycastHit. For getting the cameraToWorldMatrix and projectionMatrix, i referred to XRIV work From my side, i guess it may be the reason that the depth information is not estimated properly or the value of cameraToWorldMatrix and projectionMatrix are not correct. However, i have made efforts on this two reasons, but failed. May dear friends could give me some insights about how to mitigate this projecting devation, i show my great appreciation here in advance!
Azure Spatial Anchors Litecoin Mining Stats
https://youtu.be/5k7kb1WoHSw What could be more cyberpunk than combining holograms and cryptocurrency? I just finished creating a sample HoloLens 2 app that demonstrates how to create, locate, and delete an Azure Spatial Anchor. The app lets you pick a position in your environment to display a slate showing your Litecoin cryptocurrency mining stats from litecoinpool.org. It makes use of several MRTK components & features: hand rays, spatial awareness, surface magnetism, tap to place, a near menu, hand menus, eye tracking, and the little rotating orbs progress indicator. When you're given a box of crayons, I say Use all the colors! Source code available here: https://github.com/torynfarr/hololens
StereoKit v0.3
StereoKit, the C# OpenXR based Mixed Reality engine, just hit v0.3! It already support HoloLens as well as all the major VR headsets, and this brings in Oculus Quest support, Linux support, and a bunch more! I'm super excited about this update, since I've spent the last 7-8 months working on it, and it can do some pretty awesome stuff now! If you want to know more about it, I wrote a post about it over on my blog! https://playdeck.net/blog/stereokit-v0.3-release If anyone tries it, I'd love to know what you think! 🙂 https://stereokit.net/
sk_gpu.h WebXR Demo
Up until recently, I thought doing WebXR from C++ wouldn't work without a fair bit of effort, but I just discovered a nice chunk of code that helped me work through it! I had sk_gpu.h building to WASM with Emscripten before, so it didn't take too much more to get WebXR in there too 🙂 Here's the demo online! It's nothing too fancy, this is my test scene that lets me check on the most important features, but I'm quite excited that it works! At some point I'll be sure to ship a demo that's fun to play with too, but this also means I'm another step closer to getting all of StereoKit running on web too!sk_gpu.h - MR cross platform rendering API
I recently made some cool milestones with the cross-platform renderer I've been working on for StereoKit, so I figured I'd share a bit about it on here! So first! sk_gpu.h is a single-header, cross platform graphics library focused towards Mixed Reality rendering. It's somewhat low level, but it's definitely meant to be much easier than directly working with Vulkan or D3D12! It uses real HLSL for the shaders (I love HLSL!) and the goal is that it'll work on any platform that also has a Mixed Reality headset 🙂 The exciting part is that as of last week, I've already replaced StereoKit's old D3D11 renderer with sk_gpu in a branch, and it works great! StereoKit itself still only works with OpenXR on Windows/HoloLens platforms, but sk_gpu.h by itself works with OpenXR on Windows, HoloLens, and Oculus Quest! It even works on the flat web too, while I'd love to get to WebXR at some point, there's still some obstacles there. This is just the test scene I've been working with, but it demonstrates dynamic meshes, render to texture, cubemaps, instancing, and a few other things! But a lot of people ask me, why did I write this? Why not use something else? There's excellent tech out there already, like BGFX, and the amazing sokol_gfx! Even webgpu native could work if it wasn't so early. But here's my reasoning why: - I really wanted to use real HLSL. Not GLSL, or a pseudo shader language! - Tight focus on Mixed Reality rendering allows for simplification and hardware assumptions. - Smaller code base and more agility to respond to new MR graphics features! - I uh... couldn't figure out how to build any of the other libraries 😞 - ...and I really enjoy writing graphics code! There's still plenty more to do with sk_gpu.h, it definitely needs some cleanup, and I still need to figure out the best way to render in stereo on Quest! I'd also love to add some higher level things to it, like simple image and mesh loading to make it easier for people to try. Maybe even a Dear ImGui integration. Anyhow, I'd love to know what people think of it, or what everyone would like to see from a library like this 🙂 The source is MIT license up on Github! And definitely check out StereoKit too if this tickles your fancy!
