Hololens 2 experience is a dream to use.

%3CLINGO-SUB%20id%3D%22lingo-sub-2248450%22%20slang%3D%22en-US%22%3EHololens%202%20experience%20is%20a%20dream%20to%20use.%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2248450%22%20slang%3D%22en-US%22%3E%3CP%3EWould%20it%20be%20possible%20to%20engineer%20a%20new%20setup%20where%20you%20can%20use%20a%20more%20powerful%20computer%20that%20is%20either%20backpacked%20or%20long-wired%20to%20a%20super-computer%20in%20the%20building%3F%3CBR%20%2F%3E%3CBR%20%2F%3EJust%20have%20all%20the%20sensors%20and%20display%20be%20routed%20to%20the%20main%20more%20powerful%20PC.%26nbsp%3B%20Being%20limited%20to%204GB%20as%20well%20as%20Snapdragon%20850%20can%20only%20take%20you%20so%20far.%26nbsp%3B%20It's%20great%20for%20now%20but%20the%20potential%20of%20this%20is%20just%20so%20high%20but%20crippled%20by%20trying%20to%20compact%20it%20into%20something%20all%20headmounted%20only.%3CBR%20%2F%3E%3CBR%20%2F%3EThe%20escape%20from%20actual%20reality%20is%20clearly%20awesome%20with%20incredible%20potential.%26nbsp%3B%20It%20can%20actually%20make%20reality%20better%20too.%26nbsp%3B%20Unfortunately%2C%20a%20lot%20of%20top%20engineers%20are%20doing%20BTC%20and%20other%20easy%20ways%20to%20make%20money%20or%20just%20straight%20retiring.%26nbsp%3B%20Imo%2C%20all%20this%20makes%20life%20better%20that%20people%20would%20love%20to%20develop%20and%20use%20if%20only%20it%20could%20be%20easier%20and%20the%20toolsets%20seamless%20to%20make.%26nbsp%3B%20Some%20apps%20are%20so%20great%20that%20you%20might%20as%20well%20use%20them%20rather%20than%20try%20to%20make%20things%20yourself%20in%20Unity%20as%20the%20time%20it%20takes%20to%20make%20a%20basic%20scene%20then%20upload%2Frun%20it%20is%20super%20long.%26nbsp%3B%20Moving%20objects%20to%20the%20cloud%20and%20having%20it%20near%20instantly%20show%20up%20on%20the%20display%20to%20drag%20n'%20drop%20is%20great.%3CBR%20%2F%3E%3CBR%20%2F%3EThe%20world%20that%20I've%20seen%20in%20Hololens%202%2C%20I%20can't%20live%20w%2Fo.%26nbsp%3B%20I%20hate%20coming%20back%20to%20the%202D%20reality%20we%20have%20now.%20%3A(%3C%2Fimg%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2249518%22%20slang%3D%22en-US%22%3ERe%3A%20Hololens%202%20experience%20is%20a%20dream%20to%20use.%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2249518%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F1013110%22%20target%3D%22_blank%22%3E%40AntDX316%3C%2FA%3E%26nbsp%3BI%20definitely%20love%20your%20vision%20of%20the%20future%2C%20and%20it's%20one%20of%20the%20reasons%20I%20love%20to%20work%20in%20this%20space!%20Making%20and%20exploring%20things%20in%20MR%20is%20just...%20amazing%20%3AD%3C%2Fimg%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%3EIt%20also%20sounds%20like%20there%20could%20be%20all%20sorts%20of%20things%20you%20might%20be%20interested%20in%20here%20as%20well!%20As%20a%20graphics%20programmer%20myself%2C%20I%20really%20enjoy%20squeezing%20great%20graphics%20out%20of%20the%20Snapdragon%2C%20it's%20a%20surprisingly%20capable%20device!%20But%20it's%20definitely%20not%20a%20GTX%203080%2C%20so%20that's%20why%20tools%20like%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fwindows%2Fmixed-reality%2Fdevelop%2Fplatform-capabilities-and-apis%2Fholographic-remoting-create-player%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EHolographic%20Remoting%3C%2FA%3E%2C%20or%20%3CA%20href%3D%22https%3A%2F%2Fazure.microsoft.com%2Fen-us%2Fservices%2Fremote-rendering%2F%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EAzure%20Remote%20Rendering%3C%2FA%3E%20exist!%20Both%20allow%20you%20to%20create%20applications%20that%20are%20either%20entirely%20remote%2C%20or%20are%20a%20hybrid%20of%20remote%20and%20local%20content.%20No%20wires%20needed%20%3A)%3C%2Fimg%3E%3CBR%20%2F%3E%3CBR%20%2F%3EIf%20you're%20interested%20in%20faster%20build%20times%2C%20well...%20me%20too!%20I%20built%20a%20C%23%20Mixed%20Reality%20engine%20based%20on%20OpenXR%20called%20%3CA%20href%3D%22https%3A%2F%2Fstereokit.net%2F%22%20target%3D%22_self%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3EStereoKit%3C%2FA%3E%2C%20which%20prioritizes%20MR%20development%2C%20and%20can%20build%2Fdeploy%20%3CEM%3Ereally%3C%2FEM%3E%20fast%20compared%20to%20other%20tools!%20It's%20also%20tuned%20for%20performance%20on%20mobile%20VR%2C%20so%20you%20may%20find%20it%20a%20little%20snappier%20there%20too.%20If%20you're%20looking%20to%20iterate%20even%20faster%20still%2C%20you%20might%20be%20interested%20in%20%3CA%20href%3D%22https%3A%2F%2Fdoc.babylonjs.com%2FdivingDeeper%2FwebXR%2FwebXRARFeatures%22%20target%3D%22_self%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3Ebabylon.js%3C%2FA%3E%20and%20WebXR!%20It%20can%20be%20hard%20to%20beat%20a%20webpage%20refresh%20for%20iteration%20time!%20Also%20for%20design%2C%20and%20spatial%20prototyping%2C%20check%20out%20%3CA%20href%3D%22https%3A%2F%2Fwww.maquette.ms%2F%22%20target%3D%22_self%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3EMicrosoft%20Maquette%3C%2FA%3E%2C%20it's%20a%20gem.%3CBR%20%2F%3E%3CBR%20%2F%3ESome%20other%20things%20that%20can%20make%20life%20on%20a%20HoloLens%20nicer%3A%20%3CA%20href%3D%22https%3A%2F%2Fwww.mirage-app.com%2F%22%20target%3D%22_self%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3EMirage%3C%2FA%3E%20lets%20you%20run%20virtual%20monitors%20driven%20by%20your%20desktop%20on%20your%20HoloLens!%20Not%20immersive%20stuff%2C%20but%20can%20be%20pretty%20sweet.%20And%20a%20good%20bluetooth%20keyboard%2C%20I've%20got%20a%20nice%20foldable%20one%20that's%20only%20slightly%20larger%20than%20my%20phone.%20Makes%20life%20a%20lot%20nicer%20when%20I%20need%20to%20input%20text%20%3A)%3C%2Fimg%3E%3CBR%20%2F%3E%3CBR%20%2F%3EWelcome%20to%20the%20future!%20It's%20not%20perfect%20yet%2C%20but%20it's%20getting%20there%20%3A)%3C%2Fimg%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2251282%22%20slang%3D%22en-US%22%3ERe%3A%20Hololens%202%20experience%20is%20a%20dream%20to%20use.%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2251282%22%20slang%3D%22en-US%22%3E%3CP%3EMirage%20is%20definitely%20what%20I'm%20looking%20for%20too.%20Issue%20is%20you%20cannot%20disable%20the%20air-hand%20pointers%20completely.%20Say%20I%20have%20a%20monitor%20or%20monitors%20up.%20I%20use%20the%202nd%20monitor%20which%20is%20the%20Hololens%20as%20an%20overlay.%20If%20I%20pinch%20any%20of%20my%20fingers%20such%20as%20operating%20on%20the%20keyboard%20or%20just%20by%20habit%2C%20the%20Hololens%20picks%20it%20up%20and%20moves%20the%20Hololens%20monitor.%20Locking%20monitors%20doesn't%20disable%20the%20air-pointers%20completely%20nor%20does%20disabling%20touch.%20How%20do%20I%20disable%20the%20air-pointers%20completely%20so%20it%20doesn't%20move%20anything%3F%3CBR%20%2F%3E%3CBR%20%2F%3ESee%20vid%3A%3CBR%20%2F%3E%3CA%20href%3D%22https%3A%2F%2Fyoutu.be%2F5sxnrkEI-zA%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fyoutu.be%2F5sxnrkEI-zA%3C%2FA%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%3CP%3Eedit%3A%20Air%20pointers%20must%20not%20be%20shown%20when%20going%20in%20to%20Lock%20Monitor%20mode%20but%20if%20the%20monitor%20is%20very%20close%20to%20the%20Hololens%2C%20the%20grabbing%20donut%20pointer%20remains.%26nbsp%3B%20I%20want%20it%20not%20to%20show%20at%20all.%26nbsp%3B%20Can%20this%20be%20done%3F%3C%2FP%3E%3CP%3E%3CBR%20%2F%3EFindings%20with%20Mirage%2C%20the%20overlay%20feature%20(Hololens%20window%20on%20top%20of%20what%20you%20are%20doing%20such%20as%202D%20desktop%20monitor)%20and%20other%20windows%20in%20mid%20air%20as%20extra%20monitors%20is%20totally%20the%20future.%26nbsp%3B%20A%20way%20to%20make%20it%20better%20is%20if%20we%20can%20get%20a%20way%20to%20use%20a%20hardware%20device%20connected%20w%2F%20bluetooth%20to%20be%20able%20to%20roll%20through%20each%20window%20such%20as.%26nbsp%3B%20If%20I%20have%20say%204%20overlays%20directly%20on%20the%20front%2C%20that%20you%20can%20cycle%20between%20them%20but%20then%20I%20assume%20the%20way%20would%20just%20be%20to%20have%20that%20interact%20w%2F%20the%20program%20to%20make%20it%20cycle%20which%20requires%20more%20programming%20but%20perhaps%20easier.%26nbsp%3B%20Sometimes%20you%20want%20other%20data%20to%20be%20in-focus%20than%20another.%26nbsp%3B%20Rather%20than%20looking%20around%20the%20%22room%22%20you%20should%20be%20able%20to%20just%20move%20from%20one%20window%20to%20another.%3CBR%20%2F%3E%3CBR%20%2F%3EThe%20best%20thing%20with%20Mirage%20is%2C%20you%20can%20backpack%20your%20laptop%20then%20manipulate%20the%20cursor%20with%20many%20ways.%26nbsp%3B%20Connect%20the%20laptop%20to%20a%20bigger%20battery%20on%2Fin%20the%20backpack%20and%20then%20USB%20connect%20the%20Hololens%20to%20the%20laptop.%26nbsp%3B%20It%20just%20would%20be%20nice%20to%20be%20able%20to%20air%20touch%20w%2F%20magnification%20of%20where%20the%20point%20is%20so%20you%20can%20precision%20click%20what%20you%20want%20to%20do.%26nbsp%3B%20You%20should%20be%20able%20to%20have%20right%20and%20left-click%20as%20on%20the%20Windows%20PC.%26nbsp%3B%20Even%20middle%20mouse%20click%20would%20also%20be%20nice%20to%20have%20available.%26nbsp%3B%20Say%20you%20point%2C%20but%20the%20device%20you%20are%20holding%20can%20allow%20you%20to%20left%2C%20middle%2C%20or%20right-click%20intention.%3CBR%20%2F%3E%3CBR%20%2F%3EBeing%20able%20to%20use%20any%20Windows%20window%20and%20have%20it%20in%20mid-air%20takes%20a%20lot%20of%20mega%20programming%20out%20such%20as%20coding%20for%20iOS%20or%20Android.%26nbsp%3B%20You%20can%20have%20real-time%20data%20of%20anything%20important%20close%20to%20you%20while%2C%20if%20anything%20goes%20wrong%20you%20can%20just%20exit%20out%20of%20Mirage%20and%20have%20all%20the%20windows%20sandwiched%20into%20the%20PC%20window.%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%3CP%3EI%20will%20look%20into%20the%20rest%20too%20but%20for%20now%20Mirage.%26nbsp%3B%20I%20was%20using%20some%20AirText%20app%20and%20Spatial.io%20to%20do%20the%20other%20things%20I%20want%20to%20do%20but%20even%20w%2F%20those%2C%20they%20have%20their%20limitations.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2251560%22%20slang%3D%22en-US%22%3ERe%3A%20Hololens%202%20experience%20is%20a%20dream%20to%20use.%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2251560%22%20slang%3D%22en-US%22%3ESounds%20like%20you've%20got%20a%20lot%20of%20interesting%20ideas!%20The%20Feedback%20Hub%20application%20is%20actually%20a%20pretty%20great%20way%20to%20get%20feedback%20to%20some%20of%20the%20products%20like%20these%2C%20and%20I'd%20definitely%20recommend%20you%20give%20it%20a%20try!%20The%20MR%20team%20definitely%20pays%20attention%20to%20stuff%20that%20comes%20in%20through%20there%20%3A)%3C%2Fimg%3E%20Mirage%20is%20pretty%20recent%2C%20and%20I'm%20sure%20they'd%20welcome%20some%20feedback!%3CBR%20%2F%3E%3CBR%20%2F%3EAnd%20given%20how%20many%20ideas%20you've%20got%2C%20I%20bet%20you%20could%20put%20together%20a%20pretty%20good%20product%20yourself!%20I've%20done%20similar%20experiments%2C%20and%20would%20-love-%20it%20if%20I%20had%20more%20time%20to%20work%20on%20something%20like%20this.%20I%20think%20there's%20a%20lot%20of%20room%20for%20innovation%20in%20desktop%20experiences%20%3A)%3C%2Fimg%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2251750%22%20slang%3D%22en-US%22%3ERe%3A%20Hololens%202%20experience%20is%20a%20dream%20to%20use.%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2251750%22%20slang%3D%22en-US%22%3E%3CP%3ECan%20you%20show%20how%20to%20use%20StereoKit%3F%20I'm%20not%20even%20sure%20how%20to%20load%20the%20program.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EDoes%20Microsoft%20Maquette%20work%20with%20Hololens%202%3F%3CBR%20%2F%3E%3CBR%20%2F%3ECan%20you%20video%20some%20demos%20you've%20done%20from%20inside%20the%20Hololens%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2251760%22%20slang%3D%22en-US%22%3ERe%3A%20Hololens%202%20experience%20is%20a%20dream%20to%20use.%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2251760%22%20slang%3D%22en-US%22%3EThe%20getting%20started%20guide%20for%20it%20is%20over%20here%3A%3CBR%20%2F%3E%3CA%20href%3D%22https%3A%2F%2Fstereokit.net%2FPages%2FGuides%2FGetting-Started.html%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fstereokit.net%2FPages%2FGuides%2FGetting-Started.html%3C%2FA%3E%3CBR%20%2F%3E%3CBR%20%2F%3EIt's%20more%20code-centric%20than%20something%20like%20Unity%2C%20but%20that's%20not%20always%20a%20bad%20thing%20%3A)%3C%2Fimg%3E%20Let%20me%20know%20if%20you%20run%20into%20any%20trouble%20with%20that!%3C%2FLINGO-BODY%3E
Occasional Contributor

Would it be possible to engineer a new setup where you can use a more powerful computer that is either backpacked or long-wired to a super-computer in the building?

Just have all the sensors and display be routed to the main more powerful PC.  Being limited to 4GB as well as Snapdragon 850 can only take you so far.  It's great for now but the potential of this is just so high but crippled by trying to compact it into something all headmounted only.

The escape from actual reality is clearly awesome with incredible potential.  It can actually make reality better too.  Unfortunately, a lot of top engineers are doing BTC and other easy ways to make money or just straight retiring.  Imo, all this makes life better that people would love to develop and use if only it could be easier and the toolsets seamless to make.  Some apps are so great that you might as well use them rather than try to make things yourself in Unity as the time it takes to make a basic scene then upload/run it is super long.  Moving objects to the cloud and having it near instantly show up on the display to drag n' drop is great.

The world that I've seen in Hololens 2, I can't live w/o.  I hate coming back to the 2D reality we have now. :( I have the screens setup so when I go all the way right I have the experience.  I use the keyboard from the main PC through Developer Deploy to type.  It's not that I'm doing everything in the Hololens 2 but having the Hololens 2 along with using the main PC makes the computing experience a lot better.

7 Replies

@AntDX316 I definitely love your vision of the future, and it's one of the reasons I love to work in this space! Making and exploring things in MR is just... amazing :D

It also sounds like there could be all sorts of things you might be interested in here as well! As a graphics programmer myself, I really enjoy squeezing great graphics out of the Snapdragon, it's a surprisingly capable device! But it's definitely not a GTX 3080, so that's why tools like Holographic Remoting, or Azure Remote Rendering exist! Both allow you to create applications that are either entirely remote, or are a hybrid of remote and local content. No wires needed :)

If you're interested in faster build times, well... me too! I built a C# Mixed Reality engine based on OpenXR called StereoKit, which prioritizes MR development, and can build/deploy really fast compared to other tools! It's also tuned for performance on mobile VR, so you may find it a little snappier there too. If you're looking to iterate even faster still, you might be interested in babylon.js and WebXR! It can be hard to beat a webpage refresh for iteration time! Also for design, and spatial prototyping, check out Microsoft Maquette, it's a gem.

Some other things that can make life on a HoloLens nicer: Mirage lets you run virtual monitors driven by your desktop on your HoloLens! Not immersive stuff, but can be pretty sweet. And a good bluetooth keyboard, I've got a nice foldable one that's only slightly larger than my phone. Makes life a lot nicer when I need to input text :)

Welcome to the future! It's not perfect yet, but it's getting there :)

Mirage is definitely what I'm looking for too. Issue is you cannot disable the air-hand pointers completely. Say I have a monitor or monitors up. I use the 2nd monitor which is the Hololens as an overlay. If I pinch any of my fingers such as operating on the keyboard or just by habit, the Hololens picks it up and moves the Hololens monitor. Locking monitors doesn't disable the air-pointers completely nor does disabling touch. How do I disable the air-pointers completely so it doesn't move anything?

See vid:
https://youtu.be/5sxnrkEI-zA

edit: Air pointers must not be shown when going in to Lock Monitor mode but if the monitor is very close to the Hololens, the grabbing donut pointer remains.  I want it not to show at all.  Can this be done?


Findings with Mirage, the overlay feature (Hololens window on top of what you are doing such as 2D desktop monitor) and other windows in mid air as extra monitors is totally the future.  A way to make it better is if we can get a way to use a hardware device connected w/ bluetooth to be able to roll through each window such as.  If I have say 4 overlays directly on the front, that you can cycle between them but then I assume the way would just be to have that interact w/ the program to make it cycle which requires more programming but perhaps easier.  Sometimes you want other data to be in-focus than another.  Rather than looking around the "room" you should be able to just move from one window to another.

The best thing with Mirage is, you can backpack your laptop then manipulate the cursor with many ways.  Connect the laptop to a bigger battery on/in the backpack and then USB connect the Hololens to the laptop.  It just would be nice to be able to air touch w/ magnification of where the point is so you can precision click what you want to do.  You should be able to have right and left-click as on the Windows PC.  Even middle mouse click would also be nice to have available.  Say you point, but the device you are holding can allow you to left, middle, or right-click intention.

Being able to use any Windows window and have it in mid-air takes a lot of mega programming out such as coding for iOS or Android.  You can have real-time data of anything important close to you while, if anything goes wrong you can just exit out of Mirage and have all the windows sandwiched into the PC window.

I will look into the rest too but for now Mirage.  I was using some AirText app and Spatial.io to do the other things I want to do but even w/ those, they have their limitations.

Sounds like you've got a lot of interesting ideas! The Feedback Hub application is actually a pretty great way to get feedback to some of the products like these, and I'd definitely recommend you give it a try! The MR team definitely pays attention to stuff that comes in through there :) Mirage is pretty recent, and I'm sure they'd welcome some feedback!

And given how many ideas you've got, I bet you could put together a pretty good product yourself! I've done similar experiments, and would -love- it if I had more time to work on something like this. I think there's a lot of room for innovation in desktop experiences :)

Can you show how to use StereoKit? I'm not even sure how to load the program.

 

Does Microsoft Maquette work with Hololens 2?

Can you video some demos you've done from inside the Hololens?

The getting started guide for it is over here:
https://stereokit.net/Pages/Guides/Getting-Started.html

It's more code-centric than something like Unity, but that's not always a bad thing :) Let me know if you run into any trouble with that!

I'm not sure what the first steps are besides downloading the files and installing them. Can you record the workflow from loading windows to it working on the hololens?

@AntDX316 After installing all the pre-requisites, open Visual Studio, create a new project, and pick the StereoKit UWP template.

At the top, set Visual Studio's build settings to Debug, ARM64, and make sure your deploy target is set to Device.

koujaku_0-1617426004501.png


If you haven't already, or don't see Device as an option, you may still need to set up your HoloLens for development. More details about that over here.

After deploy, it should be running on your device! After that, you'll have to write your own code for application logic. There's plenty of documentation, resources and samples on the website