Augmented reality remote learning with Hololens2

Published Aug 24 2020 11:26 AM 2,075 Views
Microsoft

Chemical engineers at Imperial College London have been transforming how they teach practical lab skills to students based remotely, as part of the new multi-mode offering for students in the new academic year.

New and returning chemical engineering undergraduates will have the opportunity to use augmented reality (AR) to complete lab-based experiments, using Microsoft HoloLens technology to connect and interact from wherever they are in the world. 

 

Practical lab skills

Lab work is an essential component of the chemical engineering MEng degree and normally requires students to perform experiments in person. As well as learning practical skills, students develop their ability to work in teams and problem solve, by applying theory-based learning to real engineering problems.

Ensuring that more than 600 students from around 69 countries receive the same high-quality learning experience that they would have pre-COVID requires a better solution than simply moving content online.


To meet these new needs, a team of Teaching Fellows and future chemical engineers have developed the CREATE Lab (Chemeng Remote Experience Augmented through TEchnology Laboratory). 


CREATE Lab enables students to participate in their chosen lab project remotely using three key pieces of technology: Microsoft Hololens headsets, controllable cameras and QR codes.

  • QR codes are placed around the labs and direct students to additional learning resources

  • Example of an augmented reality guide which provides instructions for operating equipment

  • CCTV cameras have been placed around the undergraduate labs and can be controlled remotely

One member of each student team will be physically present in the lab and supported by a Graduate Teaching Assistant. Wearing the Hololens headset, they will connect to their remote colleagues using Dynamic Guides, Microsoft Teams and Microsoft stream what they’re seeing in real time. Their team members will be able to provide both verbal and written instructions and together they will perform experiments on working machinery and rigs. 

To further support the virtual experience, remote access cameras have been mounted throughout the labs and will continuously stream video footage. Students will be able to move the camera and zoom in and out to easily capture equipment, readings and other essential components. Camera pre-sets will also allow students to ‘travel’ between different key experiment locations in the lab. 


QR codes are strategically placed in positions of certain processes or equipment pieces and, with the help of the Microsoft HoloLens, allow students to access relevant information regarding their tasks. Students both in person and remotely can easily scan these codes with mobile phones which will take them to informative videos and information packs. To complement these, the team have also developed a set of dynamic guides which provide a step-by-step instructions on how to operate certain pieces of equipment.


Working in partnership


Thomas Cheng, one of the undergraduates involved in the project, said:

 

The CREATE Labs bring flexibility with lab work in a way I have never seen before. The use of technology is refreshing and with the use of augmented reality, I would say even the experience for those within the lab has been enhanced. For those working remotely, it is almost like you're standing there in person, able to work with your group face to face, just like before, but with access to content that helps you understand why you are actually doing the experiment.”

Dr Clemens BrechtelsbauerDirector of Chemical Engineering Education

"Often our students go on to work in environments where they are controlling machinery remotely, so it’s important that we provide them with this experience during their degree.” 

The project forms part of a wider plan to transform the Department of Chemical Engineering curriculum. The goal is to introduce advanced learning technologies and replace the traditional way of teaching lab work, to enhance the student learning experience.

Dr Clemens Brechtelsbauer, Director of Chemical Engineering Education, explained:

“Remote learning has received increased attention as a result of the pandemic, but it’s something we’ve been exploring for some time as it provides an excellent opportunity for students to prepare themselves for working in industry. “Often our students go on to work in environments where they are controlling machinery remotely, so it’s important that we provide them with this experience during their degree.”

The CREATE Lab team will continue to develop the technology over the summer and will be actively seeking participation and feedback from other students during this process. You can follow the project’s progress on the team’s Instagram and YouTube accounts.

 

Microsoft Learn - Learning Modules for Hololens

Solve problems in real time with Dynamics 365 Remote Assist

https://docs.microsoft.com/en-us/learn/paths/solve-problems-remote-assist/

Set up a mixed reality project in Unity with the Mixed Reality Toolkit
https://docs.microsoft.com/en-us/learn/modules/mixed-reality-toolkit-project-unity/

%3CLINGO-SUB%20id%3D%22lingo-sub-1608687%22%20slang%3D%22en-US%22%3EAugmented%20reality%20remote%20learning%20with%20Hololens2%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1608687%22%20slang%3D%22en-US%22%3E%3CP%3EChemical%20engineers%20at%20Imperial%20College%20London%20have%20been%20transforming%20how%20they%20teach%20practical%20lab%20skills%20to%20students%20based%20remotely%2C%20as%20part%20of%20the%20new%20multi-mode%20offering%20for%20students%20in%20the%20new%20academic%20year.%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%3ENew%20and%20returning%20chemical%20engineering%20undergraduates%20will%20have%20the%20opportunity%20to%20use%20augmented%20reality%20(AR)%20to%20complete%20lab-based%20experiments%2C%20using%20%3CA%20href%3D%22https%3A%2F%2Fwww.microsoft.com%2Fen-us%2Fhololens%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EMicrosoft%20HoloLens%3C%2FA%3E%20technology%20to%20connect%20and%20interact%20from%20wherever%20they%20are%20in%20the%20world.%26nbsp%3B%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CDIV%20class%3D%22video-player%20video-player-loaded%22%20data-src%3D%22https%3A%2F%2Fyoutu.be%2F51TflyqBSM8%22%3E%0A%3CFIGURE%20class%3D%22caption%22%3E%0A%3CFIGCAPTION%3E%3CDIV%20class%3D%22video-embed-center%20video-embed%22%3E%3CIFRAME%20class%3D%22embedly-embed%22%20src%3D%22https%3A%2F%2Fcdn.embedly.com%2Fwidgets%2Fmedia.html%3Fsrc%3Dhttps%253A%252F%252Fwww.youtube.com%252Fembed%252F51TflyqBSM8%253Ffeature%253Doembed%26amp%3Bdisplay_name%3DYouTube%26amp%3Burl%3Dhttps%253A%252F%252Fwww.youtube.com%252Fwatch%253Fv%253D51TflyqBSM8%26amp%3Bimage%3Dhttps%253A%252F%252Fi.ytimg.com%252Fvi%252F51TflyqBSM8%252Fhqdefault.jpg%26amp%3Bkey%3Db0d40caa4f094c68be7c29880b16f56e%26amp%3Btype%3Dtext%252Fhtml%26amp%3Bschema%3Dyoutube%22%20width%3D%22600%22%20height%3D%22337%22%20scrolling%3D%22no%22%20title%3D%22YouTube%20embed%22%20frameborder%3D%220%22%20allow%3D%22autoplay%3B%20fullscreen%22%20allowfullscreen%3D%22true%22%3E%3C%2FIFRAME%3E%3C%2FDIV%3E%3C%2FFIGCAPTION%3E%0A%3C%2FFIGURE%3E%0A%3C%2FDIV%3E%0A%3CH2%20id%3D%22toc-hId--1273075573%22%20id%3D%22toc-hId--1273075573%22%3E%26nbsp%3B%3C%2FH2%3E%0A%3CH2%20id%3D%22toc-hId-1214437260%22%20id%3D%22toc-hId-1214437260%22%3EPractical%20lab%20skills%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FH2%3E%0A%3CP%3ELab%20work%20is%20an%20essential%20component%20of%20the%20chemical%20engineering%20MEng%20degree%20and%20normally%20requires%20students%20to%20perform%20experiments%20in%20person.%20As%20well%20as%20learning%20practical%20skills%2C%20students%20develop%20their%20ability%20to%20work%20in%20teams%20and%20problem%20solve%2C%20by%20applying%20theory-based%20learning%20to%20real%20engineering%20problems.%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%3EEnsuring%20that%20more%20than%20600%20students%20from%20around%2069%20countries%20receive%20the%20same%20high-quality%20learning%20experience%20that%20they%20would%20have%20pre-COVID%20requires%20a%20better%20solution%20than%20simply%20moving%20content%20online.%3C%2FP%3E%0A%3CP%3E%3CBR%20%2F%3ETo%20meet%20these%20new%20needs%2C%20a%20team%20of%20Teaching%20Fellows%20and%20future%20chemical%20engineers%20have%20developed%20the%20CREATE%20Lab%20(Chemeng%20Remote%20Experience%20Augmented%20through%20TEchnology%20Laboratory).%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CBR%20%2F%3ECREATE%20Lab%20enables%20students%20to%20participate%20in%20their%20chosen%20lab%20project%20remotely%20using%20three%20key%20pieces%20of%20technology%3A%20Microsoft%20Hololens%20headsets%2C%20controllable%20cameras%20and%20QR%20codes.%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CDIV%20class%3D%22slide-show%22%3E%0A%3CDIV%20class%3D%22icflex-viewport%22%3E%0A%3CUL%20class%3D%22slides%22%3E%0A%3CLI%20class%3D%22slide%20slide-1%22%20data-order%3D%221%22%20data-thumb-alt%3D%22%22%3E%3CP%20class%3D%22caption%22%3EQR%20codes%20are%20placed%20around%20the%20labs%20and%20direct%20students%20to%20additional%20learning%20resources%3C%2FP%3E%0A%3C%2FLI%3E%0A%3CLI%20class%3D%22slide%20slide-2%20icflex-active-slide%22%20data-order%3D%222%22%20data-thumb-alt%3D%22%22%3E%3CP%20class%3D%22caption%22%3EExample%20of%20an%20augmented%20reality%20guide%20which%20provides%20instructions%20for%20operating%20equipment%3C%2FP%3E%0A%3C%2FLI%3E%0A%3CLI%20class%3D%22slide%20slide-3%22%20data-order%3D%223%22%20data-thumb-alt%3D%22%22%3E%3CP%20class%3D%22caption%22%3ECCTV%20cameras%20have%20been%20placed%20around%20the%20undergraduate%20labs%20and%20can%20be%20controlled%20remotely%3C%2FP%3E%0A%3C%2FLI%3E%0A%3CLI%20class%3D%22slide%20slide-0%20clone%22%20data-order%3D%220%22%20aria-hidden%3D%22true%22%3E%3CP%20class%3D%22caption%22%3ETeaching%20Fellow%20Vijesh%20Bhute%20wearing%20the%20Microsoft%20Hololen%3C%2FP%3E%0A%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3C%2FDIV%3E%0A%3C%2FDIV%3E%0A%3CP%3EOne%20member%20of%20each%20student%20team%20will%20be%20physically%20present%20in%20the%20lab%20and%20supported%20by%20a%20Graduate%20Teaching%20Assistant.%20Wearing%20the%20Hololens%20headset%2C%20they%20will%20connect%20to%20their%20remote%20colleagues%20using%20%3CA%20href%3D%22https%3A%2F%2Fdynamics.microsoft.com%2Fen-ca%2Fmixed-reality%2Fguides%2F%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EDynamic%20Guides%3C%2FA%3E%2C%20%3CA%20href%3D%22https%3A%2F%2Fwww.microsoft.com%2Fen-GB%2Fmicrosoft-365%2Fmicrosoft-teams%2Fgroup-chat-software%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EMicrosoft%20Teams%3C%2FA%3E%20and%20%3CA%20href%3D%22https%3A%2F%2Fwww.microsoft.com%2Fen-us%2Fmicrosoft-365%2Fmicrosoft-stream%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EMicrosoft%20stream%3C%2FA%3E%20what%20they%E2%80%99re%20seeing%20in%20real%20time.%20Their%20team%20members%20will%20be%20able%20to%20provide%20both%20verbal%20and%20written%20instructions%20and%20together%20they%20will%20perform%20experiments%20on%20working%20machinery%20and%20rigs.%26nbsp%3B%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%3ETo%20further%20support%20the%20virtual%20experience%2C%20remote%20access%20cameras%20have%20been%20mounted%20throughout%20the%20labs%20and%20will%20continuously%20stream%20video%20footage.%20Students%20will%20be%20able%20to%20move%20the%20camera%20and%20zoom%20in%20and%20out%20to%20easily%20capture%20equipment%2C%20readings%20and%20other%20essential%20components.%20Camera%20pre-sets%20will%20also%20allow%20students%20to%20%E2%80%98travel%E2%80%99%20between%20different%20key%20experiment%20locations%20in%20the%20lab.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CBR%20%2F%3EQR%20codes%20are%20strategically%20placed%20in%20positions%20of%20certain%20processes%20or%20equipment%20pieces%20and%2C%20with%20the%20help%20of%20the%20Microsoft%20HoloLens%2C%20allow%20students%20to%20access%20relevant%20information%20regarding%20their%20tasks.%20Students%20both%20in%20person%20and%20remotely%20can%20easily%20scan%20these%20codes%20with%20mobile%20phones%20which%20will%20take%20them%20to%20informative%20videos%20and%20information%20packs.%20To%20complement%20these%2C%20the%20team%20have%20also%20developed%20a%20set%20of%20dynamic%20guides%20which%20provide%20a%20step-by-step%20instructions%20on%20how%20to%20operate%20certain%20pieces%20of%20equipment.%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--593017203%22%20id%3D%22toc-hId--593017203%22%3E%3CBR%20%2F%3EWorking%20in%20partnership%3C%2FH2%3E%0A%3CP%3E%3CBR%20%2F%3EThomas%20Cheng%2C%20one%20of%20the%20undergraduates%20involved%20in%20the%20project%2C%20said%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CBLOCKQUOTE%20class%3D%22floating-pullquote%22%3E%3CSPAN%20class%3D%22quotation%22%3EThe%20CREATE%20Labs%20bring%20flexibility%20with%20lab%20work%20in%20a%20way%20I%20have%20never%20seen%20before.%20The%20use%20of%20technology%20is%20refreshing%20and%20with%20the%20use%20of%20augmented%20reality%2C%20I%20would%20say%20even%20the%20experience%20for%20those%20within%20the%20lab%20has%20been%20enhanced.%20For%20those%20working%20remotely%2C%20it%20is%20almost%20like%20you're%20standing%20there%20in%20person%2C%20able%20to%20work%20with%20your%20group%20face%20to%20face%2C%20just%20like%20before%2C%20but%20with%20access%20to%20content%20that%20helps%20you%20understand%20why%20you%20are%20actually%20doing%20the%20experiment.%E2%80%9D%3C%2FSPAN%3E%3C%2FBLOCKQUOTE%3E%0A%3CP%20class%3D%22floating-pullquote%22%3E%3CSPAN%20class%3D%22quotation%22%3E%3CSPAN%20class%3D%22quoted-from%22%3EDr%20Clemens%20Brechtelsbauer%3C%2FSPAN%3E%3CSPAN%20class%3D%22quoted-from-title%22%3EDirector%20of%20Chemical%20Engineering%20Education%3C%2FSPAN%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CBLOCKQUOTE%20class%3D%22floating-pullquote%22%3E%3CSPAN%20class%3D%22quotation%22%3E%22Often%20our%20students%20go%20on%20to%20work%20in%20environments%20where%20they%20are%20controlling%20machinery%20remotely%2C%20so%20it%E2%80%99s%20important%20that%20we%20provide%20them%20with%20this%20experience%20during%20their%20degree.%E2%80%9D%26nbsp%3B%3C%2FSPAN%3E%3C%2FBLOCKQUOTE%3E%0A%3CP%3EThe%20project%20forms%20part%20of%20a%20wider%20plan%20to%20transform%20the%20Department%20of%20Chemical%20Engineering%20curriculum.%20The%20goal%20is%20to%20introduce%20advanced%20learning%20technologies%20and%20replace%20the%20traditional%20way%20of%20teaching%20lab%20work%2C%20to%20enhance%20the%20student%20learning%20experience.%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%3EDr%20Clemens%20Brechtelsbauer%2C%20Director%20of%20Chemical%20Engineering%20Education%2C%20explained%3A%20%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CBLOCKQUOTE%20class%3D%22floating-pullquote%22%3E%3CSPAN%20class%3D%22quotation%22%3E%E2%80%9CRemote%20learning%20has%20received%20increased%20attention%20as%20a%20result%20of%20the%20pandemic%2C%20but%20it%E2%80%99s%20something%20we%E2%80%99ve%20been%20exploring%20for%20some%20time%20as%20it%20provides%20an%20excellent%20opportunity%20for%20students%20to%20prepare%20themselves%20for%20working%20in%20industry.%20%E2%80%9COften%20our%20students%20go%20on%20to%20work%20in%20environments%20where%20they%20are%20controlling%20machinery%20remotely%2C%20so%20it%E2%80%99s%20important%20that%20we%20provide%20them%20with%20this%20experience%20during%20their%20degree.%E2%80%9D%3C%2FSPAN%3E%3C%2FBLOCKQUOTE%3E%0A%3CP%3EThe%20CREATE%20Lab%20team%20will%20continue%20to%20develop%20the%20technology%20over%20the%20summer%20and%20will%20be%20actively%20seeking%20participation%20and%20feedback%20from%20other%20students%20during%20this%20process.%20You%20can%20follow%20the%20project%E2%80%99s%20progress%20on%20the%20team%E2%80%99s%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fwww.instagram.com%2Ficchemengcreatelab%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3EInstagram%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3Eand%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fchannel%2FUCUWwuHIERdxEdG5pdz3U4UQ%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3EYouTube%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3Eaccounts.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-1894495630%22%20id%3D%22toc-hId-1894495630%22%3EMicrosoft%20Learn%20-%20Learning%20Modules%20for%20Hololens%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FH2%3E%0A%3CP%3ESolve%20problems%20in%20real%20time%20with%20Dynamics%20365%20Remote%20Assist%3C%2FP%3E%0A%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Flearn%2Fpaths%2Fsolve-problems-remote-assist%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Flearn%2Fpaths%2Fsolve-problems-remote-assist%2F%3C%2FA%3E%3CBR%20%2F%3E%3CBR%20%2F%3ESet%20up%20a%20mixed%20reality%20project%20in%20Unity%20with%20the%20Mixed%20Reality%20Toolkit%3CBR%20%2F%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Flearn%2Fmodules%2Fmixed-reality-toolkit-project-unity%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Flearn%2Fmodules%2Fmixed-reality-toolkit-project-unity%2F%3C%2FA%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-1608687%22%20slang%3D%22en-US%22%3E%3CP%20class%3D%22abstract%22%3EThe%20Department%20of%20Chemical%20Engineering%20is%20transforming%20the%20delivery%20of%20lab-based%20teaching%20using%20Microsoft%20HoloLens%20technology%20and%20augmented%20reality.%20Chemical%20engineers%20at%20Imperial%20have%20been%20transforming%20how%20they%20teach%20practical%20lab%20skills%20to%20students%20based%20remotely%2C%20as%20part%20of%20the%20new%20multi-mode%20offering%20for%20students%20in%20the%20new%20academic%20year.%3C%2FP%3E%0A%3CP%20class%3D%22abstract%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Imperial%20Chemistry.jpg%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F214375i8238123B14C0A081%2Fimage-size%2Fmedium%3Fv%3D1.0%26amp%3Bpx%3D400%22%20title%3D%22Imperial%20Chemistry.jpg%22%20alt%3D%22Imperial%20Chemistry.jpg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3ENew%20and%20returning%20chemical%20engineering%20undergraduates%20will%20have%20the%20opportunity%20to%20use%20augmented%20reality%20(AR)%20to%20complete%20lab-based%20experiments%2C%20using%20Microsoft%20HoloLens%20technology%20to%20connect%20and%20interact%20from%20wherever%20they%20are%20in%20the%20world.%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-TEASER%3E
Version history
Last update:
‎Aug 24 2020 11:29 AM
Updated by: