For the past month, our Cloud Advocacy team has shared projects with you all that could be created with the Mixed Reality Toolkit. For this month, we're taking a bit of a detour to showcase a Mixed Reality project that can be created with Microsoft Power Apps!
Power Apps are organizational-specific applications or tools that allow you to create custom applications using many features, including a data platform that provides a flexible development environment to build custom apps for your business. Applications built with Power Apps offer great business logic to transform your manual business functions into automated processes. Also, these applications can be accessed via mobile devices or the browser, enabling users to create custom apps without writing code.
Power Apps provides an extensible platform for users with minimal coding experience to develop applications with rich business logic. In addition, creators find it easy to interact with data and metadata while using Power Apps.
Through Power Apps, you can create canvas applications. Microsoft Power Apps allows you to build business apps from a canvas with minimal coding. Creating a canvas app in Power Apps is as easy as dragging and dropping components onto the canvas, just as designing a PowerPoint presentation. You can also integrate business logic through various data sources.
For this week, we put Power Apps to the test to create an app that displays 3D models both directly on screen as well as in Mixed Reality! The result? An awesome learning experience that leverages the Smithsonian 3D API and open-source models! Joining me this week is Daniel Laskewitz, Sr. Cloud Advocate within our Power/Fusion Cloud Advocacy team. We partnered together to create this project and we're excited to tell you more about it!
The Idea
In my last post, I shared with you all a VR museum experience that leveraged the Apollo 11 mission models provided by the Smithsonian 3D Digitization project. I wanted to explore other collections and thus landed on the Coral Collection. As I thought about how I could turn this collection into a learning opportunity, I thought it'd be cool to bring the models into my own space and learn more about coral both in-app and in Mixed Reality! What I envisioned would be an app that provides a selection of coral to both learn facts and view the models in 'real life' without the need to visit the museum. The added convenience of bringing the museum experience to me was the icing on the cake.
One of the cool parts about the Power Platform is that there are many data sources to connect to. There are already more than 800 available out of the box! But, even when your data source is not one of those 800 connectors, you can create your own connector. The Smithsonian 3D Digitization project also has an API available which offers the possibility to search for 3D objects. If you could combine that with the idea of the app with the collection of corals, it would be a real killer app.
Finding Inspiration
I'm a huge fan of using Dribbble and Pinterest to find inspiration for creating projects. For this project, I went down the rabbit hole of searching for AR Museum Apps. Once I exhausted that option, I began to search for AR Learning Apps. Having this variety in form and function provided insight into the various ways creators are designing UI for AR learning experiences.
After browsing the work of many creative designers, I settled on the following design for the app:
A mockup of the app.
Creating the Main Screen
If this app were to ever be created as a full-fledged experience, I'd image that there'd be an introduction for each collection available in the app. Keeping that in mind, I decided to create an introduction screen for the Corals and Coral Reefs collection. I leveraged a very beautiful image on Unsplash by Scott Web for the background and configured some basic Power Apps components. I wanted to maintain the vibrancy of the sea urchins while also ensuring that the text on the screen would be legible. With that said, I added a black rectangle component behind the collection description and lowered the opacity so that the white text would stand out more without taking away from the beauty of the sea urchins. Another thing I made sure to do was try to incorporate roundness as best as possible. The default Power Apps button is round, however, I desired more curvature. Therefore, I increased the border radius to 50 and got exactly what I wanted!
Introduction screen for Corals and Coral Reefs collection.
Storing Models on OneDrive
With the Introduction screen complete, I needed to find a place to store the models. We ran into a bit of a hiccup while trying to render the models in Mixed Reality using the API, therefore, we needed an alternative solution. Fortunately, the models could be saved to OneDrive and in return a data connector could be made in Power Apps to OneDrive to reference the models. I'll admit, this part of the project took quite a bit of setup because I had to both download the models and properly structure everything within the OneDrive folder. When you're referencing models from OneDrive, you do so from an Excel spreadsheet that contains relative links to models within the overall project folder.
I'll break it down for you:
The folder structure for the project assets.
Essentially, I had to first create a folder for the models and a folder for the photos of the coral. I made sure to follow a simple naming convention that could be repurposed for naming models and photos - this proved to be beneficial for my memory! Then in the Excel spreadsheet, I created a row for each coral which included its species, a description, a photo, and a model. The photo and model columns contain a relative link to the location of the coral's model and photo.
Excel spreadsheet table for the coral.
After all the corals were added to the spreadsheet, I had to create a table of the data - this part (like all parts) is crucial because Power Apps pulls data from the table.
Creating the Galleries
After making the data connection in Power Apps to OneDrive, I was ready to create the galleries. After assigning the table as the data source, I configured the formula for the gallery components to display the image for the coral. I didn't want to take up too much space on the screen with words - especially since species names are relatively long. Instead, I opted to just show the image. I also added more roundness to the UI by increasing the border radius of the images so that its corners were rounded. It really gave a different look and feel to the gallery as opposed to the default straight edges/corners.
Gallery screen for the app.
Creating the Information Screens
The final step for this portion of the project was to add in the 3D and Mixed Reality components. Power Apps provides a variety of Mixed Reality controls. For this project, I chose to integrate View in 3D and View in Mixed Reality:
View in 3D - The View in 3D control enables you to view 3D content in the app. You can rotate and zoom into the model with simple gestures.
View in Mixed Reality - The View in MR control enables you to see how a particular item might fit within a specified space. The control creates a button in your app. When the button is pressed, an overlay of the selected 3D model (in .glb, .stl, or .obj file formats) displays onto the live camera feed of the device.
After adding the necessary components to the screen, I dragged and dropped in the Mixed Reality components to add in the wow-factor to the app. And honestly, it was like magic! I at most needed to reference the model selected in the gallery in the Power Apps formula bar - but I promise it was very easy! Specifically: galCoral.Selected.'3DModel'
Information screen for the app.
I'll hand things over to Daniel now to share more about the work he did for the custom connector - Daniel, take it away!
Creating a Custom Connector
I have been working on the Independent Publisher Connectors program for over a year now. This program enables you to create connectors for the Power Platform and make them available for every user of the Power Platform, without having to be the owner of the API. If you would like to build a connector for a service that you use, you’re welcome to submit a connector for that in the Power Platform Connectors GitHub repository.
I love building connectors for the Power Platform. So, when I heard about the API that’s available from the Smithsonian 3D Digitization project, I wanted to build a connector for that API immediately. The Smithsonian 3D API has only one operation – the File Search operation – so that makes it easy to develop a connector for it!
When you build the connector, always make sure you think about the person who will use your connector later. In the Power Platform, it could very well be that other app makers also want to include your connector in their apps, so why not make it user friendly?
On top of building the connector – I also created a search screen to search for 3D objects and a screen where you can look at and interact with the 3D object.
Smithsonian 3D Search UI
Workshop
Oh - before we go, we have to tell you something exciting! We turned this entire project into a self-led workshop! To share the ability to create and host your own Power Apps workshop, we collaborated together to create an entire workshop equipped with a slide deck and complete instructions from start to finish. You can find the workshop by visiting: https://aka.ms/mr-power-platform
The workshop consists of 5 labs and all the assets you'll need to create your own version of the app we created. We've even provided the full Power Apps solutions as well. You're welcome to swap out the models for your own! If you have any questions, feel free to submit a GitHub Issue and we'll follow up with you in the repository.
Microsoft
