When Microsoft Flight Simulator came out a year ago, I was completely entranced by it. I’m not particularly into hyper-accurate flight simulation games, but the concept of a game where you can go anywhere in the real world and see an accurate 3D representation of that real-world space, blew my mind. What a concept! That also got me thinking: could I make something like that myself? As a solo creator I don’t have the benefit of a team of many engineers working with map data for years.
How Flight Simulator’s World Works
Microsoft Flight Simulator uses three distinct types of world maps:
- Azure 3D Maps. Flight Simulator uses high-quality 3D photogrammetry data when it exists in Microsoft’s library. It’s pretty easy to identify these areas: there are relatively high-quality textures, but often buildings and landmarks like trees look a bit worn down as the 3D models aren’t perfect.
- AI-Augmented 2D Satellite Maps. For areas where no good 3D map data is available, Flight Simulator takes 2D satellite map footage instead. Deep learning techniques are applied to infer what sort of 3D buildings and landmarks might exist, and then a library of hand-authored 3D models are programmatically placed in the world. If you’re visiting a relatively non-descript location in Flight Sim where buildings look pristine and high-quality, you’re probably in an AI-augmented zone.
- Hand-authored content. A lot of Flight Simulator areas are modeled by hand to resemble their real-world locations. Notable examples include the hand-modeled airports in the Deluxe and Premium Deluxe editions of the game.
The second and third types of map data require an exorbitant amount of work, either designing procedural generation algorithms or doing 3D modeling and level design by hand. But the first one is doable!
Specifically, the Bing Maps SDK gives us exactly what we want: a drop-in Unity component that, given a location (via latitude/longitude coordinate pair) will render a 3D scene containing that real-world location, powered by the same 3D maps data that Flight Simulator uses. That’s so cool! I'm going to spend the remainder of this blog post walking you through setting up a pretty simple VR “flight sim” example that will let you move around a 3D representation of any location in the world!
I’m going to assume you have basic familiarity with Unity to configure a few GameObject components and write a small amount of custom code. If this is more than you’re comfortable with, please check out other tutorials to familiarize yourself with Unity’s UI.
Get a Bing Maps API Key
To use this API, you need an API key. Sign in or sign up at the Bing Maps Portal. Once you’re logged in, create a new API key by going to My account -> My Keys -> Create a new key. Name it whatever you want, choose a “Basic” key, and for now select “Dev/Test” as the application type.
After creating a key, keep this page handy, as you’ll need the key in a few minutes.
Download the Bing Maps SDK and open the sample project
For now, we’re going to use the sample starter project included with the Bing Maps SDK, although there are instructions if you would prefer to integrate the SDK manually with an existing or new project using the Unity Package Manager.
Grab the 0.10.0 release of the SDK from this page and unzip it. In Unity Hub, click “Add”, then locate this folder and select the “SampleProject” folder. If you don’t have a Unity 2019.4 release, Unity Hub may prompt you to download and install it.
Creating our scene
Start by adding a new scene by selecting File -> New Scene.
Add a new empty GameObject to your scene by right-clicking anywhere within the Hierarchy window and selecting “Create Empty”. I called mine “MapManager” (since it’s going to manage our 3D map component).
In the right-side inspector pane for this new GameObject, click the “Add Component” button and search for the “MapRenderer” component. This is a Unity component provided by the Bing Maps SDK for Unity that will automatically render our 3D map in our scene!
In the GameObject inspector pane, enter your API key from the Bing Maps Portal earlier into the Developer Key field in the Map Session pane. Add in the lat/long coordinates for the location where you want players to start their adventure into the “Location” fields at the top of Map Renderer.
I set mine to a latitude of 37.7958421 and a longitude of -122.3959688, which is the Ferry Building in downtown San Francisco. I found this by looking up the Ferry Building in Bing Maps and right-clicking the Ferry Building. If there’s a location you'd rather explore in 3D, you can get its lat/long coordinates the same way and enter them instead.
Now you will see a map of the world in your Unity scene! If you adjust the “Zoom Level” slider below latitude and longitude, you can see it zoom in on San Francisco (or your chosen location). I’ve found that setting the zoom to 19 is a good fit for our use case.
If you were to open this in VR now, it would look more like a diorama than a lush living world. Let’s make it bigger to match the scale of a person in VR! In the Transform section at the top of the inspector, set it’s the scale to (20, 20, 20). Down in the Map Layout section of the Map Renderer tab, set the Local Map Dimensions to (3, 3). This should make the map much larger and more to-scale with a human-sized VR viewer.
One final piece to the puzzle is moving the camera position. Since we’ve grown the map to be as tall as 20 units tall, but the camera is still down at the original starting point, trying to play this in VR would spawn you in the middle of the ground! Select the Main Camera object from the scene inspector, and set the camera position to (0, 12, 0). This should get you a good vantage point of the scene.
Give it a try!
Click the “Play” button in the top middle of the Unity editor now and see a fully-rendered 3D map of the location you’ve entered! If you have a VR headset connected to your computer, it should automatically render to the VR headset as well, including using head tracking to control the camera and let you look around.
Moving Around
It’s so cool that with such little work we’re already able to drop ourselves into a 3D rendering of the actual world! But we can take this further: let’s add some movement! As mentioned, I’m not particularly into detailed simulations, so for now let’s just add some super simple movement code. Hold down a button on your VR controller to move at a reasonable speed in whatever direction you are currently looking.
With our custom GameObject open in the inspector, click “Add Component”, then “New script”, and call it CameraMovement. Double-click the C# script name in the inspector to open it in your text editor of choice, then paste in the following code. Don’t worry, I’ll walk you through it in a moment!
using UnityEngine;
using Microsoft.Geospatial;
using Microsoft.Maps.Unity;
public class CameraMovement : MonoBehaviour
{
void Update()
{
if (Input.GetButton("Fire1")) {
var heading = Camera.main.transform.eulerAngles.y * Mathf.Deg2Rad;
var speed = 0.00015f;
var map = GetComponent<MapRenderer>();
map.Center = new LatLon(
map.Center.LatitudeInDegrees + speed * Mathf.Cos(heading),
map.Center.LongitudeInDegrees + speed * Mathf.Sin(heading)
);
}
}
}
Let’s walk through this!
if (Input.GetButton("Fire1")) {
Every time the Update function is called (for a VR project, usually 72 or 90 times per second), we’re going to check if the player has pressed the “Fire1” button on their VR controller. On most VR headsets, the Mixed Reality Toolkit maps this to a prominent “main” button on the face of the controller (and not a trigger). If the player is pressing the button on this frame, we will want to move the player!
var heading = Camera.main.transform.eulerAngles.y * Mathf.Deg2Rad;
This fetches the current y-axis value of the main camera, which is being controlled by the VR headset. The y axis is the equivalent of turning your head left or right. This means we won’t let players move up or down, but that’s fine for our simple use case. The Mathf.Deg2Rad multiplication is just to get that value as a radian instead of degrees since that’s what our trigonometry functions will take a few lines down.
var speed = 0.00015f;
How many world units players will move per frame they’re moving. I hand-tuned this value based on testing. Feel free to make yourself faster or slower!
var map = GetComponent<MapRenderer>();
Grabs a reference to the MapRenderer map object off of our GameObject.
map.Center = new LatLon(
map.Center.LatitudeInDegrees + speed * Mathf.Cos(heading),
map.Center.LongitudeInDegrees + speed * Mathf.Sin(heading)
);
This may look complicated if you’re rusty on basic trigonometry, but it’s not that bad!
The first thing to understand is a classic videogame movement trick: we’re not moving the player, we’re moving the entire world around the player like they’re on a treadmill. We’re neither moving the camera nor the position of the map view in the scene; we’re simply shifting what latitude and longitude coordinates are at the center of the map, which will result in the map moving (slightly, since we’re dealing with very small changes) and a perception that the player is moving.
Given our goal of moving 0.00015 units per frame, multiplying that speed by the sin and cosine of the angle ensures that, while we shift both the latitude and the longitude, we change them proportionately so that the total distance moved is exactly 0.0015.
If you click “Play” and put on your VR headset, it all should work now! Pressing the “fire1” button on your controllers (it may take some experimentation to find it) should move you in the direction you’re looking.
I don’t know about you, but I’m pretty blown away at how impressive this looks after such relatively little work! You could imagine expanding this to include more complex controls, real flight physics, goals and objectives, or whatever else you want, but for now this is a heck of a demo for such a short period of time and such little code.
Building more weird things with real-world data
Separate from how much I love this specific experience of being able to soar around my favorite cities of the world, this also points towards one of my favorite ways to build games and mixed reality experiences: on top of already-existing real-world data sources.
If you’re working by yourself, building a 3D scale model of whatever city you live in is likely prohibitively expensive. The thought of building a game where you can fly around an entire city-scale map, let alone a world-scale one, is extremely time-consuming and requires a whole bunch of disparate skillsets, even if you’re taking advantage of resources such as existing Unity Asset Store assets. In this case, finding something like the Bing Maps SDK that provides you with a 3D scene makes it categorically possible for you to make projects and experiences that wouldn’t be possible otherwise.