Enhance Your World With The Hololens

Imagine a pair of glasses that allows you to support your workflow using virtual Holograms. Sounds pretty convenient, right? While we are not there quite yet, the Microsoft HoloLens pushes the boundaries of Augmented Reality. In the past months, I have developed an application for the HoloLens as a proof of concept for a customer in logistics. This application shows the capabilities of the HoloLens in terms of navigation through a warehouse. In this blog, I will navigate you through the developed application and provide you with a look behind the scenes.

How does the Hololens work?

The HoloLens is a pair of mixed reality smart glasses developed and manufactured by Microsoft. You see the real world through the device’s clear lenses, and images (holograms) are projected out in front of you, up to several meters. These holograms can be locked to their position in the physical world. Meaning, you can walk around the holograms and view them from any desired angle, while they stay exactly where you put them.

The idea: navigation in a warehouse

The idea for the proof of concept: create an application that would allow a (new) employee of a company to be guided through a warehouse. Meanwhile, they have to perform tasks to complete their route. The application can help navigate users to specific points in the building. The specific use case for this application is a warehouse where workers have to pick orders. Sounds relatively straightforward, doesn’t it?

Understanding the environment

The first challenge is to understand how the HoloLens recognises the room that we are standing in. The HoloLens uses a whole range of sensors to understand its environment. The application contains spatial mapping component from the Mixed Reality Toolkit, developed by Microsoft. The toolkit provides us with a lot of tools that we can use to make our lives a little easier.  We will use a tool called “Spatial mapping” to create a virtual copy of the room that we are in. This virtual map of the room allows us to make it seem like virtual objects are interacting with the real world. This makes it possible to hide objects behind walls, but it also lets us bounce a virtual ball on a physical wall, if we want.

Move from point A to point B

For the next step, I implanted something that I learned during my study of game design and development. How would we normally make an NPC (Non-PlayerCharacter) move from point A to point B, in a logical manner? In game development, a solution to this is to work with an algorithm named A*. This algorithm uses nodes (points in space) to calculate the best route to get from point A to point B. Point A can be set by doing a check for what the closest node to our position is. This is done automatically behind the scenes by the application. Point B is the point that we want to reach, referred to as our target. We let the algorithm do what it does best, and it will return us a path with all nodes between our current position and where we want to go. With this information, we can create a virtual guide that will guide the user to their destination.

Creating a network of nodes

The nodes I talked about previously are placed in the world by an administrator. Creating a whole network of nodes in a room can be very time-consuming. The good news is that the application is capable of saving and loading our nodes between sessions. This allows us to map a room or building once and use it over and over again. Theoretically, it should even be possible to share a mapped building between multiple HoloLenses.

The promise of augmented reality

So is everything great about augmented reality? Yes and no! The technology is certainly promising, and I am sure that augmented reality will make a big impact in a lot of industries. However, for now, we have to be realistic as well. The application that I have been working on has its issues that are mostly HoloLens related. Sometimes the HoloLens loses track of its position; this stops the application momentarily with a pop-up screen. Also, the spatial mapping is sometimes lacking a bit behind. This is because it takes about 5-10 seconds for the application to update our spatial map when we walk around or move real-world objects.

Great value to the business

Despite the HoloLens still having some limitations, the application is already a great way to see what the HoloLens can currently add to a business. The navigation works well in spaces that are unique with good lighting. Another great benefit of the HoloLens is that you can get more information on the go without having to hold a phone or tablet. This can be very useful when you have to carry something. While the HoloLens and augmented reality are certainly still young technologies, I believe it already shows great potential for the future. It has been a fascinating process in developing an application for the HoloLens.