Microsoft HoloLens is a headset that combines augmented reality (AR) and mixed reality (MR) technology. It was created by Microsoft and uses the Windows Mixed Reality platform, which runs on the Windows 10 operating system. Some of the movement-tracking technology in HoloLens was originally developed for the Microsoft Kinect, a device used with the Xbox 360 and Xbox One gaming consoles that was released in 2010.

The early version of HoloLens, called the Development Edition, was released on March 30, 2016. It was sold to developers in the United States and Canada for $3,000. This allowed individuals, professionals, and companies to test the device before its full release. Samsung and Asus offered to work with Microsoft to create their own mixed-reality products based on HoloLens’ design and technology. In October 2016, Microsoft announced that HoloLens would be available for purchase in Australia, Ireland, France, Germany, New Zealand, and the United Kingdom. A special version of HoloLens, designed for businesses, included features like BitLocker security and was sold for $5,000 as of May 2017.

HoloLens 2 was introduced on February 24, 2019, at the Mobile World Congress in Barcelona, Spain. It became available for preorder at a price of $3,500.

Description

The HoloLens is a head-mounted display connected to an adjustable, padded headband. The headband allows the device to tilt up, down, forward, and backward. To wear the HoloLens, the user places it on their head and uses an adjustment wheel at the back of the headband to secure it around the top of the head. This helps distribute the device’s weight evenly for comfort, and the visor is then tilted toward the eyes.

The front part of the device contains sensors and hardware, such as processors, cameras, and projection lenses. The visor is tinted and includes transparent combiner lenses inside. These lenses display projected images in the lower half of the visor. The HoloLens must be adjusted to match the user’s interpupillary distance (IPD) or their usual vision.

Along the bottom edges near the user’s ears are two small, red 3D audio speakers. These speakers do not block outside sounds, allowing the user to hear both virtual and real-world noises. The HoloLens uses head-related transfer functions to create binaural audio, which helps the user locate sounds as if they are coming from specific points in a virtual space.

On the top edge of the device are two pairs of buttons. The display brightness buttons are located above the left ear, and the volume buttons are above the right ear. These buttons are shaped differently—one is curved inward and the other outward—so users can identify them by touch.

At the end of the left arm is a power button and a row of five small LED lights. These lights show the device’s system status, battery level, and power settings, such as standby mode. A USB 2.0 micro-B port is located along the bottom edge of the device. A 3.5 mm audio jack is also found along the bottom edge of the right arm.

Hardware

The HoloLens is an early version of an augmented reality device. The screens on the HoloLens use simple waveguide technology with a fixed focus distance of about two meters. Because the screens have a fixed focus, they cause a problem called the vergence-accommodation conflict.

The HoloLens includes an inertial measurement unit (IMU), which contains an accelerometer, gyroscope, and magnetometer. It also has four "environment understanding" sensors (two on each side), an energy-efficient depth camera with a 120°×120° field of view, a 2.4-megapixel video camera, a four-microphone array, and an ambient light sensor.

The main way to interact with the HoloLens is through a handheld device called the Clicker. This small, thumb-sized tool is used for scrolling and selecting. The Clicker has a clickable surface for selection and an orientation sensor that allows scrolling by tilting or moving the device. It also includes an elastic loop for holding it and a USB 2.0 micro-B port for charging its battery.

In addition to an Intel Cherry Trail system-on-a-chip (SoC) that includes the CPU and GPU, the HoloLens has a custom Microsoft Holographic Processing Unit (HPU). This special processor is designed specifically for the HoloLens. The SoC and HPU each have 1GB of LPDDR3 memory and share 8MB of SRAM. The SoC also controls 64GB of eMMC storage and runs the Windows 10 operating system. The HPU uses 28 custom DSPs from Tensilica to process sensor data and handle tasks like spatial mapping, gesture recognition, and voice recognition. According to Alex Kipman, the HPU processes "terabytes of information." One attendee estimated the display field of view of the demonstration units to be 30°×17.5°. At the 2015 Electronic Entertainment Expo, Microsoft Vice-President Kudo Tsunoda stated that the field of view is unlikely to change much in the current version.

The HoloLens has an internal rechargeable battery that lasts about 2 to 3 hours during active use, or up to 2 weeks in standby mode. The device can be used while charging.

The HoloLens supports IEEE 802.11ac Wi-Fi and Bluetooth 4.1 Low Energy (LE) connectivity. It uses Bluetooth LE to connect to the Clicker.

The HoloLens has been combined with hard hat systems for use in certain environments.

Applications

Since 2016, several augmented reality applications have been introduced for the HoloLens. Some of the applications available at the time of its launch included:

• Cortana, a virtual assistant developed by Microsoft.
• Holograms, a collection of 3D objects that users can place and adjust in size around them, such as animals, space shuttles, and planets.
• HoloStudio, a 3D modeling tool by Microsoft that can create models compatible with 3D printing.
• CAE VimedixAR, a commercial application that uses HoloLens technology to provide training in ultrasound and anatomy through augmented reality, improving patient safety and learning.
• A version of Skype, allowing users on devices like computers or phones to call and video chat with someone using HoloLens. During a video call, the user on a computer or phone sees what the HoloLens user is viewing, and the HoloLens user sees what the computer or phone user is seeing.
• HoloTour, a 3D virtual tourism application created by Microsoft and Asobo Studio.
• Fragments, a crime-solving adventure game developed by Microsoft and Asobo Studio.
• Young Conker, a platform game featuring a younger version of Conker the Squirrel, developed by Microsoft and Asobo Studio.
• RoboRaid (originally called "Project X-Ray"), an augmented reality first-person shooter game where players defend against robots using gaze to aim and a button or air tap to shoot.
• Actiongram, an app for creating and recording short mixed-reality videos using 3D virtual assets, released on March 30, 2016, in the United States and Canada.
• In November 2018, Microsoft announced plans to use HoloLens for military purposes, securing a $480 million contract with the U.S. government to provide AR headsets for soldiers.
• In April 2018, a new heart surgery was performed at Jagiellonian University Hospital in Krakow using HoloLens imaging. In April 2021, an augmented reality lab was opened at the Jagiellonian University Medical College to use HoloLens 2 for medical education.

Other applications introduced or demonstrated for HoloLens include:

• An interactive curriculum on human and nervous system anatomy developed by Case Western Reserve University and Cleveland Clinic.
• Architectural design tools, such as SketchUp Viewer by Trimble, the first commercially available HoloLens application.
• A version of the video game Minecraft by Mojang.
• Additional features for Autodesk Maya, a 3D design application.
• OnSight and Sidekick, software developed by NASA and Microsoft to explore mixed reality in space exploration. OnSight uses data from the Curiosity rover to create a 3D simulation of Mars, allowing scientists worldwide to view and interact with the environment using HoloLens. This tool can help plan rover activities by selecting targets in the simulation and using gestures to control menus. JPL plans to use OnSight to control rover operations by July 2015.
• FreeForm, a project by Autodesk and Microsoft that connects HoloLens with the Autodesk Fusion 3D design application.
• Galaxy Explorer, an educational app about the Milky Way, developed by Microsoft Studios and open-sourced after completion.
• A spacecraft design and visualization tool being developed by NASA’s Jet Propulsion Laboratory (JPL).
• In November 2015, Volvo and Microsoft displayed a prototype HoloLens system at Microsoft’s headquarters in Redmond, using the S90 luxury car as a demonstration.
• CAE VimedixAR, the first ultrasound training tool integrated with HoloLens, allowing learners to interact with 3D holograms of human anatomy.
• Holoportation, a technology that captures and transmits high-quality 3D models of people in real time, enabling users to see and hear remote participants in 3D using HoloLens.
• HoloSurg, used in April 2017 by surgeons in Spain to visualize MRI and radiography data during a surgery on a patient with a malignant muscle tumor.
• Microsoft Dynamics 365 Remote Assist, which can send a secure live video feed to a nearby computer screen when combined with Microsoft Teams.
• University education programs for medical and science students using 3D models to study subjects like anatomy, physiology, and pathology.
• Dawn Chorus, a mixed reality experience by artist Sarah Meyohas that combines digital birds and music with the Yamaha Disklavier piano and HoloLens technology.

Interface

The HoloLens uses voice commands, looking at something, hand movements, and a controller as the main ways to control it. Looking at something, like moving the head, helps the user focus on parts of an app. To choose items, such as buttons or apps, the user taps in the air, like clicking a mouse. Holding the tap moves an item, and some actions can be done with voice commands.

The HoloLens includes parts from the Windows desktop. A "bloom" gesture, which works like pressing the Windows key on a keyboard or the Xbox button on a controller, is done by spreading fingers with the palm up. Windows can be moved and resized. Virtual items, like windows or menus, can be fixed in a place or attached to objects, or they can follow the user as they move. The top of each window has the name on the left and buttons for managing the window on the right.

In April 2016, Microsoft created an app for Windows 10 computers and mobile devices. This app lets developers use apps on the HoloLens, type on a keyboard or computer, watch videos from the HoloLens on another device, and take photos or videos remotely.

Developing applications for HoloLens

Microsoft Visual Studio is a software tool used to create applications for HoloLens. These applications can be tested using the HoloLens emulator (included in Visual Studio 2015) or with the HoloLens Development Edition.

HoloLens can run most apps designed for Windows devices. These apps appear as flat windows on the screen. Some features from Windows 10 are not available yet on HoloLens, but many apps can work on all Windows 10 devices, including HoloLens. The same tools used to make apps for Windows computers or phones can also be used to create apps for HoloLens.

Apps that use 3D graphics rely on special tools called Windows Holographic APIs. Microsoft suggests using Unity engine and Vuforia to build 3D apps for HoloLens. However, developers can also create their own tools using DirectX and Windows APIs.

Reception

In November 2018, Microsoft received a contract to provide 100,000 HoloLens mixed reality glasses to the U.S. military for $479 million. The devices are designed to improve soldiers' ability to fight effectively, move quickly, and understand their surroundings to gain an advantage over current and future enemies. Before the start of the GSMA Mobile World Congress 2019 in Barcelona, fifty Microsoft employees wrote a letter to their CEO, Satya Nadella, and President, Brad Smith. They stated they would not help create technologies used for warfare or oppression and asked company leaders to cancel the contract.

Microsoft now offers a rental option for HoloLens devices, allowing clients to use them without purchasing them outright. Microsoft works with a company named Absorbents to provide this rental service.