Every day, we interact with our world through screens. Taking photos with our smartphones, connecting with friends on social media, setting timers, and using GPS maps—our digital devices increasingly influence our experiences.
But what if we could digitally add objects, textures, and people to our surroundings? Augmented reality technology does exactly that, blurring the boundaries between the physical world and the virtual world. As this technology develops and many industries adopt new applications, understanding augmented reality becomes more important.
What is augmented reality (AR)?
Augmented reality (AR) is a technology that uses a device like a tablet or specialty goggles to blend virtual and physical elements into one cohesive display: the AR device superimposes virtual elements onto an image of the physical world.
For example, you could point your smartphone toward your living room and use software to place different chairs in that room. You are superimposing a digital object (chair) onto a real-world setting (your room).
The difference between VR, AR, and MR
Augmented reality is not the only technology that lets a person interact with realistic digital objects or a virtual environment. There are three main categories of this technology, with important distinctions:
Virtual reality (VR)
Virtual reality (VR) dates back to at least the 1980s, but thanks to advances in technology, it’s taken off in the past decade. Virtual reality gives the user a fully immersive experience, typically through the use of virtual reality goggles. When a person enters virtual reality, they are fully unaware of the physical world around them. VR replaces the user’s surroundings with fully digital images.
Augmented reality (AR)
Unlike virtual reality, augmented reality (AR) virtually adds digital objects to a real-world environment. Using input devices like cameras, light sensors, depth sensors, and microphones, an AR app will place virtual objects in a user's environment, as seen through their mobile phone or AR-enabled devices. Augmented reality overlays digital images onto images of real-world settings and enables users to see something that doesn’t exist in their real environment.
AR has many many applications, like fashion, beauty, and interior decoration apps. For example, a beauty brand could let you try on makeup products digitally before you purchase them. Through your device's camera, the brand’s app can show you which lipsticks, blushes, and eyeshadows complement your skin tone.
Mixed reality (MR)
Mixed reality (MR), often referred to as AR 2.0, is a mix between augmented reality and virtual reality. MR is similar to augmented reality, making it easy to conflate the two. The terms “mixed reality” and “augmented reality” sometimes are used interchangeably.
However, there are some technical differences. The principal difference between mixed reality and augmented reality is mixed reality allows the interaction between virtual elements and the physical environment. In MR, digital objects can sense and react to changes in the environment. For example, augmented reality might overlay a static object onto an image, whereas mixed reality could be a filter that changes as the real-world object moves.
Types of augmented reality
There are a few main types of augmented reality, and each functions in a slightly different way:
Marker-based AR
Marker-based augmented reality hinges on image recognition of a predetermined physical cue to trigger the AR elements. This will often be a QR code, photo, or visual pattern that the AR software recognizes. As the name suggests, this type of AR depends on some sort of visual marker.
Markerless based AR
Markerless AR does not require a specific marker that the AR software recognizes. This is a sophisticated application of AR that requires more computing power. Markerless AR uses object detection to map out a physical space and add computer-generated images to a user's surroundings, as seen through mobile phone screens, smart glasses, or another interface.
Projection-based AR
Unlike marker-based and markerless AR, projection-based AR does not require the user to hold a mobile device. Instead, projection-based AR uses cameras and projectors to augment an environment by projecting digital images onto physical objects (like a wall or sculpture) or inserting images into a user’s real-life surroundings.
This could be the appearance of a hologram or a mapped projection that continually changes the texture of a wall. Projection-based AR allows multiple people to see the same augmented reality at the same time without each having to look at their device.
How does augmented reality work?
AR technology requires three main components to work:
1. Input
AR experiences need some sort of input from the real world—typically provided by mobile devices’ cameras. However, more sophisticated inputs can enhance the experience as well. Light sensors, depth sensors, microphones, and GPS can bolster a camera’s visual inputs. Essentially, the input component is any technology that allows you to collect real-time data from your environment.
2. Software
The most complex and invisible part of augmented reality is the work the computer does with the input it gets. AR software processes the real-time data, uses object recognition, and processes information on the depth, shape, and texture of the environment. Then it uses its processing power to accurately overlay virtual objects into the environment. Many augmented reality apps leverage artificial intelligence to efficiently ingest and process the data they get from the various inputs.
3. Output
Once the software has processed the input data and figured out where to overlay digital elements, it must then display the final image to the user. This is where output devices come in. An output device can be a head-mounted display that places visual elements in your field of vision. More common output devices include smartphones, tablets, and projectors that display digital objects in physical space.
For example, a popular example of augmented reality is the video game Pokémon Go. The mobile game uses a player’s smartphone camera for input, then the app processes the visual data using the mobile app as software. Finally, the mobile phone screen serves as the output, so the player can see a digital image overlaid into their environment when looking at their phone screen.
How does augmented reality work FAQ
What is the difference between VR and AR?
Virtual reality is a fully immersive experience that typically uses VR goggles to replace a user’s environment with a digital one. Augmented reality is a technology that adds digital elements to a user’s real-world environment, creating a hybrid of physical and digital. With augmented reality, the user can still see their physical environment plus a layer of digital images added to their experience of the real world.
Does AR use AI?
Sometimes. Augmented reality uses sensors to ingest data from a real-world environment, and then uses that data to accurately add digital content to that environment. The data can come from a variety of sources including cameras, light sensors,depth sensors, GPS, and microphones. Some AR apps use artificial intelligence to more efficiently process large amounts of input data.
What are the main components of AR?
Augmented reality depends on three principal components: input (cameras, depth sensors, microphones, and other sensors), software to process the inputs, and then output, or the final image on a screen or projector. These components work together in dedicated AR devices or the average modern smartphone.
