There is no Reality, Only Experience.

What is reality? It’s one of those seemingly simple questions that, once you dig a little deeper, is not as easy to answer as you might think. Even in an analogue world, each person’s reality is unique, like their identity^[1]Learn more about identity in this episode of the Pilote Podcast. In fact, reality and identity are closely linked – both are made up of thoughts, feelings and experiences. So unlike a Bored Ape NFT, each person’s reality is unique.

Digital technology has made the concept of reality even more complex. Leaving aside the ‘living in a simulation’ theories as depicted in films like ‘The Matrix’, ideas like the ‘metaverse’ are bringing virtual reality visions of the future closer to being available. Star Trek’s Holodeck or more recently, the immersive game world inhabited by the characters of ‘Ready Player One’ are not too far away… perhaps.

At a more practical level, technology has given rise to new concepts of reality. The most well known being Virtual Reality and Augmented Reality. But reality is more nuanced than that, so just as ecommerce has become commerce, the distinctions between VR and AR will blur into a kind of extended reality or XR and then it will just become… reality. However – after nearly 30 years of ecommerce, we are still making the distinction, so different reality technologies will be with us for a while.

5 Types of Extended Reality (XR)

From a technology point of view, there is a virtuality continuum – from purely analogue and physical to fully digitally immersive. On that line there are 4 kinds of technologies that are emerging.

Augmented Reality (AR)

This term describes the most basic type of extended reality. You could argue that something like an audio tour of a gallery is augmented reality, but is more commonly understood as a visual overlay – like a head’s up display. Early implementations included printing QR codes in magazine ads that could be read by a smartphone camera and display more information. Today, examples of AR in practical everyday life might be adding a Snapchat filter, using Google maps to navigate around a city or translating a menu in real-time.

AR has become more sophisticated through the development of machine learning, artificial intelligence and more accurate sensors. The applications of AR are greatly increased with the addition of gyroscopes, accelerometers, GPS, and image recognition. Suddenly AR can be used for games like Pokemon Go^[2]The highest month for Google searches for augmented reality was June 2016 when Pokemon Go was launched , working out whether an IKEA sofa will fit in your living room or seeing where the sun will set in the middle of winter from the balcony of the apartment you want to buy in New York.

Augmented reality has become pervasive for many people, but there are still a lot of applications that could be developed. Barriers to this kind of development include privacy concerns – the best experiences will be enabled by the sharing of personal data, including location and behaviour.

Mixed Reality

Mixed reality has been with us for a while – think of a flight or powerboat simulator. It adds to augmented reality by taking into account different interfaces and using more of the senses. Senses like proprioception or kinaesthesia which is your ability to know where the end of your fingers are.

Next generation mixed reality devices utilise hand and eye tracking, speech input, spatial sound and mapping and positioning of objects in both physical and virtual spaces to deliver the experience. Like augmented reality, this would not be possible without processing power and high-speed data transfer.

If done right, mixed reality may be a way to reduce cybersickness – sometimes experienced in virtual reality when there is a conflict between what the eyes are seeing and what the body is experiencing.

The applications for mixed reality are endless, but practically in the short term, training is an obvious one. Mixed reality offers the ability to train using simulated hands-on experiences interacting with 3D models. This is especially useful in professions such as medicine, engineering or aviation where intense experiential training is required. Mixed reality training can help with muscle memory and mistakes have no real-world consequences. Interacting with 3D models also has applications for product development.

One interesting application is the ability to transform your environment. For example, you could spawn walls to toggle degrees of privacy or add decorative objects into a dull space. Mixed reality has some very interesting user cases for digital pets.

Audio mixed reality is an emerging field too. Imagine your soundscape changing through the use of earbuds.

Assisted Reality (aR)

Is a subset of mixed reality, a bit further towards the analogue end of the virtuality spectrum. For those who remember Google Glass, it provides support through a small display in front of the eye, or in the field of view. While people found wearable smart glasses a bit creepy and weird, such devices have useful applications in the workplace. Most assisted reality headsets are expensive for casual use…

Assisted Reality could also utilise headphones or other wearable devices. With AI, machine learning and 5G all becoming more readily available, applications like real-time translation at an affordable cost is not far away.

Virtual Reality (VR)

Totally immersive digital experience with no sense of the real world. This is the reality that a future version of the ‘metaverse’ might be like. Most implementations and applications are still quite clunky and there is the not-so-small matter of cybersickness to deal with, but there is a lot of investment going into this space – to develop hardware like headsets and haptic user interfaces as well as virtual worlds.

Mass consumer adoption might still be a way off, but industrial and B2B uses are becoming more common – even the humble video conference may soon be a virtual reality event.

… that’s only 4 kinds of reality. What about the 5th?

There is no reality. There is only experience. The current focus is on how reality is changed, augmented or enhanced by digital technology, but humans have been altering their reality for thousands of years using substances. Perhaps altered reality needs to be in the list…