What Does the Future Hold for both VR and AR?

Where Charles Babbage may be seen as the ‘father of the computer’, Ivan Sutherland can be viewed as the ‘the father of virtual reality (VR)’. Back in 1968 Sutherland, and one of his students, Bob Sproull created the first virtual reality head-mounted display (HMD), which they named ‘The Sword of Damocles’.

This was the culmination of a paper he presented three years earlier, in 1965, in which he presented his image of the ‘ultimate display’ – a virtual world that could be viewed through an HMD and which replicated reality so accurately that the user would be unable to distinguish between that virtual reality, and real life itself. As Sutherland put it at the time: “The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal. With appropriate programming such a display could literally be the Wonderland into which Alice walked.”

It would be both remiss and inaccurate to pinpoint this event as the jumping-off point for the development of VR as Sutherland was focused primarily on VR delivered through an HMD. Two years earlier, a military engineer by the name of Thomas Furness created the first flight simulator for the USAF. Thanks to Furness, considerable funds were allocated towards the development of improved flight simulators which, consequently, greatly helped with the overall development of VR.

Thus, we have two diverse, yet closely connected paths down which the route to improved VR was concerned – VR provided by screens such as the 1972 180-degree field of vision three-screen GEC flight simulator, and VR provided by total visual immersion with a virtual 360-degree environment, such as VPL Research’s 1987 VR goggles, the first to be offered to the public on a commercial basis.

To a degree, VR then primarily focused on either the aerospace sector, such as the VR system designed to operate the Mars robot rovers using ‘Computer-Simulated Teleoperation’, or VR arcade gaming machines such as Virtuality, launched in 1991. However, some ‘out-of-the-box’ thinking in 1997 helped to remove the blinkers worn by VR creators when Virtual Vietnam was developed to help combat soldiers deal with the effects of PTSD though a form of ‘exposure therapy’. Beyond that, Google introduces its Street View, initially covering five US cities to provide an ‘immersive walkthrough’ these cities through the use of its patented dodecahedral camera mounted on the roof of a moving car.

Then came the ‘quantum leap’ in the world of VR with the arrival of Oculus Rift, the first headset offering a 90-degree field of vision and whose image quality relied on computer power. Created by an 18-year-old entrepreneur, Palmer Luckey, the company raised $2.4 million through a Kickstarter fundraising campaign, and only two years later, Oculus VR, the company, was bought by Facebook for US$2 billion.

VR headsets still had one principal restriction to the user’s total sense of being in a virtual world, and that was the need for them to be ‘tethered’ to a PC as the development of less cumbersome headsets came at a cost, computing power. It is only in the last few years that ‘wireless’ headsets have been developed, but they still lack the computing power to create a realistic virtual world, instead being aimed more at the gamer.

BY 2016 numerable major ‘players’ had entered the VR marketplace, such as Apple, Google, Facebook, Sony and Samsung. While improving the visual VR experience, further research into the audio aspects of the virtual world saw the development of dynamic binaurial audio, and Gloveone launched its highly successful Kickstarter campaign to further develop gloves which allow users to feel and interact with virtual objects.

Before we look towards the future, we should also talk about AR and the difference between AR and VR. Augmented reality (AR) involves adding a computer-generated augmented element to a real-world environment. Augmented reality research focuses on the interaction between the real world and augmented reality. It is with the inclusion of AR to the world of VR that many, many more opportunities arose for their combination, whether that was to influence our entertainment, shopping experience, how we work, and more.

Previously, much of the research into VR was financially driven. In other words, billions have been invested in VR, but primarily in relation to the world of gaming and entertainment, as that is perceived to be the area which will generate the greatest volume of revenue and prove to be the optimum return on investment (ROI).

Of course, as with any new technology, early innovation soaks up considerable financial resources where research funding is concerned. However, as technology becomes more mainstream, smaller enterprises are able to carry out their own R&D at a fraction of the cost incurred by ‘first to market’ innovators. These companies are more innovators, and it is that innovative spirit that has seen the uses for VR and AR expand exponentially.

It should come as no surprise that more and more companies are exploring the VR and AR world as, after all, that sector of the market generated US$18 billion in revenue in 2018. More to the point, that figure was virtually double that of 2017, so the trend for VR- and AR-related products is clearly on the increase.

So, let’s look at some of the less well-known areas where VR and AR are currently being explored. For example, how we perceive our personal movement alters as we age. Consequently, movement within a VR world cannot adopt a ‘one-size-fits-all’ approach if it is to remain an accurate representation of the real world across the whole age spectrum. Scientists at York University’s Centre for Vision Research have devised a fascinating challenge for people of all ages to estimate how far they have travelled in a virtual world. To avoid them ‘fixing’ upon a static object as we would do in real life, they filled the sides to the path being ‘travelled along’ with turning and disappearing/reappearing lollipops!

The COVID-19 pandemic and rules regarding social distancing and limiting of attendees at events forced VR into the spotlight, with numerous ‘virtual events’ hurriedly (and successfully) put together. This year, the purchase, or hiring of VR headsets will become a prerequisite for the film industry’s annual Cannes Film Festival which will now be a virtual event, Cannes XR. There will also be numerous virtual parties to attend – a critical element of the film industry’s need to see and be seen, to network effectively.

Recently, HTC Vive carried out a survey to ascertain what percentage of businesses used VR in the workplace and if they saw a post-pandemic place for it in the workplace of tomorrow. 26% responded that they had already used VR, while a further 52% were keen to see a greater integration of VR in the workplace. What became abundantly clear from the survey was that the biggest barrier to the adoption of VR in the workplace is not a lack of enthusiasm, but a lack of understanding of how to incorporate VR into the working environment.

The world of medicine, particularly surgery, has embraced the potential for VR as a means to train surgeons, and the future will see even greater inroads into this field. So effective and realistic is the combination of VR with haptics (the sense of touch) that the Royal College of Surgeons has recently accredited the surgeon’s version of a pilot’s flight simulator. FundamentalVR has developed a platform, Fundamental Surgery, which combines haptics with VR to create a hip replacement training module.

Richard Vincent, founder and chief executive of FundamentalVR, said: “With Fundamental Surgery, we have developed a completely safe and realistic teaching environment for surgeons to learn and hone their skills combining virtual reality with tactile feedback that is so important for developing the muscle memory associated with different procedures.”

Now, what if someone were to ask you if you wanted your business data presented on an Excel spreadsheet, or presented to you virtually? What would you say? Dell’s head of VR, Gary Radburn, thinks this is definitely an area to be explored. Dell’s DatavisVR has already created a platform that allows users to take information from a traditional 2-D spreadsheet, such as an Excel document, and convert it into 3-D using VR technology.

“You can interact with your data in different ways that you couldn’t do before,” Radburn said. “If we move into the third dimension inside of VR, we can actually look through data in 3-D, we can see different trends and data being presented in different ways. For data visualization, it’s a more efficient way of using a spreadsheet. You can make connections and linkages a lot easier because we look in 3-D anyway.” The perfect example of VR meeting AR is in the real estate business. No longer will your decision to buy a property be based on an hour spent wandering around it wondering if it ticks all the right boxes. Instead, in a virtual world, you can decorate and furnish your new home and get a realistic idea of what it would be like to live in it, without having to leave the comfort of your current, real-life living room.

While we have concentrated heavily on VR, the world of AR is no less important, and it is probably as much in its infancy as VR is. However, perhaps one of the major differences between the use of VR and AR in your life is the size of the necessary ‘hardware’. AR has the potential to become adopted more quickly and more effectively in everyday life owing to the fact that, at present, VR require you to wear a cumbersome headset that doubtless, in twenty years’ time, will look as ridiculous as the first ‘mobile phone’ which was the size of a brick and the attached battery the size of a small briefcase.

AR spectacles are only marginally larger than a standard pair of spectacles, the reason being that they don’t have to provide a totally immersive virtual world, but instead are used to provide additional imagery or information to the existing world, that imagery appearing within the lens of the spectacles. Many of you may have already come across these as they have been in use for eye tracking for several years to enable us to learn about visual patterns when presented with information. This has been effectively used by supermarkets who ask shoppers to wear eye tracking spectacles to help them decide where the optimum place on a shelf would be to place a product. Eye tracking spectacles are now seen as an effective means to better understand sight, perception, reaction and emotions.

Consequently, an EU initiative has recently introduced an eye-tracking system with AR and mixed reality to create a more immersive AR experience – an almost VR/AR hybrid, “Seamlessly connecting humans into the digital loop via smart wearables is a global challenge that will transform the market,” says Frank Linsenmaier, coordinator of the EU-funded Digital Iris project. Market interest in making biometric data accessible for wearable technologies is extremely high. “This disruptive leap forward would occur once a robust, reliable and certified head-wearable system became available.”

Viewpointsystem glasses are connected to a portable minicomputer that records, compresses and stores data, enabling video live streaming to laptops and tablets - a software solution then exploits the data’s full potential. These glasses are extremely lightweight, so can be worn full-time and they also respond well in any light conditions. Critically, where other systems require the wearer to constantly shift their eyes by jumping back and forth between digital information and real life, these glasses display information in the line of sight where it can instantly be seen.

In seeing what can be incorporated within a pair of glasses, it shouldn’t come as any surprise to discover that an area that will see substantial development in the future is the AR contact lens. This year MOJO announced it is developing “the worlds first true smart contact lens” whereby the user can obtain visual information via their eyes, but without having to look at an independent screen. Instead, the image is projected towards the user’s iris from the surface of the contact lens.

“Mojo has a vision for invisible computing where you have the information you want when you want it and are not bombarded or distracted by data when you don’t,” said Drew Perkins, CEO at Mojo Vision. “The technology should be helpful, and it should be available in the moment and fade away when you want to focus on the world around you.”

The company sees these smart contact lenses useful in any number of situations, such as for firefighters, and especially for those who are perhaps suffering with limited vision. One of the company’s driving forces is to provide an alternative and safer option to invasive surgery.

As we pack more and more ‘power’ into ever decreasing spaces, the world of VR and AR opens up new possibilities on almost a daily basis. The actuality is currently restricted by technology, the potential is restricted only by our imagination.

Written by: David Dundas