Virtual Reality in the Automotive Industry

BY DAAN TERRA – DEVELOPER @ TOPTAL (with link to the original article here).

Despite some rough years, the automotive industry is still one of the most important economic sectors. Car manufacturers are continuously trying to put the current technologies to use in order to deliver the best vehicles. Virtual Reality (VR) and Augmented Reality (AR) technology is advancing rapidly as computers are becoming more powerful. The market of AR/VR has already become a billion-dollar market and it’s projected to keep growing well beyond a $120 billion market within a few years.

At the low-end of the VR market, Google blogged that their Cardboard has already shipped over 10 million units. It upgrades one’s smartphone to a VR viewer. They also distributed over 160 million apps for them. That’s not all, though, since third-party solutions are available as well. As for the more complete commercial products, Sony reported that almost a million units of their Playstation VR headset were sold.

And then there’s the automotive industry. We already laid out the growing market of app development for Car Infotainment Systems which is projected to exceed $35 billion by the end of the decade. With AR/VR becoming more mainstream, developers can also start moving it into your car. Creative minds will have an opportunity to conquer some of that market with new AR/VR apps.

Also, autonomous driving is on the verge of becoming commercialized. Carmakers are racing to develop a self-driving car that is good enough to release to the public and unleash on our roads. Last year, Google’s self-driving Waymo car drove an average of 5,000 miles on its own before requiring human intervention. This data is available because companies testing in California are required to make their driving data public to the Department of Motor Vehicles (DMV). Autonomous driving again opens up cars to become a much more interactive environment.

To connect the two fields, this article will discuss examples of AR/VR in the automotive industry. It will outline how AR/VR technology is helping the automotive industry deliver high-quality services and improve products. The examples will range from the industry side all the way to consumers, and VR developers, obviously.

Typically, when you start exploring the market for a new car, you need to visit a dealership. They usually display a few models and often don’t even have the color you’re looking for. Now imagine a VR environment. Within it, you’re able to customize any make or model in exactly the way you’re going to buy it. You’ll be able to walk around it and step into the driver seat. With the development of quality VR headsets, this technology can be brought to any showroom, and it needn’t cost a fortune, either!

Audi is rolling out this technology for their cars with the Audi VR experience. They may soon also start offering pre-recorded virtual test rides so you can have the experience of driving the car. This also makes it possible to let your customers see future products. For example Toyota utilized ZeroLight VR to virtually launch their new C-HR car. Even if the physical version does not exist yet, everybody can still experience what it will be like to have one.

Used-car dealerships such as Vroom are adopting this technology to showcase their available assortment remotely. It can be costly to physically bring products to your customers just for demonstration purposes. Using a VR replacement makes it that much more affordable. Likewise, there’s a limit to how far your customers are willing to travel. To see a VR version of the product or experience a virtual ride, they don’t need to travel as far; sometimes, they can see your goods from the comfort of their own home. It also makes it easier for you to present more than one of your products in the same amount of time.

The great thing about VR is that it allows you to show your consumers the future. It makes perfect sense for automotive marketing, as consumers often buy cars built to their own specs months before they actually roll off the assembly line.

With a virtual reality showroom, vendors can demonstrate how a paint job will look on someone’s dream car, or how a set of bigger alloys will fit the new look. Want to convince someone to do an interior upgrade? Or maybe you do car customization? Using AR, your customer can experience what their own car is going to be like after the customization.

This shows the very real impact AR/VR can have on a company’s sales process and business model. Similar showroom solutions could apply to many other industries. Let’s go over a few examples:

• Traditionally, real estate agents face the problem of having to bring the potential customer to the property in order to provide a realistic impression of what it looks like. Virtual Reality changes that process entirely.
• If it works for cars, it works for boats, recreational vehicles, light planes, and just about any other vehicle you can imagine.
• You could experience a hotel room in virtual reality before booking it. Expedia is already working on that.
• Travel agencies can immerse you in a location before you choose if you want to go there. And with AR, they could also provide you with a cheap digital tour guide. VR can even replace travel in the form of eTourism. Virtual Reality in Tourism provides a lot of information that can be used by tourists and the industry alike.
• Need to choose a seat at an event venue or stadium? VR gives you the real perspective from your seat. Rukkus sells tickets this way. Airlines can use the same approach to demonstrate different classes and seats available in their airliners.
• Buying a new dress or tux? Using AR, you could try on many choices without a hassle. This ease will make you try options you would never try in the real world. In VR, you could virtually attend that fancy party while trying on your new look. Or better yet, you could be at that picturesque dream location while picking your wedding dresses.
• Imagine shopping for a nice new carpet and immediately seeing how it matches your furniture and “ties the room together” in AR. Houzz offers this shopping experience with their app. Amazon is considering such technology too.
• What about more drastic changes to your interior? IKEA is doing VR for kitchens, while Lowes has a Holoroom to design your entire interior.

There is obviously a lot of potential in these and many more industries, but for now, let’s get back to automotive.

Probably the biggest technological milestone in the automotive industry is currently the self-driving car. Progress towards this is being made steadily, and, over the past years, several public trials have been launched. Naturally, a major concern is the safety of such autonomous vehicles. The research and development (R&D) for it is an incredibly complex process as the car needs to be able to handle any situation.

Testing software that controls the vehicle can be aided by VR technology. Obviously, the car doesn’t wear a VR headset and doesn’t have to be driven around town for testing. Instead, its sensors are replaced by data streams simulated in VR. Think of it as VR car driving brought to the next level.

I used to work at TASS International, where I helped develop a traffic simulation platform call PreScan. Its purpose is exactly that, to help car manufacturers develop and test their cars. Such a platform ties in with a broad spectrum of technologies and techniques: interactive editing, importing of databases and datasets, generation and transformation of 3D geometry, high-performance distributed parallel computation, 3D visualization, 3D dynamics, and modeling of physical phenomena.

So how does this help the testing process? Here’s how:

Test coverage should be high even though there are infinitely many traffic scenarios that a car may encounter. The primary way to increase coverage is by “driving” for as many hours as possible. Ideally, all that driving should be done for every new release of a car and even for subsequent software updates. All this driving makes for a slow process. Simulating the tests in VR can be much faster because parallelizing it just means adding more computers that also run simulations.

In a single night of testing, the software can be “test-driven” for thousands of hours.

Corner cases in testing a vehicle tend to be dangerous or costly. Is a laden truck able to avoid a pedestrian? How does the car respond if it’s hit at full speed by another vehicle? It’s easy to see that doing the same test in a VR auto simulator removes any risk of injuring people or damaging expensive resources.

Tests should be repeatable. From a software development perspective, this is obvious and, in many domains, fairly straightforward. In the physical world, it’s much less so. Want to know how a car responds under a certain set of weather conditions? Or on roads that are not available near the R&D site? Recreating specific physical conditions is hard and expensive, whereas in VR it’s a relatively simple matter of configuring the virtual world.

On top of technical safety aspects of a self-driving car, there is always the human factor to take into account.

Think of an occupant as a user of the car and it’s immediately clear that the User Experience (UX) is incredibly important. You don’t want the car to intimidate or frustrate users. Instead, it should be a pleasant ride. Determining how to achieve that is far from trivial. Again, researching all kinds of conditions with real-world cars would be tremendously expensive. In VR, it’s much easier, faster, and cheaper to study these aspects. Running an A/B test doesn’t require the production of two physical models, but only takes two virtual models.

Until cars are entirely autonomous, there is a complex transition of control between human and car. Tesla’s Autopilot will conveniently drive you across a highway, but after it reaches your exit ramp, you are expected to take control again. A huge UX challenge is how to transition this control. Officially, the human driver is always supposed to pay attention, even in autopilot mode. However, in reality, the ‘driver’ might be reading a newspaper, having a snack, or even taking a nap. In VR, research can be done safely on how to design this crucial interaction between car and driver.

Alternatively, imagine that you’re selling taxi rides in driverless cars. You’ll probably be wondering how to help your clients feel comfortable about getting into your self-driving cars. Maybe some soothing colors work well. What shape should the interior be? How close to the passenger should it park? How will it communicate with the occupants?

In VR, it’s easier to study the effects of human preference because variables can be controlled.

Unfortunately, accidents do happen. When they do, it can be difficult to establish exactly what happened. For insurance companies and law enforcement, it’s important to reconstruct how and why an accident occurred. After hypothesizing how an accident may have happened, the events can be simulated in a VR environment. Eyewitness reports from victims and bystanders can then be matched against the line-of-sight models in VR. This can serve as verification for eyewitness reports.

Such reconstructed scenarios are also valuable for the manufacturers, because they provide a rare real-world test case. In addition to conceived hypothetical situations, the real test cases cover realistic scenarios, allowing R&D teams to investigate how their vehicles would have behaved in similar scenarios and identify potential areas of improvement.

Government agencies can use the reconstructions to study road safety. In the future this may allow for better decision-making. A city planner could analyze traffic incidents to determine if different design choices would reduce the risk of accidents on a certain stretch of road. They can also use VR to optimize the look and feel of infrastructure design. For example, when a new bridge or tunnel is proposed, a 3D model in VR can provide a much more realistic sense of it. It’s easy to see that a similar approach could be used to aid the design of other automotive infrastructure, from highways to driveways.

Reconstructed scenarios can also be reused for training purposes. Think of training for ambulance and police addressing a dangerous highway accident. A VR setting offers a safe and controlled environment in which to perform the training.

Learning how to operate a vehicle costs a good chunk of resources. For regular drivers’ licenses, it is already fairly common to use a car driving simulator to teach students how to operate a car. This could be taken a step further using VR headsets. The decreasing cost of VR hardware is making it more and more attractive to use them. Students could first learn the basics at home and only hit the road after reaching a certain level of proficiency in vehicle control and traffic.

In the automotive industry and ancillary industries, the advantage might be even bigger. Think, for example, of heavy-duty equipment. These are high-cost machines, and there usually aren’t that many to spare for training. If VR is used to train new operators, that means the machines can remain in use for the actual work. Additionally, their environments are usually dangerous. By using VR, the operators can receive the needed training so they’ll be ready when they go into the field.

Following a first phase of VR training, AR applications can take over. Software can keep providing clues about how to operate until the training is complete. Once the user is experienced and the training is completed, the system can continue to use AR to help make the operator’s life easier.

Here, again, it is easy to see how this extends far beyond the automotive domain. Many scenarios are difficult to recreate and training people to handle them can be of great use to healthcare workers in chaotic crowd control scenarios or dangerous mining environments, including various emergency scenarios for all of the above.

Many fields will undoubtedly benefit from VR training, and the automotive industry is just one of them.

VR can be used to put people in situations that would be too dangerous in the real world.

Recently, Toyota used a driving simulator to demonstrate the effects of distracted driving using their TeenDrive 365. For example, they asked participants to take a phone call while driving in the simulator. This way the driver can be made aware of their reduction in attention and reaction speed. Doing it in VR allows doing so in a controlled and safe manner.

Similarly, liquor company Diageo (known a.o. for Johnnie Walker, Smirnoff, and Guinness) uses VR to confront people with the effects of driving under the influence.

The same approach could be use for pedestrians and cyclists, not just car drivers.

Obviously, VR can also be used to train people how to behave in various emergencies, which could make for much more enjoyable fire drills. Enter AR and you can interactively guide panicking crowds to safety.

Automotive AR and VR are Just Getting Started

In this article, we showed examples of how augmented and virtual reality are changing the automotive industry. We identified use cases throughout the entire development process, all the way to driverless vehicle occupants.

It is clear that the complex process of designing a car is becoming more and more intertwined with software development. AR and VR are already a significant factor and they will increasingly be used. It’s up to creative minds to further reshape the driving experience.

In the broadest sense, a virtual showroom is more or less any VR environment created to showcase products. These are usually products that aren’t easy to display in detail otherwise, such as cars or real estate.

Yes and no. While creating a simple virtual showroom shouldn’t be too difficult or expensive, more demanding applications of virtual reality technology, including some outlined in the article, are out of reach of small companies.

They can learn the basics, and they can also be confronted with numerous scenarios that would be impractical or impossible to reproduce in real life. There is no substitute for getting behind the wheel, hence VR should be viewed as a learning aid, not a substitute for driving.

Fidelity remains a concern at this stage of development. While it’s safe to assume car makers will create immersive, high-quality VR experiences, small businesses such as used-car dealerships might not get the same results, at least not today.

Daan Terra, Netherlands

Daan is an enthusiastic and helpful person. He uses software design principles and object-oriented thinking in technical domains. He’s experienced in the development of large and complex products. His strongest knowledge and experience is in Java, Android and C#.NET. Daan has worked in interdisciplinary teams as a team lead. Projects with international partners or customers resulted in successful and timely delivery of new software components.