The car and the spaceship blast off

Accelerate, turn and brake – just like the car does. But experience it in a virtual world, rather than just as a passenger in the back seat of the car. This is how the new immersive experience from Audi and holoride uses motion data for entertainment. But how does the game on the VR headset know where and how fast the car is driving?

Thanks to location data, motion data and a connection to a mobile network, vehicles today know exactly where they are. This is primarily for safety, such as for features like the EU's automatic emergency eCall, which has been used in newly registered cars for many years.

But developers can also use this data to take in-vehicle entertainment to a whole new level. With VR headsets, passengers in the back seat can now immerse themselves in a 3D world in which they move exactly like the vehicle on the road. In a spaceship through space, for example.

Audi has just launched this gaming experience with its partner holoride for production vehicles. CARIAD is deeply involved in the development.

This is because the necessary movement data that a car can potentially make available for gaming content must first be collected and then forwarded together, almost in real time. After all, there can’t be any lag for passengers between the movements of the car and their spaceships in the virtual world.

He works closely with partners at Audi and holoride to make the data usable as a digital experience in the back seat. Gaming developers from studios such as Schell Games from the holoride original ‘Cloudbreakers: Leaving Haven’ can then transfer them to the virtual worlds and tailor game content to them. Hoffmann has to control various areas in the vehicle in which the sensors that are important to him are installed: 

• Motion and gyro sensors that measure acceleration and direction of movement
• GPS data that determines the position of the vehicle by satellite
• Route guidance from the navigation system

These sensors sit in different control units in the vehicle and their data packets have to be routed from there to a common gateway. This is a central data node from which the data packets flow to the communication control unit and can then be sent out of the vehicle. Hoffmann uses the Bluetooth Low Energy Standard for this purpose. Each data packet is specially encrypted.

Via this Bluetooth interface, the vehicle data is finally transferred to the VR headset and holoride. Thanks to the holoride Elastic SDK (Software Development Kit), game developers can use the data and integrate it into their 3D worlds on the VR headset. There, all live data from the vehicle is merged with the gaming content.

The movement data from the vehicle has come a long way when it finally flows playfully into the virtual worlds on the VR headset. From the complex navigation through the vehicle architecture, a scalable entertainment solution for Extended Reality (XR) in the vehicle is created. And CARIAD is already working on making the function available for its unified architecture of the future.