How AI powers the digital in-car experience

Modern vehicles offer customers a completely new digital in-car experience. The digital assistant plays a central role here, transforming the car into a personal companion. “The digital assistant understands and speaks, learns routines and proactively makes suggestions”, says Kim Cedziwoda, Head of Digital Assistant at CARIAD. “This is driven by multiple artificial intelligence modules and enables a new symbiosis between user and vehicle.”

Exact speaker localization

In Audi models with the new E³ 1.2 electronics architecture, which celebrates its premiere in the all-new Audi Q6 e-tron, a self-learning digital assistant is launching: the Audi assistant, which has been co-developed with CARIAD. It is deeply integrated into the infotainment system and offers to support with more than 800 voice commands. Natural language understanding allows to control vehicle functions, make calls, configure the travel route, play music, answer questions, get information on weather or simply chat. And all this is possible without taking the eyes off the road.

Deep-learning models use microphone streams to locate who is talking to the assistant. This information is then considered when executing commands. For example, if the front passenger asks to open the window, the front passenger window is opened. At the same time, certain functions can only be activated by the driver for safety reasons.

More personalization

The digital assistant also learns the user’s behaviour. It uses context information and identifies patterns in the driver's interaction with the car such as recurring operating sequences. Based on machine learning algorithms, it distils them into routines, which it then proactively suggests the next time a similar situation arises. For instance, if a driver activates the air circulation on their daily commute, the smart system will ask if it should activate the air circulation on this route every time. Other examples are predictive call lists, massage routines, climatization and much more, if the customer gives approval.

Generative AI for natural dialogs

“The goal with a voice-based assistant is to make dialogs appear as natural as possible and to cover a wide spectrum of topics and information”, says Cedziwoda. “To achieve that, the system uses latest natural language processing and accesses a multitude of data sources such as vehicle data and input from online sources.”

One new source is ChatGPT, an AI-based chatbot. ChatGPT is made available through Azure OpenAI Service. Foundation of this feature for Audi customers is the Chat Pro solution from Cerence Inc., which also offers an automotive-grade ChatGPT integration. The integration of ChatGPT into the automotive landscape is one of the first productive customer applications of large language models (LLM). LLMs are trained with large amounts of text data from the Internet, books or forums. These text components are used to calculate probabilities of word sequences with incredible accuracy. This enhances the human-machine interaction in terms of naturality and breadth of knowledge. For example, ChatGPT can provide information on tourist attractions, report on past football tournaments or help solve math problems. In the future, it will also be possible to ask questions about the vehicle, for example how to set the tire pressure.

Seamless user experience

All smart functions of the Audi assistant are integrated seamlessly, ensuring drivers will experience a smooth and personalized interaction with their car. If addressed, the Audi assistant automatically recognizes who is speaking and whether to execute a vehicle function, search for a destination, or, for example, provide the weather forecast. It only forwards queries to ChatGPT when it cannot answer general knowledge questions. Data security has the highest priority; to ensure optimal privacy, all questions and answers are deleted after processing. Moreover, ChatGPT never has access to vehicle data.