How a major automotive OEM used automated Android Auto ‘stress testing’ to resolve connection issues in the field.
Android Auto is the mobile screen projection technology developed by Google to mirror features and apps from a connected Android device to the in-car infotainment head unit.
A leading automotive OEM received customer feedback on its Android Auto equipped infotainment system indicating users are experiencing problems when connecting their Android mobile device and synchronising Android Auto functions with the head unit. Detailed consumer feedback implies that the infotainment system does not always detect the phone connected to the Android Auto enabled USB port in order to instigate the Android Auto function on the infotainment system.
Additionally a further consumer field issue reported is that during an Android Auto session an outgoing call initiated from the car does not establish the uplink audio, meaning the called party does not hear any audio from the caller when answering the call.
The OEM used Nextgen ATAM automation to investigate the user feedback, deploying stress testing to explore robustness of Android Auto connections and the performance of key functions, such as calling and audio routing. ATAM is capable of performing automated testing of Android Auto USB connections by connecting and disconnecting multiple connected Android mobile devices. The ATAM test setup is also configured to monitor audio quality and routing, audio levels, distortion and also audio stream dropouts down to a resolution of <10ms while in the call.
Any unexpected issues which occur during testing will be captured in detail. ATAM has the capability of collecting and collating a variety of logs which detail the exact circumstances of the failure. In this example ATAM logs USB protocol connections and voltages, audio system and mobile device HCI logs, network connectivity, Bluetooth and Wi-Fi activity and so creates a fully logged record of any failures for debugging. This provides an 'instant replay' function for test failures and allows the exact failure conditions to be reproduced for investigation and root cause analysis.
The test set up consists of the infotainment head unit and a selection of key Android mobile phones connected to the USB connection control Interface on ATAM, which becomes a 'middle man' with the USB connection then terminates on the infotainment system's USB port.
A diagram of the test setup is show below. The infotainment connects to the USB Interface on the ATAM System Controller. The primary Android mobile phone is designated as Device 1 connected to the USB device Interface on the ATAM System Controller. For the call audio checks ATAM is directly connected to the infotainment system’s microphone array and audio / speaker output. A secondary mobile phone is also connected as a slave mobile device to represent a target for calls from the infotainment system and to make incoming call functions. ATAM's phone control interfaces provide full automation control of normal user functions performed on the mobile phone, such as pairing, and making and receiving calls.
ATAM automatically connects and disconnects the phone connection with the infotainment system USB port, also measuring the current and voltage drawn by the phone over USB.
During a phone call ATAM monitors the audio levels for the downlink and uplink call audio and measures audio quality, dropouts and reports if any echo is detected. Multiple phones will be connected over the course of testing and the functions of Android Auto on the infotainment system are controlled and monitored using an ATAM custom control interface.
Nextgen also help to define a variety of key mobile phones drawn from Nextgen’s extensive global library of devices to fully replicate end user devices and usage. Connections to the infotainment system are made using USB and Bluetooth.
During the stress testing, ATAM confirmed the field reports finding that specific devices were not always detected upon USB connection. In a subset of cases, it was also found that audio was not always correctly routed to the infotainment head unit.
Using ATAM automation multiple phones were rapidly tested for USB and Bluetooth connection, disconnection and calling functions. The test cycles were repeated 1000’s of times, and intermittent failure conditions logged and identified. The test was able to verify and reproduce reported issues across a range of mobile devices and scenarios. The frequency of the issues was found to be below >7% - low enough to be missed in development test process but high enough to account for the level of field issues reported by end users. Additionally some further device specific failures were found and logged which had not previously been reported.
During the test process ATAM collects multiple logs when any unexpected issues are detected. The precise test failure scenarios along with detailed phone, Bluetooth and USB connection logs are collated and filed for debugging and rectification.
Nextgen ATAM automation stress testing reproduced the issues reported by customers and identified specific devices that were failing for the problems observed. The precise conditions and scenarios required to reproduce the failures were central to locating the root cause of issues based on the collected reports and logs and were able to provide recommendation and a route to fix in place.