Case Study: Delivering Bluetooth Earbud Connectivity Performance

What happened when one of the world’s leading smartphone manufacturers used Nextgen’s automated testing platform (ATAM) to maintain brand equity and customer satisfaction?

“Thanks to our ATAM testing programme, we conducted complicated tests exponentially quicker than any manual testing regime – and with 100% reliability. This smartphone went to market with the manufacturer fully confident in its Bluetooth performance.” 
Tim Golding, Senior Automated Systems Engineer, Nextgen

Background

A global reputation for excellence. Billions of dollars in worldwide revenue. And a leading position in all major international markets. When a major smartphone manufacturer made the decision to remove the physical audio port from its flagship phones, the stakes couldn’t have been higher.

The manufacturer had always appreciated the risks involved in any significant change in its hardware or software. But with a change of this scale, there was no room for error. Connectivity became critical. Only products that had gone through a world-class testing programme could be trusted to go to market.

Challenge

With no audio port on its newest phones, this smartphone manufacturer chose the obvious route forward. It designed and manufactured a new pair of wireless Bluetooth earbuds. They were to be included with each of its latest phones.

But that meant the manufacturer could not take any chances. Its reputation was on the line, with millions of customers around the world set to rely on strong connectivity between the earbuds and the new phone. The manufacturer had to have complete faith in the audio quality and connection stability between the devices, as well as the overall user experience.

What we did

Using our market-leading and patented automation platform Nextgen ATAM (Automated Test Application Manager), we devised a test to check both the ongoing connection and the audio quality between the smartphone and the new Bluetooth earbuds. We also included a test for Bluetooth Linkloss.

The test had a relatively simple set up. We connected the phone to the ATAM System Controller for phone control.

Next, we used ATAM automation to pair the new earbuds with the phone. We then connected two ATAM audio monitoring channels to the audio outputs of the left and right earbuds respectively. This enabled us to accurately monitor the quality of audio received in each earbud.

The earbuds were positioned in a dummy head. The dummy head was placed on a motorised turntable controlled by ATAM automation to create spatial movements during tests.

Automated connection testing

Nextgen ATAM is designed to replicate extended periods of connection and usage. Integration with Bluetooth protocol analysers provides the insight needed to identify the root causes of any potential Bluetooth communication problems. Combined, these two factors made Nextgen ATAM ideal for this testing scenario.

We streamed continuous Bluetooth audio for several hours, with the earbuds in the dummy head being constantly repositioned at different angles relative to the mobile phone. The test was designed to detect any gaps in the audio. Our test confirmed that the earbuds remained connected to the phone – with no dropouts – over a prolonged period of time.

Automated audio testing

Without changing the existing test script or set up, we could also assess audio quality across a range of file types and codecs. Nextgen ATAM is able to detect gaps of just 10 milliseconds. It can also check for harmonic and frequency distortion of a known tone, registering a failure if distortion exceeds defined parameters.

Our testing confirmed that the audio quality broadcast by the earbuds was excellent, regardless of the file type or codec being transmitted from the smartphone.

Automated Bluetooth Linkloss testing

By adding a programmable attenuator controlled by Nextgen ATAM, we simulated what can be a common – and problematic – real-world situation: a user walking out of the phone’s range while wearing the headphones, then returning back into the phone’s range.

This test gave us two clear insights. It enabled us to determine the precise point at which audio dropouts occurred and when the connection dropped. It also allowed us to understand how successful the devices’ auto-reconnection capabilities were when the user returned back into range.

Results

The automated testing programme conducted with the Nextgen ATAM platform gave the smartphone manufacturer all the reassurance it needed. Its earbuds provided the audio quality that would be expected of a premium product. The Bluetooth connection was strong, stable, reliable and consistent.

“Using Nextgen ATAM means QA test teams don’t need programming skills to create tests. Anyone can create sophisticated test programmes with relative ease – and absolute confidence in the results they reveal.”
James Scales, Director, Global Engineering Operations, Nextgen

This graph, for example, illustrates the audio detected from the left and right earbuds. Any errors would have delivered drops or spikes in the audio levels illustrated by the orange and yellow lines.

With no coding needed, the Nextgen ATAM test programme meant the smartphone manufacturer was able to easily check a range of spatial, physical and electronic parameters to quickly find and verify any issues ahead of its global product launch.

Tests were extensive, wide-ranging and quick to set up. They ran across weekends and overnight, significantly increasing the number of tests performed before the designated deadline. This provided a previously unachievable level of robustness to the tests.

Using Nextgen ATAM also meant each test was tightly defined. So when new software levels were introduced, a repeat test could be performed with 100% certainty. This ensured that any issue uncovered in the tests was reproducible and so definitely rectified, saving the manufacturer from the potentially huge headache of an in-market product generating consistently bad reviews.

To find out more about Nextgen's automated testing services, get in touch to speak to one of our experts today.