Revealed: The Common Bluetooth and Wi-Fi Interoperability Issues Missed by Manual Testing

Manual interoperability testing has plenty going for it. It is quick, fast to roll out and perfect for ‘smoke’ tests designed to establish if previous issues appear to have been resolved. It also gives manufacturers an understanding of the user experience of their product. Through manual testing, engineers and QA staff can put themselves in their customers’ shoes. 

But there are also significant limitations.

Manual testing can only go so far before it is simply not feasible to test every use case and permutation. In a world where billions of devices are manufactured each year, and the complexity of use cases is rocketing, relying on manual testing alone is a strategy that almost certainly guarantees missing issues.

Companies adopting a manual-only approach risk being vulnerable to a range of potential problems. Why? Because as well as being unable to scale to ensure products are released on time, manual testing is also limited by a human’s ability to identify what exactly is happening and, if bugs occur, where the issues lie and what precise scenarios are required to reproduce them.

And because of human limitations, often the same types of issues go unnoticed during manual testing. This is where deeper automated testing can provide the extra insights needed to ensure successful products and devices that are as free as possible from bugs.

Here are three of the issues that might be seen, but can be hard to find root cause unless automation is deployed.

Coexistence Issues

Bluetooth and Wi-Fi coexistence is handled automatically in most modern products. Yet challenges can still occur, especially when an antenna is shared between Bluetooth and Wi-Fi.

Difficulties can also arise if transmitters and receivers are in close proximity. If a transmitter inadvertently saturates a receiver, gaps in transmissions can become problematic. This can even happen across different frequencies, if the transmit power is high enough.

Modern devices are equipped to handle these issues by prioritising Bluetooth or Wi-Fi over the other and permitting small ‘acceptable’ losses of either. But this is a complicated task for a product to manage. Real-world scenarios, such as Wi-Fi already being heavily loaded with content streaming, can affect Bluetooth discovery capabilities.

Manually testing all the different permutations is beyond the reach of any realistic human capacity. So automation is the quickest, most reliable and most effective way to be confident in your product’s coexistence handling.

Hidden Data Retransmissions

Using data transmission protocols to retransmit data until it arrives successfully is an incredibly useful aspect of modern device connectivity. But manual testing can be ‘caught out’ by these data retransmissions, leaving engineers to record test passes in marginal conditions while unaware that the exact same use case could unexpectedly fail in the real world.

External interference is the biggest issue causing data retransmissions. But with consequences such as very short audio dropouts often unnoticeable, it can be easy to miss.

Automated interoperability testing provides a more detailed level of insight. An automated test will detect marginal conditions and highlight where successive retransmissions are occurring. This gives product developers advance warning of potential issues before the product goes to market - and facilitates resolutions that can avoid potentially costly reputational impacts.

How Bugs Manifest

Manual testing is generally only a reliable way of finding issues or failures that occur all or most of the time. Because of the limitations of manual testing and how labour intensive conducting each test is, only a few repetitions of tests are normally completed.  

But that means failures that occur relatively infrequently, yet are nonetheless still problematic in a real-world use, may be overlooked. Automated interoperability testing provides the test case repetition needed to generate necessary insights into lower failure rate issues. It can also find bugs using complex test scenarios where several variables are at play.

The illustrations below show the value of increasing the number of repetitions involved in each test. While relatively rare, external factors can impact user experiences significantly. In this 500-cycle test, 98% of connections were established in fewer than five seconds. Yet one connection took 13.5 seconds. This would have almost certainly been missed with a limited number of manual tests.

Making the Most of Automated Interoperability Testing

The value of Bluetooth and Wi-Fi automated interoperability testing is unquestionable. By extending testing scenarios significantly beyond human capability, it generates results and insights that manual testing simply cannot achieve.

Overcoming coexistence issues, managing hidden data retransmissions and understanding how bugs manifest are all key aspects of successfully bringing a new product to market. And automation is the only way to ensure all your bases are covered.

To speak to a Nextgen expert about how you can create better connected products with automated interoperability testing, contact us today.