Are Your Device Tests Ready for the IoT?

According to a recent GSMA study, the IoT market will be worth $1.1 trillion and include about 25 billion IoT connections by 2025. The majority of those connections will be in the industrial and vertical industry segments (13.8 billion connections) and the smart home market (11.4 billion).

The IoT will undoubtedly result in new revenue opportunities for telcos, particularly for companies that move beyond conventional connectivity providers. New business models may emerge in which telcos assume the role of end-to-end solution providers.

But regardless of how business models evolve, connected devices will always remain the foundation of the IoT. Effective device testing will be critical for any telco to succeed in the IoT era.

Challenges of Device Testing in the IoT Era

Scale: The sheer number of IoT-enabled devices has already exploded, and experts predict that the number will only increase in the coming years. The list of devices that require testing will grow far beyond iOS, Android, and a limited number of IoT devices. 

Profitability: In the coming years, the average revenue per connected device is projected to continue falling.

Productivity: A few years ago, human device testers could complete about ten test cases per day. Thanks to a dramatic increase in testing requirements, testers can now complete about six cases per day. That number will probably continue to drop as the network landscape gets more complex.

Narrowing performance margins: As acceptable latency margins narrow, it is increasingly challenging to detect performance outside the range.

Connectivity: Instant access to real-time data is a primary benefit of the IoT.

Varied requirements: Each network of IoT-enabled devices is essentially its own closed system, with potentially unique requirements for device density, latency, and data rates. Furthermore, each of these systems can present unique operational constraints.

Legacy system compatibility: It’s not uncommon for IoT devices to connect with older devices, which can introduce performance issues. 

Strategies for IoT-Ready Device Testing

Implement an IoT testing framework.

An IoT device can’t work on its own. It uses a network, services, and applications to function. Each of these includes an additional layer to the actual device–and must also be tested. An IoT testing framework accounts for all these layers and the tests they require:

• Devices (the “things”): Functional and security testing.
• Network: Connectivity and compatibility testing.
• Services: Interoperability, functional, and API testing.
• Application: Functional, compatibility, API, localization, and user experience testing.

The IoT testing framework should also incorporate testing for the entire IoT system (performance, interoperability, compliance, data privacy, etc.)

Incorporate automated testing.

The IoT has placed new demands on-device testing. First and foremost, the average number of test cases that a human tester can complete per day is steadily declining thanks to increased network complexities. Meanwhile, the number of devices and a variety of use cases continues to grow, a trend that will likely accelerate in the coming years.

Also, devices must now deliver constant connectivity not only with their network but also with other IoT devices. This complexity also requires constant testing to ensure performance.

Automation can address all of these challenges, maximizing test coverage and dramatically reducing the volume of tests that must be done by humans. Automated device testing also allows telcos to continuously optimize underlying code, ensure ongoing integration, and detect much smaller performance variances than humans can generally identify.

Conduct load and performance testing synchronously.

Telcos once only had to worry about testing iOS and Android phones, along with a shortlist of other devices. Nowadays, one mobile device might be synced with any number of IoT devices, from wearable fitness trackers to the home thermostat.

This relationship requires a novel approach to device testing. Rather than testing load and performance separately (and usually sequentially), the IoT-ready device testing protocol includes concurrent load and performance testing devices and their applications.

Take advantage of service virtualization.

Already prevalent in the software testing world, service virtualization is the creation of a virtual service environment that mimics the real one. Since IoT devices all run on applications, service virtualization has found its way into device testing. It allows DevOps teams to run tests without impacting the active production environment.

Reduced time-to-market is a key benefit of service virtualization because tests can be conducted in the development phase. Potential defects and interoperability issues can be detected earlier. Following deployment, service virtualization can support continuous testing against conditions that are present in the real environment.

Use Agile methodology for load testing.

The Agile methodology of software development is grounded in early delivery and continuous improvement cycles. The purpose of the approach is to create high-quality software while limiting costs and decreasing development time. Like service virtualization, Agile methodology has made its way from the software world to the device testing bench.

By using Agile methodology for load testing, telcos can ensure functionality during the development phase, instead of post-production. This approach reduces the cost and time required for testing, critical when testing protocols are becoming more and more complex.