What Is the Future of IoT Testing?

automation-framework

Lots of articles have argued that the first ever internet of things (IoT) device was a soda machine at Carnegie Mellon that was connected to the early internet in the 1980s. The machine was famously automated to let users on the local network see how recently the machine had been filled (and thus how cold the soda was).

It’s a great story—but one device does not an internet make. In other words, what we in the modern era understand as the IoT isn’t really about individual devices; it’s about a tremendous volume of different devices all working together in tandem.

This is an important distinction for telecom testers, because it gives a clearer sense of what challenges might actually arise as hordes of IoT devices move beyond industrial settings and into homes, cities, transportation networks, and elsewhere.

Testing conformance for individual devices will be critical for network operators, but it will only be one part of an equation that includes load testing for large networks of devices, conformance testing for different layers of applications and sensors, plus dealing with emergency use cases.

As a result, telco testers will soon find themselves in need of new strategies—and potentially new technologies—for ensuring high quality of service (QoS) for users.

Conformance Challenges

One of the most obvious challenges that presents itself in the current IoT landscape from a testing perspective is that the architecture of most IoT systems is a complex hash of heterogeneous technologies. IoT devices tend to work with a number of layers:

  • Sensor
  • Application
  • Network
  • Backend

And there’s no guarantee that those layers will be designed to play nicely with one another. For instance, you could easily have industrial IoT (IIoT) deployments in which the devices, applications, and middleware are all coming from different vendors, who might not be operating with the same technical specifications in mind.

In this case, you might find that each element in the IoT environment was working correctly, but the entire system wasn’t transferring production data back to the control tower effectively.

The implication here for testers is that, more so than in many other use cases, testing the entire process end-to-end is crucial. Without doing so, you risk the possibility that there are conformance failures between layers that will prevent the devices from networking together successfully in the field.

Once you’ve adopted an end-to-end mindset for IoT testing, you need to make sure that your test cases reflect it. So, on top of the standard set of use cases for the devices (validating network connectivity, verifying data packets/measuring packet loss, and validating synchronization, transmission frequency, multiple-request handling, etc.).

You’ll need to recreate the entire set of systems and subsystems to make sure that the whole end-to-end flow is actually functioning within the confines of your network. Needless to say, this can become extremely resources-intensive, especially if you’re trying to test all of the use cases by hand.

New Network Expectations

Sure, one of the biggest challenges that comes with IoT testing is the relative complexity, but that complexity is just one piece of the puzzle. An equally important piece of that puzzle is the increase in load that networks running IoT devices are likely to face.

Though most IoT devices will operate on low power and low throughput, there will be tons of them trying to connect to your network in order to constantly transmit information. As of about a year ago, estimates put the number of IoT devices in operation around the globe around 7 billion.

And that number is only going to keep rising as this nascent technology grows. If the hype around 5G is to be believed, we could see exponential growth in this number within a few short years.

To truly ensure that their networks are ready for full IoT deployments, testers will have to find a way to replicate this peculiar type and volume of network traffic within their test environments.

This will begin with larger scale deployment of test automation (such that the relevant devices can be controlled through an agile interface), and will likely involve the creation of complex testing environments that differ radically from what telco testers are used to.

Devices will have to be able to connect to your network successfully and continue transmitting information without getting booted off accidentally, and you’ll have to ensure that your network is allocating bandwidth appropriately under these new expectations and constraints.

Going Beyond End-to-end

Because of the high degree of complexity involved in testing IoT deployments, testers are also in somewhat of a tricky position when it comes to uncovering the root causes of any bugs that they encounter. For this reason, it’s also potentially worth moving beyond end-to-end workflows in your testing.

This could mean utilizing signalling analysis and CDR data from various network elements to ensure that there are no tuning issues, for instance. By moving beyond the level of success/failure and into what’s actually happening at the signalling level, you can gain a more comprehensive view of how your network is interacting with these devices.

The result of this more comprehensive view? The ability to detect potential QoS risks before they’ve actually caused a device to disconnect or a packet to get lost.

As the IoT becomes more entrenched in the technology and telco landscapes, with think that the future of testing for this area will revolve around attempts to solve the challenges we’ve been laying out above.

The trick, of course, will be not just to solve them—but to solve them efficiently at scale. To that end, testers are going to have to think long and hard about what kind of testing technology they want to employ going forward, as well as how they want to structure their test operations.

Are there ways that complex test cases can be streamlined and sped up to improve throughput? Can regression tests be made less time intensive and therefore more feasible for IoT use cases? Only time and test labs will tell.

Social Media

Share on facebook
Share on twitter
Share on linkedin
Share on xing

Follow Us

Facebook

By loading the post, you agree to Facebook's privacy policy.
Learn more

Load post

YouTube

YouTube

By loading the video, you agree to YouTube's privacy policy.
Learn more

Load video

Subscribe To Our Weekly Newsletter

No spam, notifications only about new products, updates.
Segron_ATF1-Banner-480p

Categories

Interested in our Products ?

Scroll Up

TECHNICAL PROJECT MANAGER

You will be a part of the SEGRON Project Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation.

As technical project manager, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases and you have to setup and align related project activities together with our customers.
You should have a strong knowledge and background in the telco environment paired with project management skills to align projects, track the related KPI’s further having a strong view also on the financial performance.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible
Team player:
• collaborates with others
• teach new skills to others and write tutorials
Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Several years of experience in Telecom Domain.
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• IMS/SDM/EPC domain knowledge
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack.
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge.

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus.
• Robot Framework / Python

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• PM Certification or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2500 EUR (depends on experience)

DEVELOPER TEST AUTOMATION

You will be a part of the SEGRON Test Automation Development Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation using Python, Robot.

As test automation developer, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases, building automation tools, framework, creating reusable python libraries for test automation.
You should have a strong knowledge and hands-on experience in test automation framework (Robot) and Python scripting language. Further you must continuously renew the automation & review guidelines set within the team. Ensure developed reusable keywords & test scripts meet the automation guideline.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible 


Team player:
• collaborates with others
• teach new skills to others and write tutorials


Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Experience in developing reusable test automation scripts using Python and Robot Framework
• Several years of experience in Telecom Domain
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• Test Case development
• Mobile development
• Design Patterns
• Web development/testing
• IMS/SDM/EPC domain knowledge
• Hand-on experience in Docker containers and Kubernetes.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2000 EUR (depends on experience)