A few years ago, a tester working on a typical telco project could run through about 10 use cases per day. Now, that number is closer to 8 use cases per day. This trend might be worrying from the outside, but if you’re a test engineer within the world of telecommunications it really shouldn’t be shocking.
After all, as the complexity of global networks skyrockets, it stands to reason that verifying service for any particular node or function would become incrementally more complex as a result. The question is: what can network operators do about it? How can you maintain standards and achieve a positive testing ROI in these increasingly difficult environments? The first step is to identify the sources of those difficulties in order to better understand—and thus more effectively tackle—the challenges facing network testers. To that end, here are the top 5 reasons that modern network testing is becoming more complex by the day.
1. Shorter Lead Times
This first bullet isn’t something that’s inherent to telco operators per se, but it provides a useful baseline for framing future challenges. Because the cycle of updates for modern networks is speeding up rapidly, the lead times for any given network verification flow are getting shorter and shorter.
This is due to a range of factors, from customer pressure to the rate of technological change across the industry, but regardless of cause these shortened lead times aren’t going away any time soon. No doubt, your own system updates are being implemented more rapidly than in the past, meaning that testers have very little room to maneuver.
This adds complexity in the form of increased time pressure, i.e. testers have to meet any challenges that come their way without taking any additional time to do so.
2. Device and Operating System Diversity
In some ways, this acts as a corollary to our first bullet. If you’re operating a network that primarily caters to mobile phone users, you’re also tied to the schedules kept by companies like Apple and Samsung. As they put out updates to their software every X number of weeks, you have to make sure that those updates aren’t affecting your ability to provide service.
On top of that, the hardware is being updated nearly as rapidly, with new products coming onto the market every day. Not only does this fact increase the time pressure your testers are under, it also simultaneously introduces a wider array of test cases based on which iterations of which devices (potentially including legacy devices) your customers are using.
3. More Protocols and Protocol Layers
In order to ensure that each packet is delivered to the correct node in the correct condition, your service needs to be able to integrate with any number of different network protocols. In this past, this might have meant mostly wireless-specific protocols like LTE or Bluetooth.
But as smartphones continue to dominate the landscape and the intricacies of internet protocol layers deepen, you’re more likely to be testing TCP, UDP, HTTP, FTP ARP, ICMP, etc. These protocols interact with one another in complex ways, with differing levels of abstraction leading to a multiplicity of different possible configurations for the processing of any given packet switch.
Not only does this require more individual test cases to ensure high network quality, it also requires increased coordination between network operations centers (NOCs) and those who are actually utilizing the physical testing equipment. Why? Because any issues that crop up in this kind of conformance testing can potentially affect not just the protocol layer itself, but transport or compatibility layers as well.
4. Exacting Customer Standards
Of course, just as network testing itself is growing to involve more and more test cases, expectations for network quality are getting loftier and loftier. By and large, this is being driven by customers who increasingly rely on their smartphones for any number of tasks, from navigating in their cars to responding to work emails to watching Netflix.
Years ago, the users in your network might have been more forgiving about things like slow load times, brief service outages, or any other small instances of less-than-stellar network quality—as long as it was high quality most of the time, end-users could make do.
Now, a tiny dip in service quality could have serious ramifications—potentially leading disaffected users to turn elsewhere for coverage. Like the shortened lead times we talked about above, this doesn’t exactly add complexity in and of itself, but it does reduce your average tester’s margin of error radically.
At the same time that you’re forced to test more configurations more quickly to verify your network’s functionality, you now need to do so with an extremely high level of precision—one that might quickly require some outside-the-box thinking to make possible.
5. Security Concerns
You could almost think of this challenge as a subset of the previous one, but it bears calling out on its own: network security is becoming a big watchword in the industry—and with good reason. Users aren’t just expecting your network to stream videos without any lag, they’re also frequently using services like Venmo and Apple Pay to transfer money wirelessly.
This means that in addition to ascertaining whether your network is offering all of the functionality across all of the different nodes that connect to it, you also need to be sure that your network isn’t susceptible to cyberthreats or malicious traffic.
Of course, this is far from a new concern, but it becomes more pressing when a user’s smartphone is so integral to her finances. It’s not insurmountable, but it’s one more thing to worry about as you run through your tests.
6. Loss of Domain Knowledge
None of the challenges we’ve described so far are brand new from the perspective of your average telco operator. So why does it seem like testing complexity is mounting at an unprecedented rate?
In part, this may be due to the fact that for many network providers it’s become common practice to outsource things like network testing. Thus, rather than developing and retaining a lot of domain knowledge in-house, your ability to test in a smart, efficient, reliable, and repeatable way erodes over time.
As you become more dependent on third parties for your network testing, the prospect of retaking control over such increasingly complex processes becomes more and more daunting. To counteract this, test engineers need to find some kind of middle ground—something that will help them build up whatever domain knowledge has been lost over the years without forcing them to face new complexities with insufficient resources.
Like we said above, the first step to combatting complexity is to better understand it, and that starts with your ability to gather, retain, and analyze data on configurations, edge cases, protocol changes, and everything else that impacts your network functionality.
Learn More About Test Automation
Click below to see how SEGRON’s ATF can help you automate testing.
