Does Your Test Partner Need to Be a Telco Expert?

End-to-end testing—it’s all the rage right now in any number of industries, and with good reason. As global technologies become more connected and more thoroughly interwoven into the fabric of society, more thorough and efficient testing will become not just a luxury, but a necessity.

As such, it should come as no surprise that a number of testing automation providers have cropped up in the past few years, covering everything from software security to telecoms.

This, too, makes sense on the surface: how different could testing solutions for different industries really be? If you can automate verification for a new social media platform, why can’t you do the same for VoIP verification?

Unfortunately, in reality it’s not that simple. Testing from one domain to another involves different sets of challenges and radically different domain knowledge. Even if the basic principals are the same, the particulars are going to vary enough that you’ll really want to partner with someone who is more than just an end-to-end test factory.

If you’re a telco service provider or network operator, you’ll want to seek out a testing provider with specific, in-depth domain knowledge. Why? We’re glad you asked…

Unique Challenges in Telecom Testing

As the global telecom sphere evolves, it’s not just service itself that becomes more complicated—it’s testing and service verification as well. Every year, user expectations are becoming more exacting and networks themselves are getting more complicated.

Even as 5G slowly becomes more and more available, legacy telephony continues to be a major part of the landscape, meaning that in order to verify service for a given device, network, function, etc. testers need to run through more individual use cases than ever.

Of course, this increased complexity makes the use cases themselves more complicated from a testing perspective, meaning that they’re becoming more time consuming at the same time that they’re proliferating.

Given all the above, it’s no wonder companies are seeking outsourced testing solutions. At the same time, the very factors that are pushing people to automate tests are also a clear signal of the importance of an automation provider with strong domain knowledge.

Let’s say you’re testing VoLTE after a recent rollout of updates in your core network. A generalized testing provider might use simulation to verify VoLTE functionality, but they’re unlikely to have the actual out-of-the-box iOS and Android devices that your users will be utilizing. 

As a result, small differences in the simulation vs. reality might lead you to miss small changes in latency times (which can result in a big difference in perceived quality of service for VoLTE users); by the same token, they might not test interworking towards legacy devices and networks. Thus, when you take the updates live you might find that some of your users on older devices aren’t able to access your network, while VoLTE users aren’t getting the voice quality they expect.

Creating Project Alignment

The implications here are two-fold: first, that if you don’t have robust testing framework already in place, you’ll necessarily rely on your vendor to provide such a framework. If your partner in this process can provide automation but isn’t able to develop a domain-specific set of use cases, the necessary knowledge to reach minimum acceptable test coverage might prove difficult to come by.

Second, if you’re in a position where your in-house testing expertise has largely vanished, there is fairly obvious value to be gained by partnering with someone who can provide a repeatable framework that yields sufficient documentation for you to regain some control over your service verification.

For this to be effective, telco operators need a company that can speak their language. If your testing solution provider doesn’t have telco specific expertise, they won’t have the language with which to communicate effectively before or after testing about your particular concerns and areas of interest.

Beyond End-to-End Testing

So far, we’ve seen a number of ways in which telecom expertise can make testing more comprehensive and thus more valuable in the short and long term.

But is it just about the ability to manage ever-growing complexity and to do so in a way that allows easy collaboration and transparency—or is there another level of testing sophistication that can only be powered by those who have a complete view of the entire telco structure, from high-level strategic and corporate concerns to specific engineering challenges?

As you might have guessed, the answer is “yes.” Sure, a generic testing automation provider might be able to adequately offer end-point service verification (i.e. end-to-end), but what about the level beyond that?

With real telco expertise, it’s possible to automate not just verification at end-points, but between them, i.e. what’s happening at the level of network elements, switches, packets, and signals. Even in cases where call, text, or email appears to have gone through as normal, there may (for instance) be signaling errors that a “normal” testing solution would have no way to catch.

You might be thinking that there isn’t too much added value to be extracted here, but in point of fact these sorts of “invisible” errors can be harbingers of issues that could crop up during later usage.

By tracking them and offering reporting that documents them, you improve test coverage well beyond what would have been possible under other circumstances. By increasing coverage in this way, you potentially improve the quality of your network in a way that would be functionally impossible without a high degree of telco expertise.