For the best possible results, your test lab needs to make use of the absolute latest in cutting edge technology. Because this is the arena in which you test changes and additions to your network—including new connected devices like the internet of things (IoT)—you need to find a way to replicate existing network conditions as closely as possible. Otherwise, you might test out new service offerings or network adjustments in a lab setting only to find that your service doesn’t work correctly under real life conditions—which could result in costly delays, outages, and potential subscriber churn.As the technological rate of change in the telco domain increases—and your mobile subscribers increasingly adopt new, 5G-enabled phones—equipping your test lab with the latest technology will only become more important. And yet, the rapid pace of change often makes this feel like a losing battle. In that way, it’s just as important to set your lab up for future success as to meet current technological needs. How do you do that, exactly? Read on to find out!
1. Automate Flagship Devices
We alluded to this briefly above, but one of the most important requirements for a successful test lab environment is that your testers be equipped with the latest end-user devices. This might seem like it’s as simple as going down to Best Buy or the local equivalent and picking up the latest 5G device—but in point of fact it often requires much more than simply having a device on hand in order to incorporate it into a successful test flow. If you’re working with a test lab that has some degree of automation, for instance, you’ll need to find a way to incorporate these devices into that automated workflow. Here, the temptation is often to root or jailbreak the phones, such that you can install testing software that makes these workflows easier.
Unfortunately, rooting and jailbreaking ultimately defeat the purpose of using flagship devices in the first place. Why? Because you’re essentially hacking these devices, such that they’re no longer identical to the ones your subscribers will be using (to say nothing of the fact that rooted devices don’t receive software/firmware updates), meaning that a service update being developed in the test lab might not actually work in the field. The alternative here is to make sure that you’re partnering with an automation provider who’s able to automate new devices (without rooting or jailbreaking) as soon as they hit the market. In this way, you can incorporate them into your test flows immediately.
2. …Along with Everything Else
Of course, in a perfect world there would be a lot more automation than just flagship mobile devices at play in your test lab. Because every year telco testcases are getting more complicated—and thus more time-consuming to verify—managing multiple test labs for simulating your production network, building out new service offerings, fraud detection, etc. all by hand can problematically time consuming. This is why you’ll need to automate a whole host of other devices and network elements. For instance:
- IP phones for handling PSTN activities
- Attenuators for forcing SRVCC system handover
- REST and SOAP interfaces for verifying provisioning and adjusting subscriber parameters
By including all of these elements in your automation workflow, you put yourself in a position to accurately simulate the functionality of your real network much more quickly than you could otherwise manage.
If, for instance, you had a testbed set up specifically to establish the parameters for a new roaming partnership you were developing with another network, automating attenuators would be crucial to cutting lengthy drive tests out of your test suite. Instead of seeking out the edges of your LTE network to find a spot where the signal dies out, you can incorporate a waning signal into your automated test suite. From there, you can perform the usual rounds of service verification, and finish up by ensuring that you can correctly set usage parameters for your mobile subscribers within that roaming service (and invoice for it accordingly). This gives you all the functionality of true end-to-end tests, to the tune of hundreds of use cases per day without manual effort, all within the boundaries of your testing environment.
3. Prioritize Signalling Data
Above, we stressed the different elements that need to be automated in order to achieve true end-to-end lab tests. This is important for boosting test coverage in a high-leverage environment—but it’s also a matter of future-proofing. Once you’ve got each relevant network elements and test tool digitally connected to your central test lab efforts, it will be much easier to make changes, additions, and improvements down the line. By the same token, the kind of automation under discussion—by virtue of being a digitally-native process—can also help you to make sure that your documentation and test reports are kept in a centralized location, with easy access for all those need it.
This sounds like a pretty solid position to be in. But there is one last tip we want to offer to maximize the effectiveness of your test labs: go beyond end-to-end. What does that mean in practice? It means logging and analyzing all of the signalling traffic produced by your testcases. This can include information about establishing calls, making handovers, and provisioning subscribers—all on a protocol level. In this way, you accomplish two things:
- Earlier detection of error: Individual test elements might be failing or misfiring even in circumstances where the overall test is passing. This won’t always result in a quality of service problem, but it could be a clue to things that may go wrong later.
- Quicker fixes: at the same time, when a test does fail, having signalling data can help you to identify the root cause that much more quickly.
Again, within an automation context these kinds of workflows will include centralized, readable documentation, meaning that future test flows will be able to reference the signalling data for anything that’s going in your test lab right now. This doesn’t just help improve test velocity and coverage—it establishes a solid operational foundation for success.
