3 Examples of End-to-End Telecom Testing in Action

We talk a lot on this blog about end-to-end testing, and we don’t plan to change that fact any time soon. Why? Because end-to-end still represents the only testing methodology that puts the needs of end-users at the center of the testing process—and end-user experience is only becoming more important in the ever-changing telecom domain.

So, naturally we want to give our readers the tools and information they need to outline end-to-end tests within their networks in order to maintain a high quality of service. That’s why today we’re taking a deeper look at some specific instances of end-to-end testing, in order to provide a more concrete idea of what this methodology looks like in practice.

Let’s jump right in!

VoLTE Verification

Let’s say you’re rolling out a major update to your VoLTE service. Because end-to-end testing is mapped out based on user functions and conditions, service verification for this functionality is going to relate specifically to the different circumstances under which end-users will actually attempt to access your network—making calls across different devices, in different locations, and with different outcomes.

This means that you need to measure call success or failure on any number of end-user devices, plus a whole host of different possible actions within the call itself:

• Emergency calling
• Handovers/relocation
• Roaming/interconnect
• Call holding (second-party or switch)
• Redirect to voicemail (because of call not answered or called user is busy)
• Call release (initiated at either end-point)
• Call cancel (before or after ringing)
• Signal/connectivity loss (SIP or PDN)

This list should represent most of the actions that need to be verified from the perspective of voice-only calling. From there, the same logic would produce a similar set of possibilities for voice and video calling (plus a few extra cases for multi-party calling, calls answered with voice but not video, etc.).

Which would also have to be verified across each of the various devices with which end-users might be accessing the network, in order to minimize the likelihood that any user will experience service failures.

For most of the functionality listed out above, a simple success or failure should be fairly easy to determine. For some, however, it’s necessary not just to determine whether a particular call went through, but also whether it met minimum standards for voice quality.

You can measure this according to your own standards (which will hopefully have some basis in customer data), but a good starting point would be measuring jitter, latency, and packet loss and working to keep them within acceptable levels. As customer expectations change, your standards may have to change alongside them.

BSS Testing

If we take all of the use cases outlined above and multiply them by the number of devices that require verification, all of a sudden testers are staring down a mountain of work that can seem insurmountable under the time pressure that usually accompanies network updates.

And, indeed, there are numerous areas within the average telco network structure where things like increased customer expectations and device fragmentation wind up having a direct impact on your ability to offer high quality of service and retain customers.

Luckily, though that category is extensive, it doesn’t include everything. Your billing system, for instance, will usually not be directly customer-facing in a way that impacts daily usage (though it obviously does touch the customer at some point). At the same time, successful service in this area is a prerequisite for getting paid.

Here, the end-to-end flow involves a few more steps than it does in the previous use case:

• Verify that the phone number is valid, in-service, and registered to a customer.
• Check for outstanding bills or any existing balance.
• Accurately tally up voice and data activity for the relevant number and contract parameters.
• Successfully update invoices as needed.

This is an area in which the definition of end-to-end might be extended beyond end user use cases to include users within your own business. Yes, you’ll have to be sure that customers are never presented with incorrect billing numbers.

But arguably more important is ensuring that your BSS team members are able to access the correct numbers and analyze/interact with them as needed. This means that you might want to create a parallel set of use cases that meet internal needs:

• Verify successful creation of correct billing information.
• Check that the billing info matches the equivalent data in your CRM.
• Verify that the status of the invoice (paid or unpaid) is updated appropriately and reflected in the CRM.

Application Tests

In addition to verifying voice calling over any given network type and BSS functionality, telco operators are often called upon to ensure that either built-in or downloaded mobile apps actually function as advertised on their networks. If we take a banking app as an example, this means that testers need to verify functionality for:

• User log-ins
• Checking balances
• Transferring funds
• Performing other transactions

Here, the user’s perspective is back in the driver’s seat—and the security of the user’s data is paramount! For this reason, testers need to be careful about orchestrating their tests in such a way that they can verify not just passage and failure for particular actions that a customer might take, but also that in the case of failure the correct error messages are displayed and the flow of information stops in the right place.

We spoke a little bit above about the ways in which customer expectations are changing with regards to modern telco networks, and that’s worth reiterating here. Expectations for service quality are rising at exactly the same time that the number of use cases requiring verification is skyrocketing—due not just to increasing device fragmentation.

But also new applications being made possible by decreases in latency times and increases in sophistication. As your network becomes more complex, end-to-end testing will become increasingly important as a means of ensuring high network quality, and thus retaining your customers and maximizing your testing ROI. As that happens, the trick will be finding a way to run through large numbers of test cases in record time.