How To: HSS Migration Validation

Often, when we talk about quality of service (QoS) we tend to focus on things like latency, jitter, and packet loss, i.e. the network conditions that users experience in their day-to-day mobile device usage.

But back-office processes like provisioning and invoicing can often have just as much of an impact on subscriber happiness in the long term—both because subscribers sometimes have to interact with billing systems and because smoother functioning in the back office often helps network operators to improve service elsewhere.

For this reason, savvy telco operations tend to think long and hard about storage and access to subscriber information.

In the past, testers have often had to make tradeoffs between upfront validation efforts (which can cause delays) and aftercare (which can prove costly and complex), and it can seem like there’s no right answer.

The Challenge

At the end of the day, testers trying to validate a large-scale data migration need to ensure not just that data pollution is within acceptable limits, but also that the functionality of the database itself meets with their expectations.

This can mean testing out provisioning functionality, verifying protocols in the core network’s application layer, and end-to-end tests—with the result being that test cases requiring verification can easily tick up into the hundreds.

In a manual testing environment, this is often simply too much work to get done on a tight timeline. Unfortunately, by cutting corners and triaging test cases you can set yourself up for a database rollout with high levels of data pollution (subscriber data not being stored in the correct place, incorrect subscriber data, etc.).

This ultimately creates more work in the long run, and can irritate internal users and customers in the meantime. SEGRON recently helped a customer (a leading European telco equipment producer) with exactly this test case.

The company was attempting to implement a highly redundant HSS database, meaning that they would have to perform a massive subscriber migration. Verifying the success of the migration was poised to be a Herculean task, and would ultimately involve 200 use cases for verification, each of which required:

• Automatic HSS provisioning
• MAP protocol verification/signalling analysis

On top of that, the test engineers would need to perform extensive regression tests in order to verify that the changes in configurations and software functionality hadn’t had any adverse impacts elsewhere in their network.

Given that the testing team was only able to run through 6 use cases per day by hand, the entire 200 use case testing suite would have taken 30 working person-days—a number that would have meant considerable delays for the migration. Thus, they were faced with a choice: delay, reduce expected test coverage, or automate.

The Approach

Since both a delay or a reduction in test cases would have proved costly in the short or long term for our client, they reached out to SEGRON to automate their subscriber migration verification.

Using our ATF (Automated Test Framework), they were able to quickly and easily orchestrate 200 test cases in a convenient, keyword-based scripting language. From there, they were able to run through those test cases systematically by controlling more than 10 mobile devices through the interface.

The ATF was able to situate those devices within a complex RAN environment (with a mix of 2G, 3G, and 4G functionality), and perform the tests in accordance with requirements that we laid out above.

In this way, they were able to save considerable amounts of time and manual effort. How much time and manual effort? Well, they ran through all 200 test cases in about 10 hours—a reduction of about 95%.

Not only did they speed up testing (and pave the way for easier regression tests, since the test cases had already been established), they also increased its accuracy: while human testers often have their own individual ways of doing things (to say nothing of being human and therefore prone to errors), robots are able to perform the same tasks in standardized way every time.

Thus, the test results are easy to read and help the engineers to uncover the root causes of any errors in the subscriber data that much more quickly. And, because the tests included things like signalling analysis, testers can actually go beyond end-to-end to make sure that the migration hasn’t impacted QoS.

Broader Impacts of Migration Testing

We spoke a little above about the ways that the challenges associated with subscriber data migration might impact costs, but let’s dig a little deeper into the areas where this use case intersects with your company’s bottom line.

First of all, there’s obviously the reduction in testing time and resources: not only does that save costs in its own right, it also frees up engineer hours for more valuable uses of time.

This might mean that testers are able to seek out and address root causes more quickly (potentially bolstering QoS in the process), or that they’re able to turn their attention to refining new service offerings, improving existing functionality, and other tasks that add value for users.

Beyond that, automating migration testing in this way can also have a big impact on subscriber churn. After all, no one likes to get hit with a surprise invoice because their data usage was transferred into the new database incorrectly.

By the same token, no new subscriber wants to run into issues connecting to your network because there’s a bug in your provisioning. Frankly, plenty of people don’t even want their bills to be due on a new day of the month.

The upshot here is that users and subscribers have stringent expectations, and it takes a lot to make them happy. Automation isn’t a panacea, but it can help give you the resources you need to prioritize the things that really matter.