Case Study: PCRF Regression Testing

Way back in the day, if you were on the go and needed to make a phone call, you kept your eyes peeled for the nearest payphone. You would drop in your coins and the operator would connect you to your desired call recipient.

After the time you paid for had run out, a live operator would butt in and let you know that in order to extend your call you needed to add more money. Today, the idea of something like this happening during a VoLTE call seems a little bit absurd.

But in point of fact it represents a need that all telco operators still grapple with: real-time provisioning of services based on what the user has actually paid for.

But network providers still need a way to monitor subscriber voice and data usage in real-time. Hence the rise of the Policy and Charging Rules Function (PCRF) as a key component in most EPC and LTE networks.  

What is PCRF?

Okay, first things first: what exactly is PCRF, and why do testers need to worry about it? Essentially, PCRF is a software function within a multimedia network that’s designed to manage subscriber policies and charging rules as users make use of data and voice services.

The PCRF server is located within the core network, with access to subscriber data and policy servers, and it creates rules and enforces policies for any subscribers accessing your network.

Based on each user’s quality of service (QoS) level (as determined by accessing the relevant policy in the server), PCRF will provision, route, and prioritize services and network traffic as appropriate.

This is, in some ways, analogous to the live operator on the payphone directing traffic and monitoring usage—but operating on a much more granular, sophisticated level.

If, for instance, a subscriber is using high bandwidth applications and running through huge amounts of mobile data, the PCRF can implement a policy decision (via connection with OSS/BSS systems) to charge the user appropriately.

By the same token, it might identify that a user is connected to roaming service and thus impose slower up/downlink speeds for certain apps.

In this way, telco operators are able to work towards a few goals:

• Dynamically provisioning bandwidth based on QoS personas (to improve overall network quality for users)
• Improving revenue assurance with real-time charging rules
• More effectively providing differentiated levels of service
• Managing network traffic in a more efficient manner.

Based on the above, it’s pretty easy to see why PCRF is a key element of most EPC networks. That said, the nature of this kind of functionality can present challenges for the testers who are tasked with verifying service across the entire network.

Challenges in PCRF Testing

Naturally, all of the potential QoS and revenue benefits we described above only occur if the service actually works. This leaves testers with the complex task of testing proper functionality for every QoS persona in their subscriber database.

If, for instance, you have two dozen possible policy levels, each one with a different set of restrictions and requirements, you need to create two dozen dummy subscriber accounts that correspond to the different QoS policies, and assign each one in turn to a subscriber phone.

For each test case, testers need to initiate a packet session and run through the gamut of potential data-usage scenarios, in order to verify that the PCRF creates and carries out the correct policy decision on each one.

Of course, testing this functionality is hardly a “one-and-done” kind of use case. On the contrary, any change you make to your network could result in changes to the PCRF and its protocol elements, meaning that all of the effort we outlined above has to be replicated via regression tests each time you update your network. You need to make sure that as each packet session is properly set up:

• A request is sent to the PCRF
• The PCRF successfully accesses and queries the subscriber profile database
• The PCRF installs subscriber and policy rules before the session is established
• If the user begins to exceed session credits, the PCRF is notified again
• Once notified, the PCRF consults the policy and issues additional credits as appropriate, while prompting a notification (e.g. an SMS message) to the subscriber.

And all of this has to be verified at every QoS level. Unsurprisingly, test cases pile up pretty quickly.

The Power of Test Automation

In SEGRON’s experience, the most effective remedy for the mountain of test cases that can come with PCRF regression testing is to automate as much of your testing as possible. And, in fact, we recently helped a client (a leading European operator) to do just that.

Previously, because of the depth and complexity of performing regression tests for these test cases, it would take about 4 days to run through a slate of 20 extremely granular test cases—which they had to do every time they made an update to their network.

This clearly wouldn’t be feasible in the long run, so they turned to the SEGRON ATF (Automated Test Framework) to automate their regression tests.

Rather than using simulated or rooted devices, SEGRON was able to orchestrate tests in a complex test environment using out-of-the-box Android and iOS devices within about a day.

From there, their test cases were scripted up in a keyword-based framework, at which point they were able to run through all 20 priority-1 within about half an hour. This represented a reduction in test time of more than 98%—meaning that the company could now quickly and easily run PCRF regression tests even after relatively minor changes to their networks.

Combine this reduction in test time with an increase in readability of reports, and the test team was suddenly able to find the root causes of any bugs that much more quickly.

From there, they were able to fix bugs and refocus their attention on more value-additive tasks, helping to improve the company’s overall allocation of resources, resulting in improved time-to-market and ROI.