Case Study: PCRF Regression Testing

case-study-pcrf-regression-testing

Way back in the day, if you were on the go and needed to make a phone call, you kept your eyes peeled for the nearest payphone. You would drop in your coins and the operator would connect you to your desired call recipient.

After the time you paid for had run out, a live operator would butt in and let you know that in order to extend your call you needed to add more money. Today, the idea of something like this happening during a VoLTE call seems a little bit absurd.

But in point of fact it represents a need that all telco operators still grapple with: real-time provisioning of services based on what the user has actually paid for.

Back in the day, this happened by way of live operators, who were later replaced by automated operators. Of course, with the rise of mobile telephony and mobile data usage, no one expects to experience a literal service interruption in this same way.

But network providers still need a way to monitor subscriber voice and data usage in real-time. Hence the rise of the Policy and Charging Rules Function (PCRF) as a key component in most EPC and LTE networks.  

What is PCRF?

Okay, first things first: what exactly is PCRF, and why do testers need to worry about it? Essentially, PCRF is a software function within a multimedia network that’s designed to manage subscriber policies and charging rules as users make use of data and voice services.

The PCRF server is located within the core network, with access to subscriber data and policy servers, and it creates rules and enforces policies for any subscribers accessing your network.

Based on each user’s quality of service (QoS) level (as determined by accessing the relevant policy in the server), PCRF will provision, route, and prioritize services and network traffic as appropriate.

This is, in some ways, analogous to the live operator on the payphone directing traffic and monitoring usage—but operating on a much more granular, sophisticated level.

If, for instance, a subscriber is using high bandwidth applications and running through huge amounts of mobile data, the PCRF can implement a policy decision (via connection with OSS/BSS systems) to charge the user appropriately.

By the same token, it might identify that a user is connected to roaming service and thus impose slower up/downlink speeds for certain apps.

In this way, telco operators are able to work towards a few goals:

  • Dynamically provisioning bandwidth based on QoS personas (to improve overall network quality for users)
  • Improving revenue assurance with real-time charging rules
  • More effectively providing differentiated levels of service
  • Managing network traffic in a more efficient manner.

Based on the above, it’s pretty easy to see why PCRF is a key element of most EPC networks. That said, the nature of this kind of functionality can present challenges for the testers who are tasked with verifying service across the entire network.

Challenges in PCRF Testing

Naturally, all of the potential QoS and revenue benefits we described above only occur if the service actually works. This leaves testers with the complex task of testing proper functionality for every QoS persona in their subscriber database.

If, for instance, you have two dozen possible policy levels, each one with a different set of restrictions and requirements, you need to create two dozen dummy subscriber accounts that correspond to the different QoS policies, and assign each one in turn to a subscriber phone.

For each test case, testers need to initiate a packet session and run through the gamut of potential data-usage scenarios, in order to verify that the PCRF creates and carries out the correct policy decision on each one.

Of course, testing this functionality is hardly a “one-and-done” kind of use case. On the contrary, any change you make to your network could result in changes to the PCRF and its protocol elements, meaning that all of the effort we outlined above has to be replicated via regression tests each time you update your network. You need to make sure that as each packet session is properly set up:

  • A request is sent to the PCRF
  • The PCRF successfully accesses and queries the subscriber profile database
  • The PCRF installs subscriber and policy rules before the session is established
  • If the user begins to exceed session credits, the PCRF is notified again
  • Once notified, the PCRF consults the policy and issues additional credits as appropriate, while prompting a notification (e.g. an SMS message) to the subscriber.

And all of this has to be verified at every QoS level. Unsurprisingly, test cases pile up pretty quickly.

The Power of Test Automation

In SEGRON’s experience, the most effective remedy for the mountain of test cases that can come with PCRF regression testing is to automate as much of your testing as possible. And, in fact, we recently helped a client (a leading European operator) to do just that.

Previously, because of the depth and complexity of performing regression tests for these test cases, it would take about 4 days to run through a slate of 20 extremely granular test cases—which they had to do every time they made an update to their network.

This clearly wouldn’t be feasible in the long run, so they turned to the SEGRON ATF (Automated Test Framework) to automate their regression tests.

Rather than using simulated or rooted devices, SEGRON was able to orchestrate tests in a complex test environment using out-of-the-box Android and iOS devices within about a day.

From there, their test cases were scripted up in a keyword-based framework, at which point they were able to run through all 20 priority-1 within about half an hour. This represented a reduction in test time of more than 98%—meaning that the company could now quickly and easily run PCRF regression tests even after relatively minor changes to their networks.

Combine this reduction in test time with an increase in readability of reports, and the test team was suddenly able to find the root causes of any bugs that much more quickly.

From there, they were able to fix bugs and refocus their attention on more value-additive tasks, helping to improve the company’s overall allocation of resources, resulting in improved time-to-market and ROI.

 

Interested in our Products ?

Scroll Up

pencil Download your White Paper now

TECHNICAL PROJECT MANAGER

You will be a part of the SEGRON Project Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation.

As technical project manager, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases and you have to setup and align related project activities together with our customers.
You should have a strong knowledge and background in the telco environment paired with project management skills to align projects, track the related KPI’s further having a strong view also on the financial performance.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible
Team player:
• collaborates with others
• teach new skills to others and write tutorials
Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Several years of experience in Telecom Domain.
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• IMS/SDM/EPC domain knowledge
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack.
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge.

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus.
• Robot Framework / Python

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• PM Certification or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2500 EUR (depends on experience)

DEVELOPER TEST AUTOMATION

You will be a part of the SEGRON Test Automation Development Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation using Python, Robot.

As test automation developer, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases, building automation tools, framework, creating reusable python libraries for test automation.
You should have a strong knowledge and hands-on experience in test automation framework (Robot) and Python scripting language. Further you must continuously renew the automation & review guidelines set within the team. Ensure developed reusable keywords & test scripts meet the automation guideline.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible 


Team player:
• collaborates with others
• teach new skills to others and write tutorials


Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Experience in developing reusable test automation scripts using Python and Robot Framework
• Several years of experience in Telecom Domain
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• Test Case development
• Mobile development
• Design Patterns
• Web development/testing
• IMS/SDM/EPC domain knowledge
• Hand-on experience in Docker containers and Kubernetes.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2000 EUR (depends on experience)