Can Testing Automation Improve Network Quality?

Blue virtual technology background with lines and grids

According to a recent McKinsey study, network quality concerns comprise some of the most important factors impacting a given customer’s choice of mobile carrier. While pricing is still the most important one on average, survey respondents were also quick to list national and local coverage, network speed, and quality of 4G as critical deciding factors in carrier selection.

In spite of the growing importance of network quality, however, McKinsey also found that the average quality of service for voice has decreased across Europe in recent years.

Sure, usage patterns among mobile phone users are changing, but this state of affairs still seems unsustainable. If a particular operator constantly makes gains in mobile broadband at the expense of voice functionality, they’re likely to see an increase in churn.

Why? Because customers increasingly think of their service as a holistic experience, rather than a set of discreet service offerings. Just because you’re able to offer the lowest latency for voice-over-IP, doesn’t mean that you’ll be forgiven for spotty geographical coverage, and vice versa.

The question is: how can telco operators use the resources they have to improve network quality and meet ever-changing customer requirements?

One intriguing potential answer is testing automation.

Measuring Network Quality

The use of mobile data continues to grow exponentially each year, and with that explosive growth comes a corresponding increase in complexity. The number of potential test cases for “typical” mobile device usage, for instance, increases as the diversity in potential user experiences ticks up.

A given user might be especially worried about latency times while using a particular app during periods where capacity is already strained, all while using a multi-band smartphone that potentially presents signaling problems.

At the same time, another use might be using primarily voice and messaging apps around the country, and is thus more concerned with packet loss and widespread coverage. There isn’t exactly one objective KPI that perfectly encapsulates the needs of these two users, meaning your quality of service (QoS) verification is necessarily going to be somewhat of a patchwork—increasingly like your service itself.

What this means, essentially, is that if you’re using a metric like peak-hour bandwidth or latency, or total throughput, or what have you, you may be missing crucial elements of the way that your network quality is perceived by end-users.

Small differences in speed or capacity under lab-test conditions might be important to your average telco operator from a technical perspective, but it’s not always the case that the best technical stats translate into the best customer experience.

Just as diversity is increasing rapidly among potential devices and protocols that your network needs to support, there’s also an increasing diversity of potential use cases that are increasingly difficult to replicate without reproducing the exact usage conditions being considered.

The Impact (and Limitations) of Testing

Because quality of service is more and more defined by performance in particular use cases, it stands to reason that network quality verification would become increasingly reliant on use case-based testing.

Sure, your test lab will still need to use cutting-edge equipment to optimize your performance, but from the perspective of operational regression testing the focus should be less on verifying network quality under these conditions and more about meeting customer needs across the innumerable service configurations that arise across the various and disparate touchpoints on your network.

This means putting less emphasis on simulated tests, or tests performed on rooted devices, and focusing instead on verifying the devices and scenarios that your users (the ultimate arbiters of your QoS) are likely to encounter.

By refocusing your service verification in this way, you can create better alignment between user experience and network functionality—i.e. you uncover the issues that are most likely to impact your users on a daily basis.

If you can preempt the kinds of service shortfalls that really affect public perception before they occur—whether that’s by getting ahead of a new software update or by performing routine regression testing—you can keep your users from switching to a different operator.

The only issue here is that the kind of testing we’re discussing can be prohibitively time-consuming. Your average test engineer may only be able to run through 6-10 use cases per day, meaning that any time you roll out a new update it’s likely that there will be some unexplored, unverified territory. This is where automation comes in.

The Power of Automation

We’ve seen the ways that use-case based testing can improve perceived quality of service within your network—now’ll look to automation as a way to make this kind of testing feasible. Here, by automation we don’t mean simulated tests, nor do we mean volume testing performed on rooted devices.

Rather, automation in this context means running scalable, repeatable tests on the same devices that end-users will ultimately be utilizing, e.g. smartphones, tablets, etc. Using the kinds of workflows we’re talking about, you could potentially increase test volume by orders of magnitude, going from half a dozen or a dozen tests per day to hundreds.

Thus, rather than prioritizing different configurations for verification and hoping that you get through the most common ones before you have to roll out the latest network update, you can breeze through test cases quickly and efficiently. In this way, the ideal alignment between QoS perception and service verification is made not just possible but cost- and time-effective.

Not only does this give telco operators the tools to improve network quality from the point of view of customers’ changing needs, it gives them a foundation for continued success. Since automated processes like the ones we’re discussing are, by their nature, digital, scalable, and repeatable, testing a large volume of use cases can become an everyday practice, rather than an occasional time-consuming boondoggle.

Different configurations can be documented, standardized, and tracked, meaning that, taking voice quality as an example, daily regression testing can help you collect data that can be analyzed over time.

Based on this data analysis, you can continue to improve the quality of your voice services even when your (and your customers’) primary focus might be elsewhere. In the long term, the value of your data analytics will compound—leading to even further improvements to your network quality and improved ROI for your testing workflows.

Learn More About Test Automation

Click below to see how SEGRON’s ATF can help you automate testing.

 

Interested in our Products ?

Scroll Up

pencil Download your White Paper now

TECHNICAL PROJECT MANAGER

You will be a part of the SEGRON Project Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation.

As technical project manager, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases and you have to setup and align related project activities together with our customers.
You should have a strong knowledge and background in the telco environment paired with project management skills to align projects, track the related KPI’s further having a strong view also on the financial performance.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible
Team player:
• collaborates with others
• teach new skills to others and write tutorials
Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Several years of experience in Telecom Domain.
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• IMS/SDM/EPC domain knowledge
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack.
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge.

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus.
• Robot Framework / Python

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• PM Certification or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2500 EUR (depends on experience)

DEVELOPER TEST AUTOMATION

You will be a part of the SEGRON Test Automation Development Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation using Python, Robot.

As test automation developer, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases, building automation tools, framework, creating reusable python libraries for test automation.
You should have a strong knowledge and hands-on experience in test automation framework (Robot) and Python scripting language. Further you must continuously renew the automation & review guidelines set within the team. Ensure developed reusable keywords & test scripts meet the automation guideline.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible 


Team player:
• collaborates with others
• teach new skills to others and write tutorials


Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Experience in developing reusable test automation scripts using Python and Robot Framework
• Several years of experience in Telecom Domain
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• Test Case development
• Mobile development
• Design Patterns
• Web development/testing
• IMS/SDM/EPC domain knowledge
• Hand-on experience in Docker containers and Kubernetes.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2000 EUR (depends on experience)