How Can Telco Operators Speed Up Conformance Testing?

how-can-telco-operators-speed-up-conformance-testing

When technology changes and evolves—as it does almost constantly—in the telecom domain, it typically takes standard-setting bodies like 3GPP six months to a year to establish a new set of test cases for conformance testers.

Once those test cases come out, there’s a flurry of activity while operators, device manufacturers, OEMs, and others attempt to verify compliance and interoperability with new and existing standards.

The fact that it takes 3GPP a fairly long stretch of time does very little to lessen the time pressure that testers usually face when it comes to performing each new round of service verification.

What are testers supposed to do under these conditions? Everyone wants faster conformance testing in order to ensure high network quality, but given the increasing complexity of the networks, devices, and protocols under test.

It seems like there’s virtually no way for manual testers to even maintain their current pace, let alone speed things up. If that’s the case (and we think that it is), then it’s time for the telco domain to begin moving away from manual conformance tests and towards real test automation.

Why? Because it presents testers with their best past forward for running through an increasing number of use cases to verify service.

Hurdles in Conformance Testing

Because conformance tests tend to be based, at least in part, on standards developed by outside organizations, there’s often a bit more uncertainty baked into the process than usual.

Rather than either reusing test cases or developing new ones in tandem with the developers making changes or updates to your network, you’re stuck waiting for the starting gun to go off and then adapting 3GPP’s recommendations within the context of your own LTE network, for instance.

This means that your testing velocity is based not just on how many use cases you can run through in a day, but also how quickly you can configure those test cases based on the information you’re receiving.

This will require understanding the standards themselves and breaking down each element into a set of levels and modules that can be mapped out and visualized.

The more easily you can do that, the more quickly you can get your tests underway. Even at this point, however, you’re struggling against the challenges that face all varieties of telecom testing right now—namely that increasing complexity is driving down the number of use cases a human tester can verify in a single day.

This number, on average, is down from about 10 last decade to more like 6 today. As a result, even if you establish your testing framework quickly, it’s still difficult to turn around the tests themselves in a timely manner.

The Power of Automation

For operators seeking a new approach, this is where automation comes in. First, on the level of test case design, any automation solution worth its salt will have an easy-to-use scripting language for defining use cases.

This means that when your test cases are relatively unknown during the lead-up to a test, you can still get things off the ground fairly quickly. Once the testing begins, you can boost your test output from half a dozen test cases per engineer per day to hundreds or thousands of test cases.

Broadly speaking, as more and more protocols need to be tested for compliance and interoperability, test coverage threatens to go down—but by ditching manual tests and letting an automated test framework take over the actual verification stages, you can reverse that trend, improving your test coverage in record time.

It’s worth noting here that when we talk about automated tests we don’t mean simulations. As new technologies like 5G arise that have increasingly precise standards and service quality expectations, the differences between real, out-of-the-box devices and equipment and simulated versions of the equipment will become increasingly noticeable.

This might not seem like much when you’re simply checking to see whether a call was completed successfully or not, but even when it comes to functional testing the line between success and failure can gain a certain fuzziness that testers need to fight against. In the long run, failure to identify, document, and resolve issues that may crop up in conformance testing is one of the other key factors that slow down this kind of verification.

Why? Because the accumulated issues gum up the works of your test lab and engineering teams, making it difficult to maintain agility overall. In an automated environment, this loss of agility can be staved off through faster test execution with higher coverage.

Achieving ROI

Okay, we’ve seen the reasons why conformance testing in the modern telco environment can be slow and painful, and some of the ways in which automation can help. But are faster conformance tests an adequate end in and of themselves, or should they be powering some larger goal like faster time-to-market or improved ROI?

From our perspective, it’s absolutely the latter. Why? Because whether your tests are automated or manual, it’s still far too common to see poor reporting or confusion over roles devalue test results.

Finding gaps in functionality is all well good, but how useful is it if uncovering those issues doesn’t result in the issues being resolved in a timely manner?

In this way, we can reimagine our standard for test velocity to cover not just the speed of the tests themselves, but how quickly and effectively the reporting translates into results. If your tests, automated or not, uncover an error on SIM password validation for certain equipment.

But your engineers can’t figure that out based on the test’s output (whether because it’s hard to read, hard to access, or stuck in an information silo), you’re potentially achieving and verifying compliance more slowly than you would be otherwise. Here, automated reporting has the potential to offer a massive advantage.

But only if the automation vendor you select makes reporting transparency a priority. If they do, you stand to increase not just test velocity but actual service verification velocity for your conformance tests, improving your time to market and freeing up valuable engineer time in the process.

Interested in our Products ?

Scroll Up

pencil Download your White Paper now

TECHNICAL PROJECT MANAGER

You will be a part of the SEGRON Project Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation.

As technical project manager, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases and you have to setup and align related project activities together with our customers.
You should have a strong knowledge and background in the telco environment paired with project management skills to align projects, track the related KPI’s further having a strong view also on the financial performance.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible
Team player:
• collaborates with others
• teach new skills to others and write tutorials
Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Several years of experience in Telecom Domain.
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• IMS/SDM/EPC domain knowledge
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack.
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge.

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus.
• Robot Framework / Python

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• PM Certification or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2500 EUR (depends on experience)

DEVELOPER TEST AUTOMATION

You will be a part of the SEGRON Test Automation Development Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation using Python, Robot.

As test automation developer, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases, building automation tools, framework, creating reusable python libraries for test automation.
You should have a strong knowledge and hands-on experience in test automation framework (Robot) and Python scripting language. Further you must continuously renew the automation & review guidelines set within the team. Ensure developed reusable keywords & test scripts meet the automation guideline.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible 


Team player:
• collaborates with others
• teach new skills to others and write tutorials


Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Experience in developing reusable test automation scripts using Python and Robot Framework
• Several years of experience in Telecom Domain
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• Test Case development
• Mobile development
• Design Patterns
• Web development/testing
• IMS/SDM/EPC domain knowledge
• Hand-on experience in Docker containers and Kubernetes.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2000 EUR (depends on experience)