How Risky Is Outsourced Network Testing in 2019?

how-risky-is-it-to-outsource-your-network-testing

Let’s say you’re working through a whole slate of different test scenarios to verify service on a new network that you’re rolling out. One of your first tasks is tackling VoIP (voice over IP) tests, which, as it turns out, present some very particular challenges.

Because jitter and latency in voice conversations can quickly render a call frustrating and incomprehensible, your tests have to seek out extremely granular data about packet loss and packet delay for a number of different use cases. In order to do so effectively, testers need a wealth of specialized knowledge.

Where less time sensitive forms of communication tend, to a certain extent, to maximize packet size in order to reduce overhead,  the nature of human speech means that VoIP protocols tend to prefer smaller (64 byte) packets, such that if a single packet is lost the listener will only miss a fraction of a syllable.

At anything under about 3% packet loss, your quality of service won’t take much of a hit. For packet delay, anything over 200 ms is likely to cause quality of service issues, while for jitter (the variation in delay time among packets), that number is more like 35 ms.

Because these numbers leave so little room for error, they require not just specialized knowledge but frequently also specialized equipment. Why? Because virtual test environments are often just dissimilar enough from reality that a few milliseconds here or there can slip through the cracks.

Add this to the fact that use cases that require testing are proliferating every year, and it’s no wonder more and more telco operators are opting to outsource their testing. The question is: how risky is it to do so?

Hurdles for In-house Testers

We covered this somewhat above, but let’s quickly lay out the argument in favor of outsourcing tests. First of all, testing has become more complex and time consuming than ever, with new protocols constantly popping up while legacy telephony continues to require verification.

A human tester can run through about 6 use cases per day, meaning that reaching high levels of in-house test coverage often seems daunting or impossible. At the same time, customer standards for quality of service are getting more and more stringent, meaning that low test coverage isn’t an option. Things can’t stay as they are—something’s gotta give.

By outsourcing tests, telco operators solve these immediate problems. All of a sudden it’s feasible to achieve high test coverage and work towards improved network quality without taking up the time of a small army of in-house engineers who would otherwise be working on more valuable projects.

In addressing the immediate issues in this way, however, telecoms can set themselves up for additional hurdles down the road. Why? Because most test outsourcing options don’t empower their clients to retain institutional testing knowledge, making it harder and harder to regain control of one’s tests even as new technologies like automation make in-house testing more feasible

Institutional Domain Knowledge

In the first section above, we threw out a handful of numbers for acceptable latency times and rates of packet loss. If you’re in the business of testing VoIP with any regularity, you likely have those numbers or similar ones in mind already, just as you know off-hand a host of other granular details about how to establish your test environment, which use cases to verify, and what to look for in your tests.

If you’re not in the habit of performing these tests regularly, on the other hand, you probably don’t know most of these details, and would need to rely on outside resources in order to effectively verify service on your own. When this happens at a personal scale, it’s not necessarily impossible to recover—but when it happens at an institutional scale, the results can be worrisome.

If, as a company, you find that after a few update cycles with outsourced tests you’ve lost the ability to do the work in-house, even on a small scale, you essentially put yourself in a position where you’re totally reliant on your testing provider.

This reduces your agility as a company and increases the potential for organizational disconnect and the emergence of information silos. Though your test coverage may be much higher than it would have been otherwise, you might not be able to maximize the value of that test coverage because your internal processes don’t align exactly with the way your testing partner presents their findings.

And since your team potentially can’t reproduce the test conditions, it may be difficult to actually fix any bugs that have been uncovered.

Alternate Paths

Okay, so when you outsource testing you risk losing operational agility and domain knowledge, which can result in ineffective bug response and reduced test quality in the long run. Does this mean that telco operators are stuck devoting almost unthinkable time and resources to reaching high levels of in-house test coverage manually? Thankfully, the answer is ‘no.’

Though the risks we outline above do exist, it is possible to partner with a testing provider who can help you maintain and even improve your institutional domain knowledge, rather than letting it languish.

By providing robust and comprehensive reporting that clearly defines its use cases and KPIs, a testing provider can ensure that you have exactly the information you need to reproduce and understand test results. Since tests are only truly valuable when they’re repeatable, this ought to be a deal breaker.

If we’re talking about a service that can run automated tests (which would make test coverage a non-issue), they might even be able to give you the tools you need to run automated tests in-house, thereby sidestepping the entire problem.

In this way, you can boost your use cases per day up into the hundreds, saving resources and person hours for other tasks like actually resolving service quality issues. All of sudden, your agility is restored and you have ownership over your own service verification operations.

This means fewer budget surprises and more oversight from within your corporate structure—to say nothing of the increase in practical knowledge throughout the organization.

Interested in our Products ?

Scroll Up

pencil Download your White Paper now

TECHNICAL PROJECT MANAGER

You will be a part of the SEGRON Project Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation.

As technical project manager, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases and you have to setup and align related project activities together with our customers.
You should have a strong knowledge and background in the telco environment paired with project management skills to align projects, track the related KPI’s further having a strong view also on the financial performance.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible
Team player:
• collaborates with others
• teach new skills to others and write tutorials
Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Several years of experience in Telecom Domain.
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• IMS/SDM/EPC domain knowledge
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack.
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge.

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus.
• Robot Framework / Python

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• PM Certification or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2500 EUR (depends on experience)

DEVELOPER TEST AUTOMATION

You will be a part of the SEGRON Test Automation Development Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation using Python, Robot.

As test automation developer, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases, building automation tools, framework, creating reusable python libraries for test automation.
You should have a strong knowledge and hands-on experience in test automation framework (Robot) and Python scripting language. Further you must continuously renew the automation & review guidelines set within the team. Ensure developed reusable keywords & test scripts meet the automation guideline.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible 


Team player:
• collaborates with others
• teach new skills to others and write tutorials


Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Experience in developing reusable test automation scripts using Python and Robot Framework
• Several years of experience in Telecom Domain
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• Test Case development
• Mobile development
• Design Patterns
• Web development/testing
• IMS/SDM/EPC domain knowledge
• Hand-on experience in Docker containers and Kubernetes.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2000 EUR (depends on experience)