How To: 24/7 Testbed Testing

Testbed-Testing

Back in 2017, the UK launched its 5G Testbeds and Trials Programme, the stated aim of which was to support coordinated research into 5G technology and use cases among British telecom businesses, equipment manufacturers, and scientists.

By their account, the creation of these 5G testbeds provides a crucial proving ground for technology that’s still taking shape, and whose full uses haven’t come close to being explored yet.

The UK envisions a world in which 5G powers increased connectivity in rural areas, on roads, and along rail lines, as well as smart tourism and smart cities and towns, but they won’t know whether these goals are realistic until they’ve had the chance to test things out in a safe environment.

The UK clearly wants a first mover advantage on this front, but they’re not the only ones investing heavily in 5G testbeds. Earlier this year in Thailand, three of the country’s major telco operators joined forces to establish 5G testbeds at two universities across the country to conduct trials on multiple spectrum bands.

If you dig into the news, you’ll likely find more and more stories like this around the world, with testbeds for new technology at the forefront. The tone of these pieces tends to be one of at least cautious excitement about the prospect of new technologies—but they often elide over the very real challenges that testbeds can present for telecom testers.

The Challenge

Because 5G isn’t the only new technology coming down the pipe, the typical telco operators’ dream scenario involves running multiple testbeds at the same time in order to test possible development and deployment scenarios.

And the need goes well beyond new technology. There might be one testbed for simulating the real operational production network, there might be another testbed for testing new future capabilities and functionalities, and there might be another for capacity, coverage, fraud detection, etc.

Each of these testbeds need to meticulously recreate the parameters of their real production network, and they have to be equipped with the latest devices and other technology.

Once the testbeds are set up, they require constant verification in order to give engineers a concrete sense of how their experiments are panning out. Because the work is constantly ongoing, the ideal is to test on 24/7 basis within each testbed to facilitate troubleshooting.

Unfortunately, manual testing workflows cause this to become resource intensive incredibly quickly. Based on SEGRON’s past experience, it takes approximately one working day for a test engineer to verify 6-10 use cases.

This includes results and root cause analysis. Naturally, this can vary significantly depending on the complexity of the use case and possible root cause. When test are run manually, they are also susceptible to human error, which can actually increase test volume, since you’ll have to run through test cases again.

Like we saw with the 5G examples above, these testbeds are crucial to developing new service offerings or expanding coverage, but it’s always a struggle to allocate the necessary resources.

The Approach

While manual testing makes it time and resource intensive to effectively manage multiple testbeds, automation has the potential to alleviate that hurdle significantly. Crucially, to automate for this use case in particular testers will need to be able to control the latest, most cutting-edge devices right out of the box.

While an automation solution that uses rooted or simulated devices might increase test coverage, it could have deleterious effect on test quality, considering that the point of a testbed is to replicate real network conditions as closely as possible.

Thus, for something like 5G testbed verification, testers would want to be able to control a sample of devices that’s representative of the current market, not to mention the latest routers and other network elements.

Once the relevant devices (plus 1-2 attenuators to ensure and control radio frequency power and network domain—2G, 3G, 4G, 5G, Wi-Fi, etc.) are set up, it’s just a matter of either identifying the right test cases from an existing library or scripting up your own (which, in a keyword-based framework is relatively quick and easy).

These tests can be set to run in the background of your testbed without any human intervention, 24 hours a day. All of a sudden, a task that seemed to be so time and resource intensive as to make a decent ROI look improbable becomes eminently manageable.

Rather than laboriously testing half a dozen use cases by hand per day, testers can easily achieve high degrees of test coverage while getting automated, keyword triggered alerts as bugs and service gaps become apparent.

Implications of Testbed Availability for Telecoms

Automation of the sort that we’ve been discussing above has the potential to greatly reduce costs, making it possible for telco operators to run multiple testbeds at once without stretching their resources too thin.

This should be thought of as a win in its own right—but it’s not the only potential benefit to automating. Because your engineers will be able to devote more time to addressing root causes and to coming up with innovative solutions within those testbeds (rather than simply running tests ad nauseam).

They might experience improved job satisfaction, reducing test engineer churn. By the same token, if testers are able to devote more time to finding root causes in their test lab operations, they can refine new service offerings more quickly, leading to faster time-to-market.

Of course, the ultimate goal here is to add value for your customers—which is the whole point of developing exciting new services (be they 5G rollouts, expansions into a new area, or new IoT integrations) in the first place.

They’re the ones who decide whether or not to subscribe to your service or use your products—which is why anything you can do to put the focus of your test engineers on customers, rather than technical minutiae, can create a significant advantage for your business.

Learn More About Test Automation

Click below to see how SEGRON’s ATF can help you automate testing.

Interested in our Products ?

Scroll Up

pencil Download your White Paper now

TECHNICAL PROJECT MANAGER

You will be a part of the SEGRON Project Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation.

As technical project manager, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases and you have to setup and align related project activities together with our customers.
You should have a strong knowledge and background in the telco environment paired with project management skills to align projects, track the related KPI’s further having a strong view also on the financial performance.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible
Team player:
• collaborates with others
• teach new skills to others and write tutorials
Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Several years of experience in Telecom Domain.
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• IMS/SDM/EPC domain knowledge
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack.
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge.

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus.
• Robot Framework / Python

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• PM Certification or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2500 EUR (depends on experience)

DEVELOPER TEST AUTOMATION

You will be a part of the SEGRON Test Automation Development Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation using Python, Robot.

As test automation developer, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases, building automation tools, framework, creating reusable python libraries for test automation.
You should have a strong knowledge and hands-on experience in test automation framework (Robot) and Python scripting language. Further you must continuously renew the automation & review guidelines set within the team. Ensure developed reusable keywords & test scripts meet the automation guideline.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible 


Team player:
• collaborates with others
• teach new skills to others and write tutorials


Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Experience in developing reusable test automation scripts using Python and Robot Framework
• Several years of experience in Telecom Domain
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• Test Case development
• Mobile development
• Design Patterns
• Web development/testing
• IMS/SDM/EPC domain knowledge
• Hand-on experience in Docker containers and Kubernetes.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2000 EUR (depends on experience)