The Telco Operator’s Guide to Test Case Scripting

Science graphic against researcher hands holding sprouts in petri dish

One of the biggest concerns we hear from telco operators looking to automate their tests is the relative ease or difficulty of test case scripting. Telecoms aren’t typically staffed with a huge number of technically proficient developers, and a complex testcase scripting language would therefore make it difficult to leverage non-technical personnel in the testing process.

This is an eminently reasonable concern. After all, the fewer people there are in your organization who can understand the tests being performed, the more likely you are to find yourself saddled with a testing silo—which could potentially lead to slower bug fixes and poor alignment between testing and other functions.

The antidote to this potential pitfall is to choose an automation framework where testcase scripting is easy and straightforward. Even then, however, you might have some questions about how test scripting will work within the context of your automation solution.

This is totally reasonable—and it’s why today we’ll be taking a deeper look at testcase scripting in an automated telco testing environment, taking keyword-based testing as an example of a low-barrier scripting scenario.

Robot Framework for Testing

Now, first things first: what are keyword-based tests, and why are we advocating for them? Simply put, keyword-based tests allow you to execute tests from the command line or a graphical user interface using libraries of pre-defined keywords that can remotely control specific network elements.

Crucially, these keywords can be orchestrated together (i.e. they can interact and be combined into higher-abstraction keywords); the keyword libraries communicate directly with the system under test (SUT) to verify functionality and produce test reports.

One of the most popular frameworks for this kind of testing is Robot Framework, which describes itself as “a generic open source automation framework for acceptance testing, acceptance test driven development (ATDD), and robotic process automation (RPA).”

Though it has typically gotten wider play in the world of software testing, it obviously applies to telecom acceptance testing just as easily. In Robot Framework (and platforms that run on it), users organize different keywords into test suites, which they can then execute in order to verify functionality.

For example, a tester testing out login functionality for a mobile app would define her variables based on existing keywords, then enter the relevant commands into the command line.

From there, the test suite would be run automatically, and an HTML or XML-based test report would be returned, showing metadata about the test suite and a list of the relevant steps, e.g.:

  • Open browser at login page
  • Input username and password
  • Click submit
  • View account home page
  • Close browser

From there, you could drill down on the elements that passed and failed, all at the desired level of abstraction. For a relatively non-technical user, this means that you could still understand what had happened on the level of the different elements—while a technical user could find much more granular detail as needed.

Scripting Telecom Use Cases

Okay, but what happens when we apply this kind of framework to a telecom test environment? Well, things work essentially the same way. Testers can either create a library from scratch or use an existing library that’s already populated with relevant keywords, and from there it’s just a matter of organizing them into test suites and executing those suites as needed.

If, for instance, you were testing a call between a mobile subscriber and a PSTN user, you would define the A and B parties for the call based on specific variables.

You might have a keyword, “Initiate Device”, that’s then attached to a test phone variable, PSTN_A; and you would do the same for your mobile subscriber, MS_B. With the variables established, you would execute the test call by adding “Dial And Call” into the test script and running it.

The scripts aren’t just easy to use, they’re easy to reuse.

Crucially, these keywords and variables are all maintained in global files, such that testers can make necessary changes to individual keywords without having to laboriously go through the test library and update each mention of a given element manually.

This means that the scripts aren’t just easy to use, they’re easy to reuse. This, on its own, can act as a huge time saver. How? By radically reducing the time required for test maintenance. In the example in the previous section, for instance, adding a Captcha or a check box to the login workflow test (by changing the keywords accordingly) wouldn’t require a complete revision of all the relevant testcases and test suites.

Instead, the change would be applied across the relevant libraries automatically, meaning that testcases and test suites could be performed as usual even after a network change or other new development.

How Your Test Scripts Impact Reporting

One of the ways in which easy test scripting adds value is its ability to make the testing process accessible beyond its usual boundaries. But how valuable is that accessibility if it doesn’t extend to the aftercare that follows each round of testing?

The answer: not very. For this reason, reporting is an equally important consideration for telco testers seeking out a potential automation solution—and it’s a consideration that goes hand in hand with testcase scripting. Why? Because the two are directly linked.

Like we discussed above, in Robot Framework the keywords used to run tests are the same keywords that populate the reports, making them just as readable as the scripts themselves. This lays the foundation for highly readable reports that offer different levels of granularity as needed.

At the most abstract level, you should be able to see how many tests were run, how many tests failed, and what the overall testing time was. A layer beneath that, you can see the relevant information for each individual test case, including runtime and passage/failure.

Even further, a really sophisticated solution will provide you information about what’s happening on a protocol level to the system under test.Again, these will be based on the keywords established in your libraries, offering a level of consistency and visibility that would be hard to come by otherwise.

In this way, you reach root causes faster and keep the entire organization up-to-date on the state of your network.

 

Interested in our Products ?

Scroll Up

pencil Download your White Paper now

TECHNICAL PROJECT MANAGER

You will be a part of the SEGRON Project Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation.

As technical project manager, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases and you have to setup and align related project activities together with our customers.
You should have a strong knowledge and background in the telco environment paired with project management skills to align projects, track the related KPI’s further having a strong view also on the financial performance.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible
Team player:
• collaborates with others
• teach new skills to others and write tutorials
Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Several years of experience in Telecom Domain.
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• IMS/SDM/EPC domain knowledge
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack.
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge.

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus.
• Robot Framework / Python

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• PM Certification or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2500 EUR (depends on experience)

DEVELOPER TEST AUTOMATION

You will be a part of the SEGRON Test Automation Development Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation using Python, Robot.

As test automation developer, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases, building automation tools, framework, creating reusable python libraries for test automation.
You should have a strong knowledge and hands-on experience in test automation framework (Robot) and Python scripting language. Further you must continuously renew the automation & review guidelines set within the team. Ensure developed reusable keywords & test scripts meet the automation guideline.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible 


Team player:
• collaborates with others
• teach new skills to others and write tutorials


Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Experience in developing reusable test automation scripts using Python and Robot Framework
• Several years of experience in Telecom Domain
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• Test Case development
• Mobile development
• Design Patterns
• Web development/testing
• IMS/SDM/EPC domain knowledge
• Hand-on experience in Docker containers and Kubernetes.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2000 EUR (depends on experience)