Can Robots Speed Up End-to-End Testing?

software-testing-tools

End-to-end testing: for many telco operators it’s the holy grail of service verification, but it can also be a slow, laborious process that adversely impacts time to market. Even if you’ve managed to automate your relevant equipment and collect success and failure data from the relevant end-points.

You might still find yourself in a position where hard-to-read data and hard-to-program use cases stop your end-to-end tests from running as quickly as you would like. When this happens, you’re in the uncomfortable position of either sacrificing high levels of test coverage by cutting the test off early, or delaying your network migration or device rollout to accommodate slow testing.

How is it that even automated end-to-end tests can still be too time-consuming to power a positive ROI? Because not all automation frameworks are created equal—telco operators need a system that enables them to define their use cases quickly and extract reporting that can be immediately leveraged into bug fixes.

Automation, after all, isn’t a panacea. Rather, it’s a tool that has to be leveraged in context. So how do you make sure you’re getting the most out of your automation? Start with robotic software processes.

Challenges in End-to-End Testing

Let’s back up a step: one of the reasons that the automation needle is so hard to thread is that effective telecom testing has to walk a careful line between highly technical concepts and processes and broader operational accessibility.

Test engineers need to be able to use their skills efficiently and in way that suits their needs and workstyle, but testing can’t be completely opaque to less technical members of the team.

If your tests are a complete mystery to everyone except the lone test engineer working on a particular network or device rollout, you’ve essentially created a functional silo in which it’s impossible to build in oversight or gauge the need for increased resources. The result is often slower tests (due to insufficient resources) with less operational upside.

On the flip side, testing solutions that don’t let test engineers use their skills (because they’re too abstract or high-level) don’t lead to positive business outcomes either: engineers’ knowledge is underutilized during the tests themselves, and bug fixes that require more granular technical details become harder to address.

The happy medium here that some telco businesses have adopted is keyword-based (as opposed to data-driven) testing. In keyword-based testing, individual actions are given keywords, which can then theoretically be incorporated into command line operations to verify functionality.

Though the process requires some scripting ability, it also operates at a sufficiently abstract level that the code generated will usually be fairly readable—even for non-technical personnel. In this way, someone who couldn’t necessarily script or run an automated test can at least understand the results, meaning that the opacity and siloization that we described above is less likely to crop up.

Of course, automating keyword-based acceptance tests requires an automation solution with a strong technical underpinning. This is where the robots come in.

What Is Robot Framework?

Okay, when we say robots, what we really mean is Robot Framework, which describes itself on its website as “a generic open source automation framework for acceptance testing, acceptance test driven development (ATDD), and robotic process automation (RPA).”

It has a whole host of different test libraries for different use cases within different fields, with a modular architecture designed to support flexible use. Using specified keywords, those libraries communicate with the systems being tested.

The result is that reports and documentation based on test data are detailed enough that testers are able to draw technical conclusions, but abstract enough to be easy to operate and understand.

While the use of this framework is hardly limited to telecommunications, it was actually originally developed within Nokia, and as such it’s especially pertinent to telco testers. Why?

Because it makes it not just possible but relatively easy to power keyword-based tests that can be adapted to fairly minute specifications. If you’re performing end-to-end tests to verify device acceptance for a whole host of Android and iOS devices, this type of framework creates a clear directive for testers designing use cases:

  • Identify the starting state of the system under test (i.e. the state of a device straight out of the box)
  • Identify the set of actions (signing in, activating an account, etc.) that the user will take to reach the desired state.
  • Identify the state the system should reach after the actions have been taken (i.e. connected to your network).

Once the test cases are designed in this way, they can be executed systematically to produce the relevant documentation in HTML and/or XML.

Why Should Testers Care?

Okay, at this point you might be wondering, “how can this help speed up end-to-end tests within telecom businesses?” Essentially, whether you’re performing tests manually or utilizing some type of automation solution.

The ease of scripting test cases and understanding the results in this “robotic” environment helps to alleviate two of the most significant bottlenecks that slow down service verification: preparing the test environment, and fixing the bugs identified during the tests.

Automation will obviously speed up the actual execution of the tests even further, but the point is that you reach the execution stage more quickly, and then the results are likewise translated into improved quality of service more quickly.

This might not seem like much to get excited about, but the upshot is a positive feedback loop the drives continual process improvements. Because tests are easy to repeat and reconfigure, service verification flows get easier as you go.

By the same token, faster processes make regression testing easier and more feasible, which decreases the number of gaps you’re likely to find in your service the rest of the time. In this way, it’s not just an individual test that gets faster, but testing as a broad concept—leading to faster time-to-market on average.

Learn More About Test Automation

Click below to see how SEGRON’s ATF can help you automate testing.

Interested in our Products ?

Scroll Up

pencil Download your White Paper now

TECHNICAL PROJECT MANAGER

You will be a part of the SEGRON Project Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation.

As technical project manager, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases and you have to setup and align related project activities together with our customers.
You should have a strong knowledge and background in the telco environment paired with project management skills to align projects, track the related KPI’s further having a strong view also on the financial performance.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible
Team player:
• collaborates with others
• teach new skills to others and write tutorials
Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Several years of experience in Telecom Domain.
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• IMS/SDM/EPC domain knowledge
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack.
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge.

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus.
• Robot Framework / Python

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• PM Certification or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2500 EUR (depends on experience)

DEVELOPER TEST AUTOMATION

You will be a part of the SEGRON Test Automation Development Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation using Python, Robot.

As test automation developer, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases, building automation tools, framework, creating reusable python libraries for test automation.
You should have a strong knowledge and hands-on experience in test automation framework (Robot) and Python scripting language. Further you must continuously renew the automation & review guidelines set within the team. Ensure developed reusable keywords & test scripts meet the automation guideline.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible 


Team player:
• collaborates with others
• teach new skills to others and write tutorials


Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Experience in developing reusable test automation scripts using Python and Robot Framework
• Several years of experience in Telecom Domain
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• Test Case development
• Mobile development
• Design Patterns
• Web development/testing
• IMS/SDM/EPC domain knowledge
• Hand-on experience in Docker containers and Kubernetes.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2000 EUR (depends on experience)