Can Robots Speed Up End-to-End Testing?

End-to-end testing: for many telco operators it’s the holy grail of service verification, but it can also be a slow, laborious process that adversely impacts time to market. Even if you’ve managed to automate your relevant equipment and collect success and failure data from the relevant end-points.

You might still find yourself in a position where hard-to-read data and hard-to-program use cases stop your end-to-end tests from running as quickly as you would like. When this happens, you’re in the uncomfortable position of either sacrificing high levels of test coverage by cutting the test off early, or delaying your network migration or device rollout to accommodate slow testing.

How is it that even automated end-to-end tests can still be too time-consuming to power a positive ROI? Because not all automation frameworks are created equal—telco operators need a system that enables them to define their use cases quickly and extract reporting that can be immediately leveraged into bug fixes.

Automation, after all, isn’t a panacea. Rather, it’s a tool that has to be leveraged in context. So how do you make sure you’re getting the most out of your automation? Start with robotic software processes.

Challenges in End-to-End Testing

Let’s back up a step: one of the reasons that the automation needle is so hard to thread is that effective telecom testing has to walk a careful line between highly technical concepts and processes and broader operational accessibility.

Test engineers need to be able to use their skills efficiently and in way that suits their needs and workstyle, but testing can’t be completely opaque to less technical members of the team.

If your tests are a complete mystery to everyone except the lone test engineer working on a particular network or device rollout, you’ve essentially created a functional silo in which it’s impossible to build in oversight or gauge the need for increased resources. The result is often slower tests (due to insufficient resources) with less operational upside.

On the flip side, testing solutions that don’t let test engineers use their skills (because they’re too abstract or high-level) don’t lead to positive business outcomes either: engineers’ knowledge is underutilized during the tests themselves, and bug fixes that require more granular technical details become harder to address.

The happy medium here that some telco businesses have adopted is keyword-based (as opposed to data-driven) testing. In keyword-based testing, individual actions are given keywords, which can then theoretically be incorporated into command line operations to verify functionality.

Though the process requires some scripting ability, it also operates at a sufficiently abstract level that the code generated will usually be fairly readable—even for non-technical personnel. In this way, someone who couldn’t necessarily script or run an automated test can at least understand the results, meaning that the opacity and siloization that we described above is less likely to crop up.

Of course, automating keyword-based acceptance tests requires an automation solution with a strong technical underpinning. This is where the robots come in.

What Is Robot Framework?

Okay, when we say robots, what we really mean is Robot Framework, which describes itself on its website as “a generic open source automation framework for acceptance testing, acceptance test driven development (ATDD), and robotic process automation (RPA).”

It has a whole host of different test libraries for different use cases within different fields, with a modular architecture designed to support flexible use. Using specified keywords, those libraries communicate with the systems being tested.

The result is that reports and documentation based on test data are detailed enough that testers are able to draw technical conclusions, but abstract enough to be easy to operate and understand.

While the use of this framework is hardly limited to telecommunications, it was actually originally developed within Nokia, and as such it’s especially pertinent to telco testers. Why?

Because it makes it not just possible but relatively easy to power keyword-based tests that can be adapted to fairly minute specifications. If you’re performing end-to-end tests to verify device acceptance for a whole host of Android and iOS devices, this type of framework creates a clear directive for testers designing use cases:

• Identify the starting state of the system under test (i.e. the state of a device straight out of the box)
• Identify the set of actions (signing in, activating an account, etc.) that the user will take to reach the desired state.
• Identify the state the system should reach after the actions have been taken (i.e. connected to your network).

Once the test cases are designed in this way, they can be executed systematically to produce the relevant documentation in HTML and/or XML.

Why Should Testers Care?

Okay, at this point you might be wondering, “how can this help speed up end-to-end tests within telecom businesses?” Essentially, whether you’re performing tests manually or utilizing some type of automation solution.

The ease of scripting test cases and understanding the results in this “robotic” environment helps to alleviate two of the most significant bottlenecks that slow down service verification: preparing the test environment, and fixing the bugs identified during the tests.

Automation will obviously speed up the actual execution of the tests even further, but the point is that you reach the execution stage more quickly, and then the results are likewise translated into improved quality of service more quickly.

This might not seem like much to get excited about, but the upshot is a positive feedback loop the drives continual process improvements. Because tests are easy to repeat and reconfigure, service verification flows get easier as you go.

By the same token, faster processes make regression testing easier and more feasible, which decreases the number of gaps you’re likely to find in your service the rest of the time. In this way, it’s not just an individual test that gets faster, but testing as a broad concept—leading to faster time-to-market on average.

Learn More About Test Automation

Click below to see how SEGRON’s ATF can help you automate testing.