How Automation Can Impact Testing ROI in 2019

The most obvious benefits of automation for any industry include increased efficiency and decreased reliance on human employees. But for telcos, automation, and particularly automated testing, offers multiple other sources of ROI, from reduced time to market, to better implementation of the Continuous Delivery model.

Shorter Time to Market

Decreasing time to market has always been critical for telcos to remain competitive; releasing new products faster than competitors has always offered an advantage. Meanwhile, upgrade cycles have gotten progressively shorter in recent years, placing even more pressure on testing teams to keep up. Automated testing contributes to shorter time to market both directly and indirectly:

• Automated testing simply takes less time than manual testing, immediately facilitating shorter time to market. 
• Automation streamlines daily regression testing, which provides a strong foundation for future development initiatives and ongoing service verification. Thus issues with existing service can be diagnosed and addressed sooner, before they crop up as problems with new offerings.
• Automated testing actually helps you create more testable products, so you can test in more ways, with less chance of missing use cases. These both contribute to shorter time to market. 

Fewer Resources Required for Testing

In a manual testing environment, one testing engineer can handle about six use cases per day. That number skyrockets with the addition of automation–in some cases, one engineer can oversee hundreds of tests per day. In many cases, the introduction of automation helps significantly reduce the human resources necessary to complete testing.

Furthermore, the testing team can use their time for more revenue generating activities. For example, they will have more time to test new products, or to address glitches that impact customer service. Thus the ROI here is twofold: fewer employees are required, and those employees can do more meaningful work.

One caveat when determining ROI: Time will still be required for manual testing. Not everything will be automated, and the testing team will also have to address the issues uncovered by manual testing. It’s important to accurately, honestly estimate what percentage of testing will be automated and the corresponding time savings.

Decreased Defect Costs

The costs associated with identifying and correcting defects can be difficult to track, especially because there are often delays between development and testing. Manual testing is often the source of the bottleneck–due to backlogs in testing, the engineering team often cannot start testing as soon as development is complete. Thus engineers frequently spend additional time “relearning” code they wrote weeks or months before so that they can fix newly discovered bugs.

Automated testing protocols can be executed as soon as development is complete, effectively shortening the feedback loop. Engineers can get almost immediate feedback on new products. They don’t have to waste time reacquainting themselves with code long ago written.

A lag between development and testing can also mean that engineers don’t learn about flawed code soon enough to refrain from reusing that code in subsequently developed products. Thus more time is spent detecting and fixing the same issues multiple times. Automated testing significantly reduces the time required to remove the same bug from multiple products.

Support of the Continuous Delivery Model

The software industry has long used test automation to enable agile development and the Continuous Delivery model. Leading telcos have also adopted this approach for product development because automated testing offers repeatable checks for acceptance, performance, and regression scenarios.

These checks not only reduce bugs and support faster delivery of new applications, but they also contribute to improved customer service.

At the strategic level, automated testing provides consistent, accurate information about high-value features and associated risks, so that business leaders can make better choices. For example, the results of automated testing can help define the necessary development and processing environment, or determine when new updates should be released.

How to Maximize the ROI of Automated Testing

Automation is often considered a “silver bullet” for operational inefficiencies. It’s important to remember that introducing automation requires an investment in both time and money. These best practices can help ensure the success of testing automation and maximize the ROI of automation.

• Determine the project scope.The most successful automated tests are those that are extremely repetitive or require frequent human intervention. That’s why regression testing is often automated. These tests are ideal candidates for automation. But other factors, such as hardware dependencies, can impact the success of automation. Limit the list of tests to automate to those that fit your criteria.

• Don’t neglect test design. The purpose of test automation is to maximize test coverage with the fewest number of tests, and in the shortest time frame possible. Before the first automation script is ever written, perform a comprehensive review of all functional and non-functional requirements. This step can actually help identify previously overlooked workflows or test scenarios that could be automated. 

• Pay attention to data quality. Quality test data is the foundation for successful automation. You’ll need comprehensive, accurate data on an ongoing basis, so review data regularly to ensure that it is still valid.

• Remember that products evolve. Conventional monolithic testing architecture doesn’t support the frequent product updates that today’s telco environment requires. Strive to build automated test suites that facilitate frequent change and include reusable test cases. 
• Implement a robust automation framework. Simply put, a framework is a set of predefined testing standards and protocols that reduce risk and maximize test speed and accuracy, while also decreasing costs associated with test maintenance. A proper framework also supports Continuous Delivery.