Frequently Asked Questions About SEGRON’s Automated Testing Framework
- Reading Time: 5 minutes
Making the transition from manual to automated tests can be daunting—particularly in the telecom domain, where so much specialized knowledge is required for almost every aspect of a tester’s job.
At SEGRON, we know first-hand that anyone who’s thinking about adopting a new testing framework is going to have a lot of questions. Luckily, when it comes to our own ATF (Automated Testing Framework), we’ve got answers.
Below, you’ll find a sampling of frequently asked questions about our testing framework. Not seeing your question on the list? Feel free to contact us through this link, and we’ll be happy to walk you through any concerns you might have.
Question: How complicated is the test scripting process? Can we write test scripts ourselves?
Answer: Our ATF is designed to be as user-friendly as possible, which means it is incredibly simple to script up test cases yourself. Test cases are written in a keyword-based scripting language, so users with no previous experience can write basic tests after a tiny bit of training. This low-code approach to test scripting makes the full range of SEGRON’s ATF functionality easily accessible across the entire organization.
You can learn more about test case scripting here.
Question: What do ATF reports look like? How can we use them to evaluate test results and uncover root causes?
Answer: Every time the ATF executes tests, it automatically produces a report referencing the same keywords from the test scripts. At a high level, you can see which tests were run, how long they took, and if they passed or failed.
From there you can drill down into more granular information, including protocol-level analysis in order to find the root causes. The ATF aggregates reports and creates easy-to-understand charts and statistics. These tools can identify problems that will help optimize networks and improve services to your customers.
All reports can be accessed from within your ATF dashboard, and because they’re keyword-based, it’s easy for anyone in your organization to access, read, and understand them.
Learn more about reporting here.
Question: What about notifications? Can we implement automated system notifications for specific failure scenarios?
Answer: Yes, you can absolutely receive automatic notifications in order to respond to incidents and failures as quickly as possible. Notifications can also be integrated in the network monitoring system as needed.
Question: Does ATF require rooted Android devices or iOS devices that have been jailbroken in order to run tests on them?
Answer: ABSOLUTELY NOT! SEGRON’s ATF orchestrates testing using only out-of-the-box devices—exactly the devices that your subscribers will be using to access your network. In doing so, we avoid all of the risks that come from testing on rooted devices: security issues, loss of device integrity, and inaccurate test results.
The latest Android and iOS devices can easily be added to the test environment because most device configuration steps are automated in the ATF.
Click here to learn more about the risks of testing on rooted or jailbroken devices.
Question: Okay, but what about newly released flagship devices? How long does it take to incorporate them into the framework?
Answer: Integration of a new device can be done in a matter of minutes. All it takes is a subscription to SEGRON iDA (our AI based device automation product), which discovers and integrates most devices in about 10 minutes.
Question: What other devices can be incorporated into the test flows? Can you orchestrate testing with tools like load generators and simulators? What about fixed line (PSTN) phones?
Answer: You bet! Existing testing tools and databases with an API can be easily integrated to the ATF to widen the coverage of testing. SEGRON’S ATF has feature-rich keyword libraries supporting a wide range of devices.
Testing can be automated for fixed line phones, fax connections, and IP phones, in addition to a large variety of mobile phones. We can even do phones with custom firmware, such as specialized Android versions. This can obviously be crucial for testing out legacy interworking and other similar functionality.
Question: What about apps? Can you automate Facebook Messenger, WhatsApp, and other apps that subscribers use frequently?
Answer: Yes, we can. We already support the most commonly used apps. It’s basically the same logic as with devices—if it’s got an API, we can incorporate it into an ATF environment as needed.
Question: Can the ATF automate mobility-related use cases like SRVCC, handovers, etc.?
Answer: Indeed, it can. The ATF simulates a waning LTE signal to force a fallback or handover by controlling both the relevant end-user devices and digital attenuators. This effectively reproduces the conditions that subscribers experience when they run into LTE dead zones while moving around your network (or testers doing the same). In this way, you can cut out the lengthy drive tests that would otherwise slow down service verification and adversely impact time-to-market.
Learn more about SRVCC and mobility management here and here.
Question: Can the ATF handle devices spread across multiple locations?
Answer: Yes, our ATF can orchestrate tests from a central location and easily manage devices in different locations and even other countries. This can be especially useful functionality when it comes to verifying roaming service.
Question: Can you analyse signalling traces with the ATF?
Answer: Yes, SEGRON’s ATF can capture signalling traces in order to provide protocol-level information to testers about the system under test. This is the basis of the “Beyond End2End®” functionality of SEGRON’s ATF. In this way, testers can verify the actual protocols on a message level sent by their phones or network elements and check the results against what the signalling data should have revealed.
Ultimately, you get a much more accurate, in-depth picture of what’s happening on your network, and when bugs arise, you can find the root causes that much more quickly.
Learn more about going Beyond End2End® here.
Question: Can your system execute tests 24/7?
Answer: Unlike human testers, the ATF never gets tired, never needs a break, and never deviates from the established way of doing things. This means that all test suites are conducted on the timeline that you specify, whether that means recurring executions every hour, performing regression tests at night while the system isn’t being used by engineers, or around the clock network monitoring. Not only does this potentially speed up time-to-market, but it can also help boost your testing ROI through increased flexibility and improved time efficiency.
Still have questions? Interested in taking SEGRON’s ATF for a test drive? Follow this link to schedule a demo.
Categories
Tags
Thomas Groissenberger
Thomas has more than 25 years of experience in software, testing and management in Europe and the US. He is the Founder and CEO at SEGRON and he is a member of the Forbes Technology an Invitation-Only Community for World-Class CIOs, CTOs, and Technology Executives.
Recent Posts
Interested in our Products ?
