How (and Why) You Should Test on Flagship Devices

Mobile_Testing_Segron

According to Apple, in the last quarter of 2019, the recently-released iPhone 11, iPhone 11 Pro, and iPhone 11 Pro Max made up nearly 70% of all American iPhone sales. It’s often been remarked that any data Apple happens to share should be taken with a grain of salt (they don’t release information about how many units they actually move), but even if this trend is exaggerated, it’s still fairly telling. In the US at least, mobile users are going out of their way to seek out the latest and greatest smartphones—and the odds are pretty high that some of those mobile users are subscribers on your network.

So, imagine their chagrin when they activate their new phone—excited to use its OLED screen and triple camera system—only to find themselves mired in network connectivity issues. iPhones are such high-profile devices that there would be an immediate outcry, but with new phones from other manufacturers, it might take a while for the issues to come to light. By that time, you’ve already lost the support of many of these users.  

 

How Out-of-the-Box Smartphones Impact

The situation we outlined above might seem grim, but as luck would have it, there are proven ways to minimize that. Step one: test on the devices that users are utilizing to connect to your network. This sounds simple enough, but testers are often tempted to go the other way due to time pressure or perceived ease of automation. Usually, this takes the form of testers making use of devices that have been subject to rooting or jailbreaking. Considering the increasing complexity of the average telco network, it’s easy to imagine why someone would try to take a shortcut by mainly hacking devices to install testing software, but the result is tests that are less reliable than tests performed on devices out-of-the-box. Why? Because by making changes to the device iOS, you’re changing it just enough that it’s no longer a representative sample of other devices of its make, model, and firmware version. As a result, tiny differences in functionality can begin to add up to significant discrepancies.

Luckily, you don’t need to root or jailbreak your devices to automate testing. Instead, you can connect out-of-the-box devices—identical to the ones your subscribers are using—to a test server that primarily uses robotic process automation to dial numbers, perform calls, send SMS messages, and track the results in terms of both passage/failure of tests and protocol-level activity. These devices slot into automation flows just as seamlessly (we would often argue more seamlessly) then their rooted counterparts, but they don’t present the same downsides. Instead, they provide you with the closest possible recreation of real network usage conditions. 

 

The Importance of Devices

When it comes to testing on flagship devices, in particular, the rationale is essentially the same as the reasoning that was presented above. If your users are flocking to flagship devices as they’re released, they’re going to expect high network quality once they connect with their new devices. If you haven’t tested using these same devices, there’s a chance that something can go wrong, and that they won’t be able to place calls, use data, or otherwise use their phones as they expect to. Even when it comes to less high profile devices, that desire is still going to be there, and without proper service verification, as new devices enter the market, you’re going to run into issues. As technology changes and evolves, this will only become more true. To wit, the first generation of proper 5G-enabled phones has been trickling into the market for about a year now—and they’re equipped with different antennae and radio hardware than the prior generation of smartphones, such that they can connect to the new spectrums designated for millimeter-wave use. This presents a challenge for testers since there’s no realistic way to replicate these changes without using the devices themselves.

And this line of reasoning doesn’t just extend to phones. As we enter the IoT (internet of things) era, the number of cutting edge connected devices is increasing every year—putting considerable pressure on telco testers. Because IoT use cases can often be extremely high leverage (think of the eCall modem in a modern EU-based car that facilitates emergency calls in the case of an accident), the stakes are high, and it’s essential to have workflows in place, where you can test on the latest devices as efficiently as possible. Otherwise, your test lab runs the risk of missing important issues that could impact your subscribers. 

Flagship Devices

As we said above, telco testers often look to rooting and jailbreaking to incorporate devices more easily into automation flows. After all, if you can’t test flagship devices within the context of test automation, there’s little hope of maintaining high test coverage or throughput. Thus, you might still be thinking of rooting as a last resort for testing on the absolute latest phones. But does rooting really speed up the process? Not necessarily. In SEGRON’s ATF (Automated Test Framework), it’s actually possible to incorporate new devices into the framework within a matter of hours. In this way, testers can incorporate the latest devices into their test lab operations without a hitch.

As you can probably guess from the preceding, once they’ve been automated in this way, there’s no need to root them. Instead, you can simply orchestrate tests, as usual, no doubt putting extra emphasis on conformance tests to ensure that any new firmware specifications play nicely with your network. Not only does this put you on a path to higher test coverage, but it also puts you in a position to address your subscribers’ needs proactively. With any luck, this can give you a competitive advantage over other operators in your area who might not be verifying functionality for the flagship iOS or Android devices that are just coming onto the market.

 

Download our latest eBook and Learn more about how Test Automation can speed up Time-to-Market, improve CX and more.

Interested in our Products ?

Scroll Up

pencil Download your White Paper now

TECHNICAL PROJECT MANAGER

You will be a part of the SEGRON Project Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation.

As technical project manager, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases and you have to setup and align related project activities together with our customers.
You should have a strong knowledge and background in the telco environment paired with project management skills to align projects, track the related KPI’s further having a strong view also on the financial performance.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible
Team player:
• collaborates with others
• teach new skills to others and write tutorials
Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Several years of experience in Telecom Domain.
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• IMS/SDM/EPC domain knowledge
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack.
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge.

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus.
• Robot Framework / Python

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• PM Certification or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2500 EUR (depends on experience)

DEVELOPER TEST AUTOMATION

You will be a part of the SEGRON Test Automation Development Team. This will be an amazing opportunity for someone to be part of a team working in next generation cutting edge technologies such as 5G, IOT, DevOps, CI/CD and test automation using Python, Robot.

As test automation developer, you will be responsible for understanding the different customer requirements and latest technologies around 5G, VoLTE use cases, building automation tools, framework, creating reusable python libraries for test automation.
You should have a strong knowledge and hands-on experience in test automation framework (Robot) and Python scripting language. Further you must continuously renew the automation & review guidelines set within the team. Ensure developed reusable keywords & test scripts meet the automation guideline.

Soft Skills:
Problem Solving skills:
• Analytical & technical mindset
• Good in abstraction i.e. seeing the “big picture”
• Self-Motivated & Responsible 


Team player:
• collaborates with others
• teach new skills to others and write tutorials


Others:
• Good Communication Skills
• Flexible/Entrepreneurial

Hard Skills:
• Experience in developing reusable test automation scripts using Python and Robot Framework
• Several years of experience in Telecom Domain
• Good expertise on 5G/4G/3G technologies and should be familiar with the network call flows
• Test Case development
• Mobile development
• Design Patterns
• Web development/testing
• IMS/SDM/EPC domain knowledge
• Hand-on experience in Docker containers and Kubernetes.
• Sound understanding of Telco Cloud technologies such as VNFI, Open Stack
• Sound knowledge in Linux/CentOS, TCP/IP stack
• DevOps knowledge

Nice to have skills:
• Understanding of 5G network elements and interfaces is plus
• Wireshark packet analysis
• Experience in Protocol testing including HTTP2, SIP, MAP, Diameter

Specific additional information:
• You have a Telecommunications/Electronics/SW/Computer master’s degree or equivalent through experience
• Very high English proficient

Others:
• Full time job
• Variety of financial and non-financial benefits
• Offered salary: from 2000 EUR (depends on experience)