Why Is Beyond End-to-end Testing for IoT Devices Important?

It’s become a fairly well established stereotype that people in many parts of the world can’t or won’t look up from their smartphones. Back in the day, the only reason to look at or think about your telephone was because it was ringing or because you intended to make a call fairly imminently.

As a consequence, if a signaling error was preventing your home phone from completing calls as intended, you might not even notice for a few days. Nowadays, on the other hand, if your LTE service is interrupted for two or three minutes you’ll probably notice immediately—and you’ll be none too happy about it.

Needless to say, this puts a lot of pressure on telco operators to minimize network downtime, lest they face the wrath of their subscribers. But that’s nothing compared to the pressure that the increasing proliferation of the internet of things (IoT) is going to place on those same networks.

These devices are increasingly present in our smart factories, homes, and automobiles, and they’re based on a principal of constant communication. If your smart oven loses connectivity while it’s in use, for instance, it immediately presents a real fire hazard.

If your automobile’s eCall service cuts out, it could be a matter of life and death. As a result, device testers in this emerging field have their work cut out for them.

Device/Protocol Diversity

Because service verification standards in the world of the IoT are so stringent, device testers need to tailor their efforts to the specific challenges that the IoT presents. Chief among these challenges?

The tremendous amount of hardware and software diversity among these devices. Mobile phones have been trending towards market fragmentation for a while now, with more unique Android and iOS phone models coming out every year—but they’ve got nothing on the IoT.

Yes, 3GPP has set its NB-IoT standard for how these devices should interconnect with LTE, but that doesn’t mean the interplay between each network element has been entirely standardized.

The devices themselves vary significantly, and a wide variety of different platforms, apps, and GUIs all have to work together to create appropriate service even after connections have been established to the application server and EPC framework.

And while diversity and interoperability are critical concerns for testers, they’re also not the only factors adding complexity to testing for IoT devices. There’s also the sheer number of communication protocols and design layers that go into even a straightforward-seeming device.

Depending on the device and the device maker, you might be looking at Constrained Application Protocol (CoAP), Extensible Messaging and Presence Protocol (XMPP), Message Queuing Telemetry Transport (MQTT), or other unique protocols that have vastly different standards for conformance.

Not only does this raise the bar for the amount of domain knowledge required to test a variety of devices effectively, it also increases the potential number of test scenarios even further.

Beyond End-to-end

All of these moving parts mean that while a test case that measure the success or failure of a particular action might not tell you everything you need to know. If a car’s emergency call service connects to the wrong call center or doesn’t connect at all, that doesn’t automatically tell you the layer at which the issue occurred.

This is where signalling protocol analysis and access to the network elements under test come into play. Though these aren’t traditionally within the scope of end-to-end test flows, in the case of IoT devices they may prove particularly impactful for turning bugs found into bugs fixed.

Thus, where an end-to-end test might require access to the various endpoints—whether that’s smartphones or application servers—an IoT test flow that went beyond end-to-end would include signal analyses of the sort we discussed above that could point testers to the root causes of service gaps on a much more granular level.

If a test case involved a user using an Android smartphone to ask a smart refrigerator if she was out of eggs, for instance, a test that went beyond end-to-end might give a clearer suggestion of whether the test failed because the IoT device lost its LTE connection or if the API malfunctioned.

This adds value for engineers when it comes time to address the test results—but it does potentially make the tests more complicated, which can slow things down in test environments that already don’t allow manual testers to power through more than half a dozen or so use cases per day.

Constant Communication

Okay, now we’re going to move the goalposts a little bit farther. We mentioned in the opening paragraph that IoT devices function based on a principal of constant communication—but what implications does this have for testers, exactly?

For one thing, it greatly increases the importance of regression testing. Especially when it comes to things like emergency calling protocols for automobile accidents, it’s not enough to do a “one and done” round of testing—instead, you need to follow up on a consistent basis to ensure that everything is still working properly.

The trouble here is that testing these devices ahead of a release is already going to require a huge outlay of resources, for exactly the reasons outlined above. The more complex these test cases become, the more time and effort they require of testers.

Once you add 24/7 regression testing to the mix, the engineer resources required become even more considerable. For this reason, the IoT may be one of the first areas to emphatically embrace automated testing. Automated testing in this context would, of course, mean orchestrating ongoing tests using the IoT devices themselves and other network elements.

Testers would use a repeatable framework that could be adjusted to meet the needs of different test cases, helping to reduce the need for engineers to devote huge swaths of time to never-ending tasks like regression tests. In this way, automated tests could potentially increase the number of test cases verified per day by a huge percentage.

Considering the proliferation of different potential use cases in the IoT realm, improved testing velocity and test coverage are nothing to shy away from.

Search

Interested in our Products ?

Scroll to Top
Segron logo black blue

Senior SaaS System Administrator

Technical Skills :
  • Oversee the sysadmin related tasks in our SaaS infrastructure (partially cloud based, partially bare metal)
  • Daily operation and maintenance of the system
  • Analysing and resolving incidents
  • Follow and help improving the incident and change management procedures
  • Design procedures for system troubleshooting and maintenance
  • Incorporating base OS updates and security patches
  • Ensure that systems are safe and secure against cybersecurity threats by raising change requests where potential threat is possible
  • Performing SW updates for the Segron SaaS SW stack (distributed architecture with clusters)
  • Configuring solutions like reverse proxy, firewalls, etc.
  • Building tools to automate procedures & reduce occurrences of errors and improve customer experience
  • Tutoring & coaching newcomers & less senior experts in the team
  • Interworking with the architects and IT admins of Segron to have the SaaS procedures inline with the Segron processes
Non-technical skills:
  • We are looking for a self-motivated, self-improving individual with a highly independent mindset and open and straightforward technical communication to help us to improve and maintain our cloud infrastructure of our powerful end-to-end testing solution ATF (Automated Testing Framework)
  • 3+ years hands-on experience with operation and monitoring of cloud / linux systems
  • 3+ years of hands-on experience with network devops elements: configuring routers, switches, networks
  • Hands-on experience with running live systems with infrastructure as a code mode of operation
  • Specific knowledge which brings direct advantage: Docker, Docker Compose, Grafana, Prometheus, Ansible, Debian Linux OS administration, Security
  • Experience in building and maintaining distributed systems (incl. redundancy, resiliency, load-balancing) is welcome
  • Excellent knowledge of English
Location :
  • Place of work: Bratislava (partially home office possible)
  • Rate: from 30 EUR/hour (possible higher rate, depends on experience)
Segron logo - The Next Generation of Active Testing
Segron logo black blue

CI/CD Senior Developer

Technical Skills :
  • A senior role with a proven expertise in software development, cloud computing, DevOps, and CI/CD
  • Experience in planning, designing, and overseeing the CI/CD strategy and architecture on the level of organization
  • Ability to tailor testing strategies which define and follow the best practices, standards, and policies for the software delivery process
  • Hands-on experience in creating and managing CI/CD pipelines and workflows (PaaC)
  • Ability to evaluate and recommend the best tools, technologies, and methodologies for the CI/CD implementation
  • Prior hands-on experience working with different CI/CD toolsets (Jenkins, Bitbucket, GitLab, artifactory, Ansible ..)
  • Proficient with DevOps tools API automation capabilities
  • Proficient with Atlassian Tools (BitBucket, Jira, Confluence) and agile SW development methodologies
  • Familiar with cloud patterns and best practices
  • Familiar with web performance best practices
  • Comfortable working in cloud DevOps ecosystem
  • Comfortable working with Linux platforms
  • Initial working experience in SW development is an advantage.
Non-technical skills:
  • Effective communication with technical as well and business stakeholders
  • Self-motivating, self-improving mindset
  • Ownership of relevant industry certificates is a plus
Location :
  • Location: Bratislava, Slovakia (with hybrid flexibility)
  • Rate: from 30 EUR/hour (possible higher rate, depends on experience)
Segron logo - The Next Generation of Active Testing
Segron logo black blue

Test Automation Engineer

Job description, responsibilities:

Hardware Testing: Conduct verification and testing of hardware during the HW/SW products
assembly process.
Technical Support: Provide technical assistance during testing activities and troubleshoot any
hardware issues that arise. Support to internal teams and customers by managing priorities and
meeting deadlines.
Equipment Maintenance: Configure and maintain SEGRON laboratory and customer equipment.
Logistics Support: Handle hardware ordering and logistics.
Troubleshooting: Analyze and resolve SEGRON product issues through effective problem-solving.
Ability to troubleshoot and resolve issues that arise during the integration process.
Collaboration: Work with the SEGRON Development Team on product or software issues and assist
the Project Team with planned activities.
Root Cause Analysis: Investigate production errors to identify underlying causes.
Software Deployment: Implement software updates and fixes.
Team Collaboration: Ability to work collaboratively in a team environment, especially in serving global
customers.

Requirements/ Skills:

Networking knowledge: Understanding TCP/IP networks and LAN/WAN configurations. Routing and
switching and related protocols strongly desired.
Hardware Proficiency: Understanding of hardware components, including servers, routers, and
peripherals, to ensure proper assembly and integration. Experience in data centers is a plus.
Travel Abilities: Willing to travel for customer on-site support for hardware installations, migrations,
upgrades, troubleshooting, etc. (few times a year, 3-5 days)
Hand Tools: Proficiency in using hand tools like screwdrivers, pliers, crimpers, and punches for
assembling and disassembling components.
Operating Systems: Knowledge of Linux and Windows. VM experience (Hyper-V, VMWare, Oracle
Virtual Box, Proxmox etc.)
Tools Proficiency: Basic understanding of Ansible.
Telecommunications Experience: Previous experience in the telecom sector is a plus.
Education: Degree in Computer Science/Engineering or equivalent work experience.
Experience: At least 2 years of relevant work experience preferred. A strong desire to continuously
improve skills and knowledge in hardware and software integration, with an openness to feedback and
new ideas. Key is openminded person willing to learn fast in great team.
Communication Skills: Effective oral and written Eng

Others:

• Full time job (employment)
• Onsite work during onboarding period, then it can be 3 days onsite, 2 days home office
Offered salary: from 2000 Euro (depends on seniority and skills level)
Variety of financial benefits
Place of work: Bratislava

Segron logo - The Next Generation of Active Testing
Segron logo black blue

Senior Python Developer

Technical Skills :

  • A solid, experienced SW developer with at least 10 years of experience in active SW development in different programming paradigms
  • Minimum 5 years of professional Python development experience
  • Master or college degree from Computer Science, Mathematics or STEM domain
  • Well educated in design and programming patterns that increase software’s efficiency and readability.
  • Very good analytical and problem solving skills.
  • At least three skills out of the following 4 skills are requested:
    • Microservices based architectures (Docker containers)
    • Linux
    • Ansible
    • Robot  Framework
  • Comfortable with sysadmin and DevOps skills (Ansible, YML files, Network Programming, IP protocols, designing and developing proxy servers for different protocols – example: streaming, integrating and compiling third party libraries on Linux (Debian))
  • Proficient with Atlassian Tools (BitBucket, Jira, Confluence) thorough understanding of Git and version control best practices
  • Familiar with cloud patterns and best practices
  • Familiar with web performance best practices

Non-technical skills:

  • Ability to work under pressure
  • Ability to abstract and explain your work
  • Strong understanding of Agile development process and experience working in an agile team
  • Strong communication skills with both technical and non-technical stakeholders
Location :
  • Bratislava, Slovakia (with hybrid flexibility)
  • Rate: from 35 EUR/hour (possible higher rate, but depends on experience)
Segron logo - The Next Generation of Active Testing