segron logo

How Telco Test Automation Impacts User Experience

If you’re a smartphone manufacturer, you’ve likely encountered this issue: you’ve been working on a significant upgrade to your operating system, and you finally hit the ‘go’ button and push it out to your users. Most of your users get a notification asking them to update, they do so, and everything is fine. But some of your users try to install the update only to find that it crashes their system or has other unwanted consequences for their ability to use their phones. Some users, fearing precisely that scenario, likely won’t download the updates at all—potentially leaving themselves open to security vulnerabilities and quality of service (QoS) issues. 

 This situation certainly isn’t uncommon, but it’s a problematic state of affairs for telco businesses. As more test engineers in the telco space turn to tactics like CICD (continuous integration/continuous delivery—a testing tactic in which small updates are being pushed, built, tested, and delivered on a daily basis). However, the cycle of significant updates with patchwork adoption begins to change. Thus, instead of a mishmash of slightly different OS specifications across your users—resulting in security, compliance, and QoS issues—most users get small, periodic updates that don’t break anything. 

This is just one example of how a particular testing tactic can have a direct, immediate impact on user experience. Though your test flows might feel a little abstracted from users’ realities on our devices or networks, it’s becoming clear that what you do on the test bench can reach your users in the form of something other than fluctuating jitter latency and packet loss.   

As a baseline, you can start by automating tests as generally as possible using out-of-the-box smartphones, i.e., phones that haven’t been rooted or jailbroken. This puts you in a position to replicate the conditions under which your users will actually be operating your phones or accessing your network, such that you can be sure that you’re seeing exactly what they are. 

What Impacts User Experience on Telco Networks?

Like we said above, most testers think of things like jitter and packet loss that can be directly measured and tested for as the most apparent factors impacting user experience. And, to some extent, that’s true: without a baseline level of network quality that allows users to make calls and use data without frustration, there’s little point in talking about the rest of it. At the same time, however, user experience goes beyond just those metrics. This is why some businesses have started tracking the quality of experience (QoE) in addition to just QoS (Quality of Service). This makes it possible to examine more intangible factors that might make or break the user experience. 

When it comes to QoE, in addition to things like latency times and outages, users care a lot about how long it takes for issues (whether that’s their particular issues—the ones that lead them to call your customer service hotline—or systemwide problems) to get resolved. It might also include ease of use for anything that interacts with your backend OSS/BSS systems, like changing account settings or paying bills. This, again, could put testers in somewhat new territory, pushing their focus towards provisioning, BSS integrations, and specific UX questions in a way that wasn’t common even a few years ago. Integrating automated testing of web-based user portals and provisioning systems gives a clearer picture of how long actions take. 

The Power of Test Automation

From there, you can begin to get more sophisticated: with a testing framework that can automatically run tests on your OSS/BSS systems, you can start to create alignment between those systems and your QoE goals. For instance, by including things like invoicing and billing in your standard suite of regression tests. If you have a system capable of going “beyond end-to-end,” you can also start to test things like audio quality and speech path connections. For an area like this, there are, among others, three prominent use cases: 

  •   Tone verification 
  •   Announcements identification 
  •   Speech path testing 

Testing here might involve some extra elements (like an audio matrix and an isolator). It could include injecting audio samples into one side of the connection and recording the audio output on the other side of the connection. In that way, you’re able to expend resources on optimizing things that your customers will experience in a fairly direct way. 

For audio QoE measurements, the global standard Perceptual Objective Listening Quality Analysis (POLQA) is used to model the speech path audio quality and produce the QoE as a mean opinion score (MOS). Combining such QoE estimates with technical QoS measures gives a clear picture of how network performance translates into customer satisfaction. 

How to add QoE-focused tests without hurting time-to-market?

Luckily, in an automated testing environment, testers don’t have to worry about fitting in all of these test cases that might typically have fallen by the wayside. Instead, they can plug in a few keywords, sit back, and let the magic happen (until it’s time for aftercare and root cause analysis). The trick is to adopt the right testing tools. It might include: 

  •   A test automation framework 
  •   An audio matrix and isolators (for audio QoE testing)   
  •   Shielded boxes and attenuators (for testing mobility-related use cases)
  •   Tracing tools like TCPdump and Wireshark (for going beyond end-to-end to capture signalling traces) 
  •   Testcase databases and libraries 

To employ some CICD workflow, you’ll also need a development repository and version control system like Git and an integration server like Jenkins (an open-source CI server that automatically builds and tests developer code after each commit). With the right tools set up, you can begin to create test flows that aren’t just about checking off boxes and verifying latency speeds—but are instead built around your customers and their needs. At the end of the day, it’s the customer’s priorities that determine your QoS, QoE, and ROI—a fact that your test flows need to reflect. 

Search

Interested in our Products ?

Scroll to Top
Segron logo black blue

Senior SaaS System Administrator

Technical Skills :
  • Oversee the sysadmin related tasks in our SaaS infrastructure (partially cloud based, partially bare metal)
  • Daily operation and maintenance of the system
  • Analysing and resolving incidents
  • Follow and help improving the incident and change management procedures
  • Design procedures for system troubleshooting and maintenance
  • Incorporating base OS updates and security patches
  • Ensure that systems are safe and secure against cybersecurity threats by raising change requests where potential threat is possible
  • Performing SW updates for the Segron SaaS SW stack (distributed architecture with clusters)
  • Configuring solutions like reverse proxy, firewalls, etc.
  • Building tools to automate procedures & reduce occurrences of errors and improve customer experience
  • Tutoring & coaching newcomers & less senior experts in the team
  • Interworking with the architects and IT admins of Segron to have the SaaS procedures inline with the Segron processes
Non-technical skills:
  • We are looking for a self-motivated, self-improving individual with a highly independent mindset and open and straightforward technical communication to help us to improve and maintain our cloud infrastructure of our powerful end-to-end testing solution ATF (Automated Testing Framework)
  • 3+ years hands-on experience with operation and monitoring of cloud / linux systems
  • 3+ years of hands-on experience with network devops elements: configuring routers, switches, networks
  • Hands-on experience with running live systems with infrastructure as a code mode of operation
  • Specific knowledge which brings direct advantage: Docker, Docker Compose, Grafana, Prometheus, Ansible, Debian Linux OS administration, Security
  • Experience in building and maintaining distributed systems (incl. redundancy, resiliency, load-balancing) is welcome
  • Excellent knowledge of English
Location :
  • Place of work: Bratislava (partially home office possible)
  • Rate: from 30 EUR/hour (possible higher rate, depends on experience)
Segron logo - The Next Generation of Active Testing
Segron logo black blue

CI/CD Senior Developer

Technical Skills :
  • A senior role with a proven expertise in software development, cloud computing, DevOps, and CI/CD
  • Experience in planning, designing, and overseeing the CI/CD strategy and architecture on the level of organization
  • Ability to tailor testing strategies which define and follow the best practices, standards, and policies for the software delivery process
  • Hands-on experience in creating and managing CI/CD pipelines and workflows (PaaC)
  • Ability to evaluate and recommend the best tools, technologies, and methodologies for the CI/CD implementation
  • Prior hands-on experience working with different CI/CD toolsets (Jenkins, Bitbucket, GitLab, artifactory, Ansible ..)
  • Proficient with DevOps tools API automation capabilities
  • Proficient with Atlassian Tools (BitBucket, Jira, Confluence) and agile SW development methodologies
  • Familiar with cloud patterns and best practices
  • Familiar with web performance best practices
  • Comfortable working in cloud DevOps ecosystem
  • Comfortable working with Linux platforms
  • Initial working experience in SW development is an advantage.
Non-technical skills:
  • Effective communication with technical as well and business stakeholders
  • Self-motivating, self-improving mindset
  • Ownership of relevant industry certificates is a plus
Location :
  • Location: Bratislava, Slovakia (with hybrid flexibility)
  • Rate: from 30 EUR/hour (possible higher rate, depends on experience)
Segron logo - The Next Generation of Active Testing
Segron logo black blue

Test Automation Engineer

Job description, responsibilities:

  • ATF system configuration, integration, operations & maintenance in customer environments.
  • Building tools to automate procedures & reduce occurrences of errors and improve customer experience.
  • Hardware Verification, Testing and Preparation within the Staging Process.
  • Contribution to customer and service partner technical support across multiple accounts by sufficiently managing priorities and deadlines for own work.
  • Segron Laboratory equipment configuration and maintenance support.
  • Hardware order and logistics support.
  • Problem analysis of ATF issues, troubleshooting and fault correction.
  • Interface towards SEGRON Development Team in case of product or software issues.
  • Interface towards the SEGRON Technical Sales Team to support planned activities.
  • System and Integration documentation and guidelines.
  • Perform root cause analysis for production errors.
  • Deployment of software updates and fixes.
  • Ability to work in a team environment serving multiple global customers.
  • Willing to travel for 3-5 days onsite deployments
Requirements/ Skills:
  • Excellent knowledge of English
  • Operating Systems: Linux, Windows, MacOS
  • Good Knowledge of Containers and Virtual Machines
  • Telco experience welcome
  • Python or other scripting experience or knowledge preferable
  • Educational Qualification: Computer Science/Engineering or work experience equivalent
  • Work Experience: 3-4 years preferred

Others:

  • Full time job (employment)
  • 3 days onsite, 2 days home office
  • Offered salary: from 1800 Euro (depends on seniority and skills level)
  • Variety of financial benefits
  • Place of work: Bratislava
Segron logo - The Next Generation of Active Testing
Segron logo black blue

Senior Python Developer

Technical Skills :

  • A solid, experienced SW developer with at least 10 years of experience in active SW development in different programming paradigms
  • Minimum 5 years of professional Python development experience
  • Master or college degree from Computer Science, Mathematics or STEM domain
  • Well educated in design and programming patterns that increase software’s efficiency and readability.
  • Very good analytical and problem solving skills.
  • At least three skills out of the following 4 skills are requested:
    • Microservices based architectures (Docker containers)
    • Linux
    • Ansible
    • Robot  Framework
  • Comfortable with sysadmin and DevOps skills (Ansible, YML files, Network Programming, IP protocols, designing and developing proxy servers for different protocols – example: streaming, integrating and compiling third party libraries on Linux (Debian))
  • Proficient with Atlassian Tools (BitBucket, Jira, Confluence) thorough understanding of Git and version control best practices
  • Familiar with cloud patterns and best practices
  • Familiar with web performance best practices

Non-technical skills:

  • Ability to work under pressure
  • Ability to abstract and explain your work
  • Strong understanding of Agile development process and experience working in an agile team
  • Strong communication skills with both technical and non-technical stakeholders
Location :
  • Bratislava, Slovakia (with hybrid flexibility)
  • Rate: from 35 EUR/hour (possible higher rate, but depends on experience)
Segron logo - The Next Generation of Active Testing