Are QoS and QoE the same thing?

Sometimes, as a tester, you might feel like your users are experiencing a completely different system from the one that you’re testing and deploying. You might, for instance, devote countless person-hours and resources to finding areas in your network where improvements to the code could lead to better KPIs for Quality of Service (QoS), reducing jitter and packet loss substantially, only to find that your users haven’t noticed a difference. You might even find that your users experience decay in network quality, despite the objective metrics that say otherwise. What gives? Simply put, a given user’s experience on your network is never going to correlate 100% with objective quality metrics. This might be for any number of reasons—for new 5G services, differences in latency time might be too minute for most humans to notice. For the same reason, users will weight small issues with some services (e.g., streaming videos or video-conferencing) disproportionally to others. Whether their perceptions are rational or not, they’re still deciding whether to renew your services at the end of every month, so churn prevention depends on finding a way to keep them happy. How can network operators and testers pull off this feat? One option is to incorporate QoE (Quality of Experience) metrics into your standard suite of KPIs.

Quality of Service (QoS) vs. Quality of Experience (QoE)

How do QoS and QoE differ, and why does this difference matter? QoS tends to involve more traditional, objective metrics that testers can easily measure when extracting data from the System Under Test (SUT). This includes jitter, latency, packet loss, and other parameters, all of which are critical indicators of your service’s health. 

Segron Metrics

Not only do they give you the objective facts for what users will experience when they try to access services on your network, but they also let you know whether you have any potential compliance issues to worry about (e.g., whether your 5G speeds are within the 3GPP’s official range, or whether you’re successfully completing fallbacks and handovers without dropping calls at the rate that regulators recommend).

For QoE, things can get a tiny bit fuzzier. Since it seeks to show the service quality from a subjective, user-focused point of view, you need to move beyond existing QoS metrics. To begin with, you might establish comparative metrics based on the numbers other service providers are showing. These metrics need to be truly End-to-End because many factors in the SUT may be playing a role. If a file download is slow, the end-user doesn’t know (and often doesn’t care) if the cause of the delay is at the server or in the carrier network—they know that their service isn’t up to the standard of quality that they’re paying for. From there, you can begin to develop mean opinion scores for how different users experience various facets of your service.

By combining such QoE estimates with technical QoS measures, you can give a better picture of how network performance translates into customer satisfaction. Since user experience data is such a reliable indicator of the churn risk, cross-validating QoE estimates with network performance data allows you to home in on the weak links in the service delivery chain. In this way, you work towards more optimal retention plans.

 

How POLQA can inform QoE measures

How POLQA can inform QoE measures

We alluded briefly above to some potential avenues for developing QoE metrics, but let’s dig a little deeper into one of the existing standards being put to use right now to sketch out QoE estimates: POLQA (Perceptual Objective Listening Quality Analysis). For audio QoE, this is the closest thing telco operators have to globally agreed-upon benchmarking systems. 

The POLQA standard has been around since 2011, and it uses a collaboratively designed algorithm to model your network’s audio quality as a mean opinion score. In this way, you can complement your existing QoS test data to identify potential service faults and address them in a way that will speak specifically to users’ issues.

The trick here is to incorporate these measurements seamlessly into your existing testing streams and reports. This requires an automation framework with the flexibility to integrate with other tools and cover a wide range of functionality in an agile way. For starters, you’ll need something that uses real, out-of-the-box mobile devices to orchestrate testing. However, you might also look for a solution that offers robust reporting and analytics capabilities. You can uncover the true meaning and any hidden correlations in the data that you capture.

 

QoE Beyond End-to-End

So far, we’ve talked about QoE in terms of end-to-end tests, but what about testing beyond end-to-end? For instance, once you’ve got your test environment automated, you can incorporate an Audio Matrix to begin getting more objective measurements for things like sound and audio quality that aren’t covered by traditional End-to-End tests.

SEGRON ATF Beyond END2END Test Automation

The Audio Matrix is based on virtual sound cards and facilitates tone pattern identification, acoustic fingerprinting, and the POLQA speech quality algorithm using digital audio transport.
Why leverage your resources on something like this? These audio quality metrics give you the foundations you need to begin putting user experience front and center in your tests. End-to-End tests like tone validation can give you a simple pass-fail on whether the sound is coming through; this gives you real insights into the things that your users really care about. By combining this with stuff like trace captures and CDR analyses, you begin to develop a much more comprehensive picture of network functionality than you ever could have formed in the past. With more knowledge in hand, you can address issues that arise in your network more quickly and effectively. Thus, by combining QoS and QoE metrics, it suddenly becomes possible to address the factors that most significantly impact churn.

Download our latest eBook and learn more about what Automated Testing Ecosystems should consider to be ahead in the market.

Search

Interested in our Products ?

Scroll to Top
Segron logo black blue

Senior SaaS System Administrator

Technical Skills :
  • Oversee the sysadmin related tasks in our SaaS infrastructure (partially cloud based, partially bare metal)
  • Daily operation and maintenance of the system
  • Analysing and resolving incidents
  • Follow and help improving the incident and change management procedures
  • Design procedures for system troubleshooting and maintenance
  • Incorporating base OS updates and security patches
  • Ensure that systems are safe and secure against cybersecurity threats by raising change requests where potential threat is possible
  • Performing SW updates for the Segron SaaS SW stack (distributed architecture with clusters)
  • Configuring solutions like reverse proxy, firewalls, etc.
  • Building tools to automate procedures & reduce occurrences of errors and improve customer experience
  • Tutoring & coaching newcomers & less senior experts in the team
  • Interworking with the architects and IT admins of Segron to have the SaaS procedures inline with the Segron processes
Non-technical skills:
  • We are looking for a self-motivated, self-improving individual with a highly independent mindset and open and straightforward technical communication to help us to improve and maintain our cloud infrastructure of our powerful end-to-end testing solution ATF (Automated Testing Framework)
  • 3+ years hands-on experience with operation and monitoring of cloud / linux systems
  • 3+ years of hands-on experience with network devops elements: configuring routers, switches, networks
  • Hands-on experience with running live systems with infrastructure as a code mode of operation
  • Specific knowledge which brings direct advantage: Docker, Docker Compose, Grafana, Prometheus, Ansible, Debian Linux OS administration, Security
  • Experience in building and maintaining distributed systems (incl. redundancy, resiliency, load-balancing) is welcome
  • Excellent knowledge of English
Location :
  • Place of work: Bratislava (partially home office possible)
  • Rate: from 30 EUR/hour (possible higher rate, depends on experience)
Segron logo - The Next Generation of Active Testing
Segron logo black blue

CI/CD Senior Developer

Technical Skills :
  • A senior role with a proven expertise in software development, cloud computing, DevOps, and CI/CD
  • Experience in planning, designing, and overseeing the CI/CD strategy and architecture on the level of organization
  • Ability to tailor testing strategies which define and follow the best practices, standards, and policies for the software delivery process
  • Hands-on experience in creating and managing CI/CD pipelines and workflows (PaaC)
  • Ability to evaluate and recommend the best tools, technologies, and methodologies for the CI/CD implementation
  • Prior hands-on experience working with different CI/CD toolsets (Jenkins, Bitbucket, GitLab, artifactory, Ansible ..)
  • Proficient with DevOps tools API automation capabilities
  • Proficient with Atlassian Tools (BitBucket, Jira, Confluence) and agile SW development methodologies
  • Familiar with cloud patterns and best practices
  • Familiar with web performance best practices
  • Comfortable working in cloud DevOps ecosystem
  • Comfortable working with Linux platforms
  • Initial working experience in SW development is an advantage.
Non-technical skills:
  • Effective communication with technical as well and business stakeholders
  • Self-motivating, self-improving mindset
  • Ownership of relevant industry certificates is a plus
Location :
  • Location: Bratislava, Slovakia (with hybrid flexibility)
  • Rate: from 30 EUR/hour (possible higher rate, depends on experience)
Segron logo - The Next Generation of Active Testing
Segron logo black blue

Test Automation Engineer

Job description, responsibilities:

Hardware Testing: Conduct verification and testing of hardware during the HW/SW products
assembly process.
Technical Support: Provide technical assistance during testing activities and troubleshoot any
hardware issues that arise. Support to internal teams and customers by managing priorities and
meeting deadlines.
Equipment Maintenance: Configure and maintain SEGRON laboratory and customer equipment.
Logistics Support: Handle hardware ordering and logistics.
Troubleshooting: Analyze and resolve SEGRON product issues through effective problem-solving.
Ability to troubleshoot and resolve issues that arise during the integration process.
Collaboration: Work with the SEGRON Development Team on product or software issues and assist
the Project Team with planned activities.
Root Cause Analysis: Investigate production errors to identify underlying causes.
Software Deployment: Implement software updates and fixes.
Team Collaboration: Ability to work collaboratively in a team environment, especially in serving global
customers.

Requirements/ Skills:

Networking knowledge: Understanding TCP/IP networks and LAN/WAN configurations. Routing and
switching and related protocols strongly desired.
Hardware Proficiency: Understanding of hardware components, including servers, routers, and
peripherals, to ensure proper assembly and integration. Experience in data centers is a plus.
Travel Abilities: Willing to travel for customer on-site support for hardware installations, migrations,
upgrades, troubleshooting, etc. (few times a year, 3-5 days)
Hand Tools: Proficiency in using hand tools like screwdrivers, pliers, crimpers, and punches for
assembling and disassembling components.
Operating Systems: Knowledge of Linux and Windows. VM experience (Hyper-V, VMWare, Oracle
Virtual Box, Proxmox etc.)
Tools Proficiency: Basic understanding of Ansible.
Telecommunications Experience: Previous experience in the telecom sector is a plus.
Education: Degree in Computer Science/Engineering or equivalent work experience.
Experience: At least 2 years of relevant work experience preferred. A strong desire to continuously
improve skills and knowledge in hardware and software integration, with an openness to feedback and
new ideas. Key is openminded person willing to learn fast in great team.
Communication Skills: Effective oral and written Eng

Others:

• Full time job (employment)
• Onsite work during onboarding period, then it can be 3 days onsite, 2 days home office
Offered salary: from 2000 Euro (depends on seniority and skills level)
Variety of financial benefits
Place of work: Bratislava

Segron logo - The Next Generation of Active Testing
Segron logo black blue

Senior Python Developer

Technical Skills :

  • A solid, experienced SW developer with at least 10 years of experience in active SW development in different programming paradigms
  • Minimum 5 years of professional Python development experience
  • Master or college degree from Computer Science, Mathematics or STEM domain
  • Well educated in design and programming patterns that increase software’s efficiency and readability.
  • Very good analytical and problem solving skills.
  • At least three skills out of the following 4 skills are requested:
    • Microservices based architectures (Docker containers)
    • Linux
    • Ansible
    • Robot  Framework
  • Comfortable with sysadmin and DevOps skills (Ansible, YML files, Network Programming, IP protocols, designing and developing proxy servers for different protocols – example: streaming, integrating and compiling third party libraries on Linux (Debian))
  • Proficient with Atlassian Tools (BitBucket, Jira, Confluence) thorough understanding of Git and version control best practices
  • Familiar with cloud patterns and best practices
  • Familiar with web performance best practices

Non-technical skills:

  • Ability to work under pressure
  • Ability to abstract and explain your work
  • Strong understanding of Agile development process and experience working in an agile team
  • Strong communication skills with both technical and non-technical stakeholders
Location :
  • Bratislava, Slovakia (with hybrid flexibility)
  • Rate: from 35 EUR/hour (possible higher rate, but depends on experience)
Segron logo - The Next Generation of Active Testing