segron logo

The Top 5 Causes of Network Outages

As  mobile voice and data services have become a communication necessity for business and private purposes,  network outages will continue to be a serious problem for telco operators. A Heavy Reading report from a few years ago suggested that network outages in the telco domain seemed to be getting both longer and more frequent, with a rise in the volume of outages that exceeded 48 hours.

Though network conditions, and the networks themselves, have changed since then (it was still relatively early in the transition of IP replacing other protocols for real-time services), the possibility of an outage is still top of mind for most operators. After all, outages can do serious damage to your corporate reputation and result in subscriber churn and revenue losses.

At SEGRON, we believe that the first step to preventing network outages is understanding their primary root causes. The second step… well, that depends on which cause was actually responsible for the outage in the first place.

1. Overloaded Networks

In the report we mentioned above, congestion and overloaded networks were listed as the top causes of outages. On one hand, the timing of this report means that part of what we were seeing several years ago was networks grappling with the shift away from mostly voice and SMS traffic to increasingly diverse network traffic that emphasized low-latency mobile data and VoLTE usage.

On the other hand, the industry is currently entering a new era of transition from 4G/LTE to 5G. You can now expect another sustained increase in network traffic, this time from the internet of things (IoT). As 5G powers an increasing volume of low-latency network connections going forward, load testing will be more crucial than ever for telco testers.

In order to ascertain whether your network can actually hold up during periods of congestion, you’ll have to find practicable ways to stress test the network itself. This will require automation frameworks that are capable of leveraging entire networks of connected devices at the push of a button.

2. New Services

While network congestion was cited as the most frequent cause of outages, network issues were the leading culprits when it came to serious and severe outages. These issues can take any number of forms, but some of the most common were outages related to new service offerings.

Again, this is something that the 5G era is likely to see a lot of, for example:

You’re upgrading your network so that subscribers can access 5G data speeds when streaming video, but something goes awry with    the implementation and the network crashes.

 You’re updating your roaming partnerships and the new interconnections cause issues that you didn’t anticipate.

The traditional bulwark against this kind of issue is extensive regression testing, but that can be time-consuming and slow down time-to-market, which companies are often hesitant to do. The trick here is finding a way to speed up regression testing to improve test coverage without impacting time-to-market.

3. New Devices

For network failures, new service offerings are more frequently cited than the introduction of new devices, but the latter still causes its share of outages. At the risk of sounding like a broken record, this is another area where the risk is only going to increase as mobile networks evolve and more devices enter into common usage.

To decrease the odds that a new device will cause issues for your network, you’ll need to implement a robust testing infrastructure for devices that includes both end-to-end tests (dialing and calling, sending SMS messages, using mobile data, etc.) and tests that go beyond end-to-end.

What does this mean, exactly? Essentially, it means gathering signalling traces and tracking EDRs/CDRs from the systems under test. This allows you to get protocol-level information about how each element of each test is being carried out. From there, you can identify potential issues that aren’t yet manifesting themselves at either endpoint.

The only trick here is incorporating newly released, flagship devices into your automation framework quickly enough to maintain test velocity, but AI-powered workflows can be a big help here.

4. Misconfigurations

Another frequently cited cause of network outages is misconfiguration, which can typically be translated to human error. In the telecommunications industry, there’s a lot of manual effort that goes into any new service offerings, changes to your network, or adjustments to the equipment you’re using. While there’s no way around the fact that “to err is human,” it is possible to decrease the amount of manual intervention that goes into any of these updates and changes.

You can start with ramping up automation in your test labs by creating test scripts for each use case that do the same thing, the same way, every time without deviation. From there, it will produce test reports the same way every time, leading to greater transparency between testing and other functions.

Obviously there’s no way to root out human error completely, but you can detect it and reduce it through tactics like this.

5. Physical Failures

This is another term that works as something of a catch-all. Obviously, a physical failure can be the result of anything from a hurricane taking down phone lines to rats gnawing through the wiring of your network equipment. Some of these may be partially human error again, but often times they really will be situations that are out of your control.

That said, there’s plenty you can do to prepare your organization for things like this when they do happen. For instance, any infrastructure you can put in place to help you identify the issue more quickly will be a huge advantage. This can be a matter of testing, but it’s also important to retain documentation in a way that’s accessible for key stakeholders.

For instance, if someone has identified a similar issue in the past, it’s of the utmost importance that whoever’s responding to the outage can locate, access, and understand that fix and other similar fixes. Reporting matters; it’s crucial that your incident report be readable and consistent, one of the many benefits of keyword-based testing. From there, you’ll also want an official operational plan for any possible outage scenario, outlining who’s in charge of doing what and when.

Of course, there’s no way to stop outages completely, but an ounce of prevention is worth a pound of cure. If you are eager to know more about how SEGRON can help you mitigate the costs of network outages, contact us.

Search

Interested in our Products ?

Scroll to Top
Segron logo black blue

Senior SaaS System Administrator

Technical Skills :
  • Oversee the sysadmin related tasks in our SaaS infrastructure (partially cloud based, partially bare metal)
  • Daily operation and maintenance of the system
  • Analysing and resolving incidents
  • Follow and help improving the incident and change management procedures
  • Design procedures for system troubleshooting and maintenance
  • Incorporating base OS updates and security patches
  • Ensure that systems are safe and secure against cybersecurity threats by raising change requests where potential threat is possible
  • Performing SW updates for the Segron SaaS SW stack (distributed architecture with clusters)
  • Configuring solutions like reverse proxy, firewalls, etc.
  • Building tools to automate procedures & reduce occurrences of errors and improve customer experience
  • Tutoring & coaching newcomers & less senior experts in the team
  • Interworking with the architects and IT admins of Segron to have the SaaS procedures inline with the Segron processes
Non-technical skills:
  • We are looking for a self-motivated, self-improving individual with a highly independent mindset and open and straightforward technical communication to help us to improve and maintain our cloud infrastructure of our powerful end-to-end testing solution ATF (Automated Testing Framework)
  • 3+ years hands-on experience with operation and monitoring of cloud / linux systems
  • 3+ years of hands-on experience with network devops elements: configuring routers, switches, networks
  • Hands-on experience with running live systems with infrastructure as a code mode of operation
  • Specific knowledge which brings direct advantage: Docker, Docker Compose, Grafana, Prometheus, Ansible, Debian Linux OS administration, Security
  • Experience in building and maintaining distributed systems (incl. redundancy, resiliency, load-balancing) is welcome
  • Excellent knowledge of English
Location :
  • Place of work: Bratislava (partially home office possible)
  • Rate: from 30 EUR/hour (possible higher rate, depends on experience)
Segron logo - The Next Generation of Active Testing
Segron logo black blue

CI/CD Senior Developer

Technical Skills :
  • A senior role with a proven expertise in software development, cloud computing, DevOps, and CI/CD
  • Experience in planning, designing, and overseeing the CI/CD strategy and architecture on the level of organization
  • Ability to tailor testing strategies which define and follow the best practices, standards, and policies for the software delivery process
  • Hands-on experience in creating and managing CI/CD pipelines and workflows (PaaC)
  • Ability to evaluate and recommend the best tools, technologies, and methodologies for the CI/CD implementation
  • Prior hands-on experience working with different CI/CD toolsets (Jenkins, Bitbucket, GitLab, artifactory, Ansible ..)
  • Proficient with DevOps tools API automation capabilities
  • Proficient with Atlassian Tools (BitBucket, Jira, Confluence) and agile SW development methodologies
  • Familiar with cloud patterns and best practices
  • Familiar with web performance best practices
  • Comfortable working in cloud DevOps ecosystem
  • Comfortable working with Linux platforms
  • Initial working experience in SW development is an advantage.
Non-technical skills:
  • Effective communication with technical as well and business stakeholders
  • Self-motivating, self-improving mindset
  • Ownership of relevant industry certificates is a plus
Location :
  • Location: Bratislava, Slovakia (with hybrid flexibility)
  • Rate: from 30 EUR/hour (possible higher rate, depends on experience)
Segron logo - The Next Generation of Active Testing
Segron logo black blue

Test Automation Engineer

Job description, responsibilities:

  • ATF system configuration, integration, operations & maintenance in customer environments.
  • Building tools to automate procedures & reduce occurrences of errors and improve customer experience.
  • Hardware Verification, Testing and Preparation within the Staging Process.
  • Contribution to customer and service partner technical support across multiple accounts by sufficiently managing priorities and deadlines for own work.
  • Segron Laboratory equipment configuration and maintenance support.
  • Hardware order and logistics support.
  • Problem analysis of ATF issues, troubleshooting and fault correction.
  • Interface towards SEGRON Development Team in case of product or software issues.
  • Interface towards the SEGRON Technical Sales Team to support planned activities.
  • System and Integration documentation and guidelines.
  • Perform root cause analysis for production errors.
  • Deployment of software updates and fixes.
  • Ability to work in a team environment serving multiple global customers.
  • Willing to travel for 3-5 days onsite deployments
Requirements/ Skills:
  • Excellent knowledge of English
  • Operating Systems: Linux, Windows, MacOS
  • Good Knowledge of Containers and Virtual Machines
  • Telco experience welcome
  • Python or other scripting experience or knowledge preferable
  • Educational Qualification: Computer Science/Engineering or work experience equivalent
  • Work Experience: 3-4 years preferred

Others:

  • Full time job (employment)
  • 3 days onsite, 2 days home office
  • Offered salary: from 1800 Euro (depends on seniority and skills level)
  • Variety of financial benefits
  • Place of work: Bratislava
Segron logo - The Next Generation of Active Testing
Segron logo black blue

Senior Python Developer

Technical Skills :

  • A solid, experienced SW developer with at least 10 years of experience in active SW development in different programming paradigms
  • Minimum 5 years of professional Python development experience
  • Master or college degree from Computer Science, Mathematics or STEM domain
  • Well educated in design and programming patterns that increase software’s efficiency and readability.
  • Very good analytical and problem solving skills.
  • At least three skills out of the following 4 skills are requested:
    • Microservices based architectures (Docker containers)
    • Linux
    • Ansible
    • Robot  Framework
  • Comfortable with sysadmin and DevOps skills (Ansible, YML files, Network Programming, IP protocols, designing and developing proxy servers for different protocols – example: streaming, integrating and compiling third party libraries on Linux (Debian))
  • Proficient with Atlassian Tools (BitBucket, Jira, Confluence) thorough understanding of Git and version control best practices
  • Familiar with cloud patterns and best practices
  • Familiar with web performance best practices

Non-technical skills:

  • Ability to work under pressure
  • Ability to abstract and explain your work
  • Strong understanding of Agile development process and experience working in an agile team
  • Strong communication skills with both technical and non-technical stakeholders
Location :
  • Bratislava, Slovakia (with hybrid flexibility)
  • Rate: from 35 EUR/hour (possible higher rate, but depends on experience)
Segron logo - The Next Generation of Active Testing