How to Decide on Mobile Bug Fixes

How to Decide on Mobile Bug Fixes

Any software product, be it a web, mobile or desktop application, is under suspicion unless it proves the features are working as expected. In short, software is never bug free. Depending on the software product, fixing bugs in production is not easy and, in most cases, it’s expensive.

Before the Release of an App

Before a mobile development team ships a native app to the app store, an intensive testing phase must happen to minimize the need for any hotfixes. Distributing the app to beta testers or external testing providers to gain early feedback is another approach to take. However, we all know that the real nasty bugs happen in the wild on customers’ real phones and often during scenarios that only exist in the real world.

The Mobile Bug Matrix

But how to decide which mobile bug is worth doing a hotfix? Over the past few years, I have seen many teams dealing with exactly this question. Some teams tried to fix every bug in production but ended up doing fixes for the released app and stopped doing feature development. Therefore, teams need to find the right balance between doing a hotfix and waiting until the next upcoming release.

To make the decision of doing a hotfix easier, I created the mobile bug matrix. The mobile bug matrix has 2 axes. The x-axis is the criticality/severity for the user, and the y-axis stands for the criticality/severity for the business.

Mobile bug matrix: low lower left corner, high top right: Next release for business (top left), critical hotfix not critical (top right), Upcoming sprint not critical (lower left), next release for users (lower right)


This results in three options. I’ve included an example bug along with each scenario:

  1. Hotfix: This area has the highest impact to the business and the user. If a bug was found on the live app and fits to this area, the team must fix it as soon as possible.
    Example Bug:
    The login of the app is not working anymore.
  2. Next Release: Bugs in this area either affect more the business side or the user side. The issue is not critical enough to perform a hotfix, but the team must fix it in the next release.
    Example Bug
    : UI elements of the app are in the wrong place.
  3. Upcoming Sprints: These problems need to be fixed but there is usually no pressure from either the business or user. These kinds of bugs can be added to the backlog and can be planned in upcoming sprints.
    Example Bug:
    Wrong translations or typos.

Raise Questions For Clarification

When talking to different stakeholders of the app, there will be different opinions on the criticality or severity of the found bug and in which area the bug fits. In order to prevent exhausting discussions, it might help if the team raises the following questions to make a decision easier to reach.

  1. How many users are affected by the bug?
  2. Are we losing activity on our app?
  3. Is the company losing trust?
  4. Do we have a legal issue with this bug?
  5. Are we losing money because of the bug?
  6. Is the problem reproducible on the supported devices and operating system versions?
  7. Which sections are affected?
  8. Can we switch off the feature from the backend?

Once a team has decided if the problem is worth performing a hotfix on, the existing software development approach must be used. Find the root cause for the issue, fix the issue and cover it with some automated checks. Once development is done, a software tester or beta testers must verify that the bug has been fixed. Furthermore, regression testing has to be performed to check that the fix has not introduced additional problems. Last but not least, the whole test automation suite must be executed, and the results must be green.

Find a Workflow for the Team

In summary, the mobile bug matrix is one approach to make a decision on whether or not a bug gets fixed immediately. Every team must define a workflow to handle this situation of critical or non-critical bugs in the production system. It’s important to find the right balance between a hotfix and the upcoming release because fixing a mobile bug isn’t usually fast and cheap. Next to the development and testing costs, there are also app review and submission waiting times for some app stores until the fix is live for the customers, and it might be the case that the next planned app release is just around the corner.

Whitepapers

The Essential Guide To Mobile App Testing

To effectively test mobile apps, now and into the future, enterprises must ensure a strong user experience from the beginning. Here's how to implement an effective mobile testing strategy.

Read Now
Want to see more like this?
Daniel Knott
Daniel Knott
Mobile Testing Expert
Reading time: 5 min

Digital Quality Matters More Than Ever: Do Your Experiences Keep Customers Coming Back?

Take a deep dive into common flaws in digital experiences and learn how to overcome them to set your business apart.

4 Ways to Get Maximum Value from Exploratory Testing

Well-planned exploratory testing can uncover critical issues and help dramatically improve the customer experience. See how to guide testers to where exploration can yield the greatest returns.

3 Keys to an Effective QA Organization

Get your internal, external and crowdsourced testers on the same page

What is the Metaverse? And What Isn’t It?

It’s not far-flung sci-fi anymore — the metaverse is here, and it requires companies to rethink their approach to UX and testing

Why Machine Learning Projects Fail

Read this article to learn the 5 key reasons why machine learning projects fail and how businesses can build successful AI experiences.

How Localization Supports New-Market Launches

Success or failure in a new market is all about how you resonate with customers — don’t skimp on prep work