REPORT Acronym For Better Bug Reports
The scariest part of any project is the User Acceptance Test (UAT). REPORT will help you and your customer to successfully get through this phase.
Picking up issues reported by customers often is a big source of frustration for developers and customers alike. Customers are generally in the dark about what developers need to fix a bug. Developers waste a lot of time trying to figure out customer bug reports.
To help customers write useful bug reports I have created a handy acronym called REPORT. REPORT should function for bugs like INVEST for User Stories.
REPORT stands for:
Reproducible
Expected Results
Priority
Observed Results
Relevance
Traceable
Reproducible
The customer generally overestimates how much developers actually know about the whole application. Developers normally do not have an overview of how the whole product works. When a developer picks up a bug report, it might be functionality they see for the first time. Bug reports need to be as easily reproducible as possible. Developers should develop and not waste time on trying to reproduce a poorly reported issue. All steps necessary to reproduce the issue should to be present. These steps should be so clear a developer without background knowledge can perform them regardless of the environment they are using. It is important to mention if the issue is consistently reproducible or not.
Common pitfalls:
Necessary preconditions are missing such as the browser used or specific configuration settings.
The issue depends on the environment where it was reported. Sometimes the cause cannot be figured out and this cannot be prevented.
Reproduction steps are missing or described in a vague way. The developer has to guess or play around to reproduce the issue.
The variable nature of the issue is missing. The result is that the developer gives up on reproducing the issue too early and sends it back to the customer.
Expected Results
Expected results are often missing from bug reports. The customer writes:”It does not work the way I expect”. This expectation is never made explicit. The developer pulls out his crystal orb and rubs on it to guess what the customer expects. With varying degrees of success. If you do not want to leave it up to chance, it is necessary to state how you expect it to work. The chance you get what you want is higher and the chance of rework is lower. It could also be that how you expect it to work conflicts with original requirements which were agreed upon. Making your expectation explicit makes it open for discussion.
Priority
All relevant information should be present to determine the priority of an issue. The priority of an issue results from the impact. By knowing the impact, the priority can be determined and validated by the customer and developers.
Who are affected? All users or only a specific subgroup of users? Are they always affected?
How are they affected? Can they still use the application for the most part? Is a suitable work-around available?
I once worked for a customer who reported every bug as blocking or critical. This was the result of us not picking up issues with lower priority. As a result the product missed deadlines and the customer was not happy with the delivered quality. Not all bugs are equal and almost always it is impossible to fix all bugs before a deadline. Being clear about the impact allows to prioritize effectively. If everything has equal priority, then nothing has priority. By stating the impact, the priority becomes negotiable for customers and developers. With a higher-quality product and a happier customer as the result.
Observed Results
A description of what happens when you perform the steps as described in the bug report. Information about when the bug does not happen may help to pinpoint the root cause.
Common pitfalls:
Observed results are incomplete. The user does not see a value and checks if it is present in the back-end. The back-end check is not mentioned in the bug report even though it was performed.
The cases where it work as expected are not mentioned. These might provide useful information about the root cause of the issue.
Relevant
A bug report needs to be atomic. Bug reports should only contain only the information relevant to reproduce a single issue. If many different issues are present in the same bug report this introduces noise. Remove all unnecessary steps and superfluous information to prevent confusion.
Common pitfalls:
The user describes twenty steps to reproduce an issue. Investigation results in only two of those steps actually required to reproduce the issue.
A bug report contains a collection of different issues. The issues originate from different pieces of code. Mentioning them in the same bug report restricts developers from working in parallel. Testing becomes more complicated. If a single issue is not fixed, then the whole bug report needs to be reopened. What is working and what is not working becomes unclear.
Traceable
Tracing bug reports to the original requirements gives developers valuable background information. Developers can check if what they are fixing conflicts with original requirements. In fixed-price contracts traceability is vital. It allows to determine if an issue is in scope or treated as a change not part of the original contract. Tracebility provides valuable context and decreases the chance that after a fix something else will break. Sometimes traceability is not possible, for example when you use software for which the customer does not have any requirements documentation. In this case the developers should link the bug report to the original requirement.
Conclusion
When your customer uses REPORT to judge the quality of their bug reports it will help developers to:
Pick up issues efficiently.
Reduce rework.
Decrease the chance of picking up something that conflicts with requirements.
It will help your customer to understand what developers need without a lengthy explanation. It provides a handy acronym to fall back on and increase understanding in a simple yet effective manner.