Tag Archives: programming

How to write a great bug report

“My client has a problem with a file”

The above is the full text of an actual bug report received by a former colleague of mine.

She could’ve contacted the person who raised the report asking him to tell her “Which client?”, “Which file?”, “What kind of problem?”, “Which system were they using?” or any other number of essential pieces of information. As my colleague had a list of bug reports numbering three figures, instead she shrugged and moved on to a report with better information.

No doubt the person who raised that report was also very busy, but they only wasted their own and others’ time by raising such an unhelpful report. I think that when reports like this are commonplace it suggests that there are some misunderstandings about communication between developers and users.

Can’t the developer just ask me for more information?

Perhaps the user whose client had a problematic file was only intending his very terse report to start a conversation.

That’s natural enough. When someone visits the doctor and says, “I feel unwell” they don’t expect a diagnosis straight away, rather a series of questions from the doctor to gain an  insight into the patient’s illness. That’s fine when both parties are sitting next to each other discussing it. When they are working in different offices, maybe different time zones, then it becomes a painfully slow process. If a developer asks for more information and the user is in a meeting or asleep, they can’t simply sit around waiting for a response, they’re forced to work on something else, with the associated cost of context switching. Sometimes further questions may be needed, but the user can jump-start that process with a clear report.

Surely the developer knows what I mean?

Users may have the impression that the software engineer knows more about the software than they do.

Software engineers may spend lots of time looking at code but much less, if any, time using the software. Users are the experts at how the software works as they often use it day in and day out. Their insights about how it works and how it should work are invaluable to developers. Sadly they are often left out of bug reports on the assumption that developers already know everything that users know.

The developer must be aware of this bug already!

A bug report might be seen by some as an official way to force a developer to fix an issue that they already know about.

Again, this isn’t usually the case. If a developer discovers an issue, they’d normally raise a bug report for it, even if they didn’t have time to fix it immediately. Everyone who raises reports should also be able to search the list of bugs to see if anything similar has already been reported. If the bug isn’t listed in the database, it’s likely that no one else knows about it. If it is, then there’s no point creating a duplicate.

Raising a duplicate report only creates confusion. Instead the user should be able to update the existing bug report perhaps to say, “This is also affecting client X” or “I have observed the issue on Sundays as well as Bank Holidays.”. This extra clue may make fixing the bug easier.

What does a good bug report look like?

If you write helpful bug reports, with a few minutes of your time you could save the developers hours, if not days, of their time. You do want this fixed quickly, right? Where developers can choose what to work on next, they’ll naturally prefer quick wins. Thorough yet concise bug reports focus the developer on a solvable problem. Vague ones feel like being asked to find a needle in a field full of haystacks.

Now I’ve lectured you on why it’s important to write good bug reports, here are my suggestions of their most important features. This list is generalised. For certain projects there may be additional essential information.

  1. Search for any similar bug report
    If found, optionally add to it. Do not create a duplicate. If unsure, link to the original.
  2. A short, descriptive title
    Enough to distinguish it from similar issues. Probably 4-10 words.
  3. Include all relevant details
    e.g.: Server, product, client(s) affected, file type, time of day, web browser version. Make that haystack smaller!
  4. Steps needed to reproduce the issue
    A developer may not know how to reproduce the issue. If they can’t do that they probably can’t fix it and definitely won’t know whether they’ve fixed it.
  5. Expected result
    Make it absolutely clear what you wanted the software to do. e.g.: “Display the total number of rows the uploaded file has at the top of the page.”
  6. Actual result
    Again this ensures clarity when compared to the above. e.g.: “Null is displayed and the menu-bar disappears.”

Those are the essentials. If you have time to investigate further then any other clues you can add are a bonus. For example, insights like these may also be helpful and speed up the process of debugging.

“This only happens on days immediately following bank holidays”

“It worked on Friday 4th, but failed on Monday 7th”

“It only affects admin users”.

I hope my suggestion helps users to get bugs fixed more quickly and developers to enjoy fixing them more.

What is the point of code reviews?

In most of the jobs I’ve had peer code review was an essential and regular part of the software development process. My experience is that it improves code quality and is well worth the effort. I’d also say that at least half of what I learnt in that time was through code review. Either someone would suggest improvements to my code, or I’d discover new ways of doing things from reviewing other people’s work. It’s a great way to share knowledge. This is not only true in specialist domains where the answers are not always easily found by searching the web, but any time someone is not aware of a better way of doing things. You won’t search for something if you don’t know it exists.

That’s my opinion, but there is also plenty of evidence that code review improves code quality, helps find bugs early and ultimately saves money. The numbers vary between studies, but finding bugs early is not just cheaper, it’s significantly cheaper.

What about testing?

No doubt some people will say that the purpose of unit testing is to find bugs early or, in the case of TDD, prevent them ever being created. So why do we need code review as well?

I agree that unit testing is powerful, indeed I’ve used TDD thoroughly to tame some seemingly intractable problems, but I still highly value code review. I think testing and code review achieve different things. Both are important.

Unit tests are great for quickly checking that everything that worked previously still works after a recent change. They provide confidence to refactor or experiment with code in the knowledge that the essential functionality can be quickly checked and rechecked. Achieving this via a code review would be slow, boring and error-prone.

Code reviews on the other hand can ask bigger questions, like:-

  • Is the code understandable?
  • Are the unit tests testing the right requirements?
  • Is there a more efficient way to do this?
  • Does it increase unplanned technical debt?

In fact, any check that can’t be automated. Coding style, static analysis, spell checking etc can all be automatic and the most a human reviewer should do is check the results.

Code review suggestions

Others have already written good things about how to conduct code reviews without annoying people or what tools you should use, so I won’t repeat that. Here are my observations and suggestions about how to get the most out of code review:-

  • The most important thing for a code review to check is whether the code is functionally correct. Does it do exactly what it says on the tin? Are the code and tests fully implementing the requirements of the story/ticket? Or has the developer misunderstood what’s needed in some way? If so then they’ve probably written the unit tests with the same misunderstanding, so they all pass fine. Yes, this is a hard thing to check – it takes some time to fully understand, but it is important. Get this wrong and nothing else matters.
  • Code review is a great way to share knowledge. Again, this does mean you have to take the time to properly understand the change, but it develops the team and mitigates risk.
  • There should be no reviewer hierarchy. Different perspectives are useful. More than once a more “junior” developer has reviewed my code and asked a “naïve” question only for me to scratch my head and say, “You’re right, I’ve done that totally wrong”. Even when it doesn’t happen quite like that, reviews tend to stimulate questions and the transfer of ideas.
  • That said, it can be worth making sure the local domain expert is one of the reviewers of any important change, but they don’t have to be the only reviewer.
  • Applying more engineer hours to the review does find more bugs, but each additional hour will be slightly less valuable on average than the last, so there’s a balance to be struck.
  • Don’t get bogged down in matters of opinion. When it comes to how to make code understandable and maintainable, there’s a big slice of judgement involved. “I wouldn’t do it that way” is not sufficient justification for raising a comment. There’s a danger of getting into “tomayto/tomahto” arguments. Be open-minded, take a step back and think about how important your point is.
  • Following on from the last point, it’s OK to have no comments. It might seem like you haven’t done your job as a reviewer, but it’s better than nitpicking for the sake of feeling productive.
  • If possible, do not manually check anything that can be automated. Style guidelines should be agreed across the project and checked quickly and automatically. Reviewers are human beings; they’re above that stuff and anyway they’re not very good at it.
  • Pair programming can function as a kind of instant review, but there are also risks. The reviewer can be too “close” to the code to be impartial or may make the same false assumptions about the requirements. A slower or more timid engineer might not want to question the other’s work without the time to think it through properly. So, while pairing can find bugs very early, I don’t think it should be the only kind of review a change receives.