Defects are inherent to any software development project and cannot be avoided. The best way to minimize the impact of defects is to put together a defect management process and ensure that defects are captured, reported, categorized and resolved with minimal effort and cost spends. There is huge. What is defect management process? Generally, defect management can be defined as a process of detecting bugs and fixing them. It is necessary to say that bugs occur constantly in the process of software development. They are a part of the software industry. That is because of the fact that software development is quite a complex process.
A complete guide to Defect Triage Process and Effective ways to handle Defect Triage Meeting:
In today’s article, we will learn about Defect Triage meeting and how to handle a triage meeting in an easier and effective way.
Before proceeding further with this article, I wish that everyone knows what is meant by a Defect, Defect Life Cycle, and how to set Priority & Severity for each defect. And it is necessary to understand these basic concepts related to a defect or bug.
You can also go through my earlier article “Defect Life Cycle and Defect Management Process” to understand these concepts quickly.
What You Will Learn:
- Roles and Responsibilities
Overview
So what can we do to improve the process? There are some goals of Defect management process: Senior management must understand, support, and be a part of the Defect Management Program. The Defect Management process should be integrated into the overall software development process and used by the team to improve the process. Defect cycle or defect life cycle is ride of a defect from discovering defect to closure of defect. Defects management in defect cycle is important to ensure the software quality. Preventing, identifying, rectifying defect is important to improve the quality. Defect is managed and tracked easily. A Full Life Cycle Defect Process Model That Supports Defect Tracking, Software Product Cycles, And Test Iterations Jim Nindel-Edwards [email protected] Gerhard Steinke Seattle Pacific University [email protected] ABSTRACT There are a variety of models, methods and tools to help organizations manage defects found in the development of software.
The word “Triage” is basically used in the medical field. Actually, it used to decide the order in which the patients should be treated. Usually, in big hospitals, where there are thousands of patient’s approaches for consultation or actual treatment on a daily basis. But not all the patients are admitted or treated immediately.
The severity of the illness or the injury is the main criteria for consultation and based on this all the patients are categorized accordingly. If the injury or health of any patient is very critical then the doctors usually treat such patients as a priority and get admitted if required.
Normal diseases or non-critical injuries are considered at a lower priority and such patients are treated later.
Similarly, the term Triage is introduced in software testing for defects in the application or a project. Usually, the Defect Triage process is implemented in large projects and in many cases, it is not applicable for small-scale projects. There are chances to identify a huge number of defects in bigger projects than medium or small projects.
Also in bigger projects, the frequency of defect identification is quite higher.
Take a look at the below image which shows the outcome of Defect triage meeting and gives answers to specific questions like:
Defect Triage Meeting
The main objective of a triage meeting is to track all the defects and ensure the correct resolution in a timely manner.
During the test execution phase, the testers start reporting defects in the Defect Management tool like HP ALM, QC etc. Then Defect Triage Meeting is held in which the developers and testers are required to be present as these people will discuss all the defects and take the necessary further course of action.
Mainly the presence of the below participants is required mandatorily:
- Project Manager
- Test Lead
- Development Lead or Developer
- Tester
- Test Manager
- Business Analyst
- Environment Manager
Although I have given an exhaustive list of all the participants in the meeting, it is not necessary to involve all of them like Business Analyst, Environment Manager, Test Manager, etc in the daily meeting. Whenever necessary the Test Lead or Project Manager invite them and they can share their valuable feedback and opinion regarding a specific defect.
And the entire team is known as a Triage Team. Now, I'm going to explain the exact process of triage meeting and how this meeting is set-up.
Consider one hypothetical Example: We have one project related to the Banking application, size is very large and the frequency of identifying and reporting the defect is high. Hence the Test Lead decides to set-up a Defect Triage Meeting with the required participants.
For setting up a meeting the Test Lead sends a meeting invite via email to everyone and sets a particular timing for Triage Meeting. The below given hypothetical image shows the meeting invite sent by a Test Lead via outlook to all the participants.
Here everything is imaginary in the below image like – the participant names, meeting room, conference call details, date, time etc.
(Note: Click on any image for an enlarged view)
Every day before the start of the triage meeting, the Test Lead sends a list of all the “Open” defects is a spreadsheet format to all the participants so that they can go through all the defects before the meeting and understand what exactly the defect is and what kind of fix is required for it.
Before the start of every triage meeting, ensure that each defect:
- Has enough information to understand the defect for all the participants in the meeting.
- Has reported under correct project and category.
- Has mentioned the priority and severity of the defects.
- All the detailed information provided in the defect to understand it correctly to all the participants.
Recommended Read => A Complete Guide to Defect Management Process
Defect Triage Template
Before the kickstart of every Defect Triage Meeting, the Test Lead shares the defect report to all the participants in a specific format and the report pulled out from the Defect Management Tool like HP ALM, HP QC etc. I am showing one sample format in the below image which will give a high-level idea of which fields are mentioned in the Defect Report Template.
Usually, the fields included in the defect report are:
- Defect ID
- Description
- Priority
- Severity
- Detected Date
- Detected By
- Status
The list is not exhaustive but as per the project need, the other fields in the defect report template can be included.
Usually, the spreadsheet format is used as a template for defect reporting, hence I have given the hypothetical defect details in the spreadsheet format. Please note that all the information provided in the above defect report is only imaginary and not related to any project or actual application.
Defect Triage Process
A commonly heard and experienced situation in test teams is limited availability of resources. Defect triage is a process which tries to do some re-balancing as a result of this phenomenon. So when there are many defects and limited Developers/testers to fix/verify them, defect triage helps to get as many defects resolved as possible by balancing the technical personnel based on defect parameters like priority and severity.
Typically, a defect triage session is attended by the Product Manager, a development lead, a test lead and sometimes business analysts. In some cases, certain other members may also be invited to give their opinions and perspectives regarding certain defects. These are collectively called a triage team.
Most systems use priority as the main criteria to assess the defect, however, a good triage process considers the severity as well.
Let's take a closer look at the triage process with two examples that we've talked about in the previous section. In both the examples above, it would actually be the first defect that would be given a very high priority. Despite it being only a cosmetic defect, the impact of not fixing would be huge.
The second one, on the other hand, is a surely functionality defect, however, its occurrence is in only certain conditions which are seldom practiced customer scenarios. Fixing it may need more time and people, which could be better utilized for other defects. Hence it would deem lower priority than that of the first and maybe deferral candidate to another release.
Thus the triage process involves triage team sitting down together, reviewing all the defects including rejected defects. They draw an initial assessment of the defects based on its content, their respective priority, and severity settings; with each person in the triage team presenting their perspective on how to prioritize the defects.
The product manager then sets the priority based on all the inputs and assigns the defect to the correct release I.e. in the current release or any future release. He also redirects the defect to the correct owner/team for further action. Rejected defects also are put through a similar analysis. Based on the reason for rejection, the futuristic action of whether it needs to be deferred or canceled is determined.
In the triage meeting, each and every defect should be discussed including the defects which are categorized as a lower priority one. The triage team review evaluates all the defects and takes necessary action on each defect. If a defect is running short of information then the developer assigns back such defects to the testers and requests for necessary information.
The triage meeting can be held in the meeting room if all the participants are at the same location. But in many organizations, the work is carried out from a different location and all the teams are spread across various locations so that the meeting is also held using teleconference or business Skype.
[image source]
Step by step process of the defect triage meeting:
- Test Lead kicks off the meeting with the defect report which was sent earlier on the day.
- The discussion starts with the actions pending from the previous triage meeting. The necessary updates or action that was taken on any defect is discussed initially.
- If there are new defects in the defect report then these defects are reviewed and evaluated. It also verifies if the priority and severity are assigned properly, if not, then these are corrected in the meeting.
- All the defects are discussed in the meeting and the development team also discusses the complexity of fixing the defect. The risk associated with the defect is also discussed by the triage team.
- Triage team comes to a conclusion on, which defect should require immediate attention & fix and which defect needs to wait for some time and if required those defects can be postponed to future releases.
- All the defects are assigned to the respective team in QC or ALM simultaneously during the meeting. Appropriate comments are also added in the QC/ALM.
- All the essential updates and action items are noted down and the Test Lead calls out for the end of the meeting.
- After triage meeting completion, Test Lead sends out minutes of meeting to all the participants.
Roles and Responsibilities
Roles and responsibilities based on each category are explained below:
Test Lead
- Test Lead schedules a defect triage meeting and sends out a formal meeting invite to the required team.
- Sends the defect report before every triage meeting.
- Kicks off the meeting with the pending action items from the previous triage meeting.
- Discuss each defect and impact on the schedule if any functionalities are blocked due to the defect.
- Helps in assigning priority and severity of each defect if it was not assigned correctly earlier.
- Update the QC/ALM with appropriate comments.
- Note down all the updates, action items, risk related to a defect, etc.
- Sends minutes of meeting to all the participants.
Development Lead/Developer
- Share updates on the action items pending from the last triage meeting.
- Discuss all the defects from a technical perspective.
- Identify how much time it will require for fixing based on the complexity of the defect and functionality.
- Discuss the complexity of the defect and risk associated with the defect if any.
- Development Lead assigns defect to the appropriate developer after validating all the available detailed information.
- Updates the defect with the expected resolution date.
- Assists in identifying the root cause of the defect.
Project Manager
- Ensure that if all the representative from every area is available for the meeting.
- If necessary, project manager invites Business Analyst in the meeting for their opinion on a specific defect.
- If the defects are not moving or if there is any major blocker then escalates with the escalation process.
- If required, acts as a mediator if any dispute or conflict happens between the teams and takes the necessary decision.
- Take the confirmation from the development team for the next release date for fixed defects.
- Make aware of the updated schedule and release date of the project to all the teams.
At times, it is also a good idea to involve the other team members in the triage call so that they can also understand and contribute to the meeting and if required they can also provide their feedback.
Conclusion
Every defect logged should be discussed at the triage meeting.
Even if a defect is rejected, the testing team should know the reason for rejection. Also if any of the defects is not reproducible then during the triage meeting the developer can ask the testers for real-time details and they can try to reproduce the defect.
Defect Triage is important as everyone will know when the defect will get fixed and be available for re-test. If any of the defects is non-critical and in order to fix the defect, it requires huge efforts from the development team and the decision will be taken by the project manager.
The project manager will decide the priority of such defect and if required the defects can be postponed to the next release.
Hope you would have got a clear idea of Defect Triage, Defect Triage Process and ways to effectively handle Defect Triage Meetings!
Recommended Reading
What is Bug?
A bug is the consequence/outcome of a coding fault
What is Defect?
A defect is a variation or deviation from the original business requirements
These two terms have very thin line of differnce, In the Industry both are faults that need to be fixed and so interchangebaly used by some of the Testing teams.
When a tester executes the test cases, he might come across the test result which is contradictory to expected result. This variation in the test result is referred as a Software Defect. These defects or variation are referred by different names in a different organization like issues, problem, bug or incidents.
In this tutorial, you will learn-
Bug Report
While reporting the bug to developer, your Bug Report should contain the following information
- Defect_ID - Unique identification number for the defect.
- Defect Description - Detailed description of the Defect including information about the module in which Defect was found.
- Version - Version of the application in which defect was found.
- Steps - Detailed steps along with screenshots with which the developer can reproduce the defects.
- Date Raised - Date when the defect is raised
- Reference- where in you Provide reference to the documents like . requirements, design, architecture or maybe even screenshots of the error to help understand the defect
- Detected By - Name/ID of the tester who raised the defect
- Status - Status of the defect , more on this later
- Fixed by - Name/ID of the developer who fixed it
- Date Closed - Date when the defect is closed
- Severity which describes the impact of the defect on the application
- Priority which is related to defect fixing urgency. Severity Priority could be High/Medium/Low based on the impact urgency at which the defect should be fixed respectively
Click here if the video is not accessible
Resources
Consider the following as a Test Manager
Your team found bugs while testing the Guru99 Banking project.
After a week the developer responds -
In next week the tester responds
As in the above case, if the defect communication is done verbally, soon things become very complicated. To control and effectively manage bugs you need a defect lifecycle.
Defect Management Process
This topic will guide you on how to apply the defect management process to the project Guru99 Bank website. You can follow the below steps to manage defects.
Discovery
In the discovery phase, the project teams have to discover as many defects as possible, before the end customer can discover it. A defect is said to be discovered and change to status accepted when it is acknowledged and accepted by the developers
In the above scenario, the testers discovered 84 defects in the website Guru99.
Let’s have a look at the following scenario; your testing team discovered some issues in the Guru99 Bank website. They consider them as defects and reported to the development team, but there is a conflict -
In such case, as a Test Manager, what will you do? A) Agree With the test team that its a defect
B) Test Manager takes the role of judge to decide whether the problem is defect or not
C) Agree with the development team that is not a defect
B) Test Manager takes the role of judge to decide whether the problem is defect or not
C) Agree with the development team that is not a defect
InCorrect
In such case, a resolution process should be applied to solve the conflict, you take the role as a judge to decide whether the website problem is a defect or not.
Categorization
Defect categorization help the software developers to prioritize their tasks. That means that this kind of priority helps the developers in fixing those defects first that are highly crucial.
Defects are usually categorized by the Test Manager –
Let’s do a small exercise as following Drag & Drop the Defect Priority Below
1) The website performance is too slow |
2) The login function of the website does not work properly |
3) The GUI of the website does not display correctly on Mobile devices |
4) The website could not remember the user login session |
5) Some links doesn’t work |
No. | Description | Priority | Explanation |
---|---|---|---|
1 | The website performance is too slow | High | The performance bug can cause huge inconvenience to user. |
2 | The login function of the website does not work properly | Critical | Login is one of the main function of the banking website if this feature does not work, it is serious bugs |
3 | The GUI of the website does not display correctly on mobile devices | Medium | The defect affects the user who use Smartphone to view the website. |
4 | The website could not remember the user login session | High | This is a serious issue since the user will be able to login but not be able to perform any further transactions |
5 | Some links doesn’t work | Low | This is an easy fix for development guys and the user can still access the site without these links |
Resolution
Once the defects are accepted and categorized, you can follow the following steps to fix the defect.
- Assignment: Assigned to a developer or other technician to fix, and changed the status to Responding.
- Schedule fixing: The developer side take charge in this phase. They will create a schedule to fix these defects, depend on the defect priority.
- Fix the defect: While the development team is fixing the defects, the Test Manager tracks the process of fixing defect compare to the above schedule.
- Report the resolution: Get a report of the resolution from developers when defects are fixed.
Verification
Software Defect Management Process Definition
After the development team fixed and reported the defect, the testing team verifies that the defects are actually resolved.
For example, in the above scenario, when the development team reported that they already fixed 61 defects, your team would test again to verify these defects were actually fixed or not.
Closure
Once a defect has been resolved and verified, the defect is changed status as closed. If not, you have send a notice to the development to check the defect again.
Reporting
The management board has right to know the defect status. They must understand the defect management process to support you in this project. Therefore, you must report them the current defect situation to get feedback from them.
Important Defect Metrics
Back the above scenario. The developer and test teams have reviews the defects reported. Here is the result of that discussion
How to measure and evaluate the quality of the test execution?
This is a question which every Test Manager wants to know. There are 2 parameters which you can consider as following
In the above scenario, you can calculate the defection rejection ratio (DRR) is 20/84 = 0.238 (23.8 %).
Another example, supposed the Guru99 Bank website has total 64 defects, but your testing team only detect 44 defects i.e. they missed 20 defects. Therefore, you can calculate the defect leakage ratio (DLR) is 20/64 = 0.312 (31.2 %).
Conclusion, the quality of test execution is evaluated via following two parameters
The smaller value of DRR and DLR is, the better quality of test execution is. What is the ratio range which is acceptable? This range could be defined and accepted base in the project target or you may refer the metrics of similar projects.
In this project, the recommended value of acceptable ratio is 5 ~ 10%. It means the quality of test execution is low. You should find countermeasure to reduce these ratios such as
Software Defect Management Process Example
- Improve the testing skills of member.
- Spend more time for testing execution, especially for reviewing the test execution results.