Organizing the state of practice on technical debt prevention, monitoring, and payment in software projects

Abstract

Context: Technical debt (TD) describes the effects of immature artifacts on software development that can bring benefits in the short term but may have to be paid with interest in the long term. TD management balances short-term and long-term goals, supporting development teams to decide on the need and the best time to eliminate the debt. TD management activities include prevention, monitoring, and payment. Through prevention, it is possible to prevent teams from incurring TD, while monitoring helps them follow the evolution of TD items concerning the cost-benefit of eliminating them or not, that is, paying the debt items. Knowing the practices used to prevent, monitor, and pay TD items can help development teams to choose the best practice to be used in their projects. Identifying the practice avoidance reasons (PARs) that lead to non-prevention, non-monitoring, and non-payment of TD can help teams understand which aspects need to be improved to enable TD management. Although the technical literature has investigated the prevention, monitoring, and payment of TD, current results only reflect the viewpoint of a small number of professionals and organizations. To achieve the benefits of TD management, it is necessary to investigate more deeply the practices and PARs associated with these TD activities. Aims: This Ph.D. dissertation aims to investigate, through the continuous and independent replication of a family of surveys conducted globally, the state of practice on the prevention, monitoring, and payment of TD items in software projects. Method: Initially, we conducted a literature review on the current state of research on TD and its prevention, monitoring, and payment. Then, we analyzed data collected by six replication teams from the InsighTD project, which is a family of globally distributed surveys on the causes, effects, and management of TD. From the body of knowledge resulted from the analysis of InsighTD data, we defined three artifacts: an updated version of the conceptual model for TD, a set of conceptual maps, and IDEA (Impediments, Decision factors, Enabling practices, and Actions) diagrams. Finally, we assessed these artifacts through empirical studies in academic and industrial settings. Results: This Ph.D. dissertation presents the leading practices used to prevent, monitor, and pay off TD items and the PARs that justify the non-application of these practices. Regarding the prevention of TD, well-defined requirements, adopting good programming practices, and better project management are among the five most cited practices related to prevention, while short deadlines, ineffective management, and lack of predictability in the software development are among the five most cited PARs to justify the non-prevention of debt. About TD monitoring, TD item backlog, use of tools, and team meetings are among the five most cited practices related to monitoring, while lack of interest, focus on short-term goals, and lack of time are among the five PARs used to explain the non-monitoring of TD items. Regarding TD payment, code refactoring, investing effort in TD payment activities, and design refactoring are among the top five payment-related practices, while focusing on short-term goals, lack of organizational interest, and lack of time are among the five most cited PARs to explain the non-payment of TD. We update the conceptual model for TD by including the knowledge we learn from the state of practice and organize all practices and PARs along with their types, natures, and categories into maps and IDEA diagrams. From the conceptual model and TD payment map assessment, we found that they are well organized and provide valuable information to define strategies for TD management. The IDEA diagrams assessment provided positive evidence that the diagrams are easy to read and follow and can influence decisions on how to manage TD items. Conclusion: Using the InsighTD data, this Ph.D. dissertation explores the state of practice on TD prevention, monitoring, and payment, revealing the primary practices used to perform these activities and the PARs that avoid their execution. All body of knowledge was organized into three artifacts that can drive new investigations on TD and support software practitioners in increasing their capabilities and reducing their issues in managing debt items.