Assessing developer contribution with repository mining-based metrics

Abstract

Productivity as a result of individual developers' con-tributions is an important aspect for software companies to main-tain their competitiveness in the market. However, there is no con-sensus in the literature on how to measure productivity or devel-oper contribution. While some repository mining-based metrics have been proposed, they lack validation in terms of their applica-bility and usefulness from the individuals who will use them to as-sess developer contribution: team and project leaders. In this pa-per, we propose the design of a suite of metrics for the assessment of developer contribution, based on empirical evidence obtained from project and team leaders. In a preliminary evaluation with four software development teams, we found that code contribution and code complexity metrics received the most positive feedback, while participants pointed out several threats of using bug-related metrics for contribution assessment. None of the metrics can be used in isolation, and project leaders and developers need to be aware of the benefits, limitations, and threats of each one. These findings present a first step towards the design of a larger suite of metrics as well as an investigation into the impact of using metrics to assess contribution.