|||"Milestone-Oriented Usage of Key Performance Indicators – An Industrial Case Study", In e-Informatica Software Engineering Journal, vol. 12, no. 1, pp. 217–236, 2018.
DOI: , 10.5277/e-Inf180109.|
Get article (PDF)View article entry (BibTeX)
Miroslaw Staron, Kent Niesel, Niclas Bauman
Background: Key Performance Indicators are a common way of quantitative monitoring of project progress in modern companies. Although they are widely used in practice, there is little evidence on how they are set, and how many of them are used in large product development projects. Goal: The goal of this paper is to explore how KPIs are used in practice in a large company. In particular, it is explored whether KPIs are used continuously or only during short, predefined periods of time. It is also explored whether software-related KPIs are reported differently from non-software-related KPIs. Method: A case study of 12 projects at the Volvo Car Group in Sweden was conducted. The data from the project progress reporting on tools was collected and triangulated with data from interviews conducted with experts from the company. Results: KPIs are reported mostly before the milestones and the manual assessment of their status is equally important as the automated data provision in the KPI reporting system. The trend of reporting software-related KPIs is very similar to the non-software-related KPIs. Conclusions: Despite the documented good practices of using KPIs for project monitoring, it is difficult to develop a clear status-picture solely using quantitative data from progress reporting tools. It was also shown that the trends in reporting the software-related KPIs are similar to the trends in reporting the non-software related KPIs.
software metrics, key performance indicator, project management, case study
 M. Staron, W. Meding, J. Hansson, C. Höglund, K. Niesel, and V. Bergmann, “Dashboards for continuous monitoring of quality for software product under development,” System Qualities and Software Architecture (SQSA), 2013.
 M. Staron, W. Meding, J. Hansson, C. Höglund, K. Niesel, and V. Bergmann, “Dashboards for continuous monitoring of quality for software product under development,” System Qualities and Software Architecture (SQSA), 2014.
 A. Assila, K. Marçal de Oliveira, and H. Ezzedine, “Integration of subjective and objective usability evaluation based on ISO/IEC 15939: A case study for traffic supervision systems,” International Journal of Human–Computer Interaction, Vol. 32, No. 12, 2016, pp. 931–955.
 M. Staron, “Critical role of measures in decision processes: Managerial and technical measures in the context of large software development organizations,” Information and Software Technology, Vol. 54, No. 8, 2012, pp. 887–899.
 M. Staron, J. Hansson, R. Feldt, A. Henriksson, W. Meding, S. Nilsson, and C. Hoglund, “Measuring and visualizing code stability–a case study at three companies,” in Software Measurement and the 2013 Eighth International Conference on Software Process and Product Measurement (IWSM-MENSURA), 2013 Joint Conference of the 23rd International Workshop on. IEEE, 2013, pp. 191–200.
 R. Feldt, M. Staron, E. Hult, and T. Liljegren, “Supporting software decision meetings: Heatmaps for visualising test and code measurements,” in Software Engineering and Advanced Applications (SEAA), 2013 39th EUROMICRO Conference on. IEEE, 2013, pp. 62–69.
 M. Staron, W. Meding, C. Hoglund, and J. Hansson, “Identifying implicit architectural dependencies using measures of source code change waves,” in Software Engineering and Advanced Applications (SEAA), 2013 39th EUROMICRO Conference on. IEEE, 2013, pp. 325–332.
 M.L. Drury-Grogan, “Performance on agile teams: Relating iteration objectives and critical decisions to project management success factors,” Information and Software Technology, Vol. 56, No. 5, 2014, pp. 506–515.
 L. Raymond and F. Bergeron, “Project management information systems: An empirical study of their impact on project managers and project success,” International Journal of Project Management, Vol. 26, No. 2, 2008, pp. 213–220.
 G. Marques, D. Gourc, and M. Lauras, “Multi-criteria performance analysis for decision making in project management,” International Journal of Project Management, Vol. 29, No. 8, 2011, pp. 1057–1069.
 M. Staron, W. Meding, and K. Palm, “Release readiness indicator for mature agile and lean software development projects,” in Agile Processes in Software Engineering and Extreme Programming. Springer, 2012, pp. 93–107.
 J.W. Lainhart IV, “COBITTM: A methodology for managing and controlling information and information technology risks and vulnerabilities,” Journal of Information Systems, Vol. 14, No. s-1, 2000, pp. 21–25.
 V. Basili, J. Heidrich, M. Lindvall, J. Munch, M. Regardie, and A. Trendowicz, “GQM+ Strategies–Aligning Business Strategies with Software Measurement,” in Empirical Software Engineering and Measurement, 2007. ESEM 2007. First International Symposium on. IEEE, 2007, pp. 488–490.
 J. Münch, F. Fagerholm, P. Kettunen, M. Pagels, and J. Partanen, “Experiences and insights from applying GQM+Strategies in a systems product development organisation,” in Software Engineering and Advanced Applications (SEAA), 2013 39th EUROMICRO Conference on. IEEE, 2013, pp. 70–77.