|||"A Systematic Mapping Study on Software Measurement Programs in SMEs", In e-Informatica Software Engineering Journal, vol. 12, no. 1, pp. 133–165, 2018. ,|
Touseef Tahir, Ghulam Rasool, Muhammad Noman
Context: Software measurement programs are essential to understand, evaluate, improve and predict the software processes, products and resources. However, successful implementation of software measurement programs (MPs) in small and medium enterprises (SMEs) is challenging.
Objective: To perform a detailed analysis of studies on MPs for highlighting the existing measurement models, tools, metrics selection methods and challenges for implementing MPs in SMEs.
Methods: A Systematic Mapping Study (SMS) is conducted.
Results: In total, 35 primary studies are comprehensively analyzed. We identified 29 software measurement models and 4 tools specifically designed for MPs in SMEs. Majority of the measurement models (51%) are built upon software process improvement approaches. With respect to measurement purposes of models, the distribution of MPs was identified as: characterization (63%), evaluation (83%), improvement (93%) and prediction (16%). Majority of primary studies discussed the use of measurement experts and experience (60%) followed by the use of measurement standards (40% and the use of automated tools (22%) for metrics selection in MPs. We found that the SMEs and large organization face different challenges as studies in SMEs report challenges that exist even before the implementation of MPs due to infrastructure and management processes of SMEs. The challenges reported by studies in large organizations are mostly related to the issues discovered while implementing MPs.
Conclusion: The analysis of measurement models, tools, metrics selection methods and challenges of implementing MPs should help the SMEs to make a feasibility study before implementing a MP.
 M. Khalique, N. Bontis, J. Abdul Nassir bin Shaari, and A. Hassan
Md. Isa, “Intellectual capital in small and medium enterprises in Pakistan,”
Journal of Intellectual Capital, Vol. 16, No. 1, 2015, pp. 224–238.
 M. Díaz-Ley, F. García, and M. Piattini, “MIS-PyME software
measurement capability maturity model – Supporting the definition of
software measurement programs and capability determination,” Advances in
Engineering Software, Vol. 41, No. 10, 2010, pp. 1223–1237.
 A.M. Bhatti, H.M. Abdullah, and C. Gencel, “A model for selecting an
optimum set of measures in software organizations,” in European Conference
on Software Process Improvement. Springer, 2009, pp. 44–56.
 M.K. Sharma, R. Bhagwat, and G.S. Dangayach, “Practice of
performance measurement: experience from indian SMEs,” International
Journal of Globalisation and Small Business, Vol. 1, No. 2, 2005, pp.
 V. Claudia, M. Mirna, and M. Jezreel, “Characterization of software
processes improvement needs in SMEs,” in International Conference on
Mechatronics, Electronics and Automotive Engineering (ICMEAE. IEEE,
2013, pp. 223–228.
 O. Gómez, H. Oktaba, M. Piattini, and F. García, “A systematic
review measurement in software engineering: State-of-the-art in measures,”
in International Conference on Software and Data Technologies. Springer,
2006, pp. 165–176.
 D. Radjenović, M. Heričko, R. Torkar, and A. Živkovič, “Software
fault prediction metrics: A systematic literature review,” Information and
Software Technology, Vol. 55, No. 8, 2013, pp. 1397–1418.
engineering,” IEEE Transactions on Software Engineering, Vol. 38, No. 6,
2012, pp. 1276–1304.
 M. Unterkalmsteiner, T. Gorschek, A.M. Islam, C.K. Cheng, R.B.
Permadi, and R. Feldt, “Evaluation and measurement of software process
improvementóa systematic literature review,” IEEE Transactions on
Software Engineering, Vol. 38, No. 2, 2012, pp. 398–424.
 K. Petersen, S. Vakkalanka, and L. Kuzniarz, “Guidelines for
conducting systematic mapping studies in software engineering: An update,”
Information and Software Technology, Vol. 64, 2015, pp. 1–18.
 F. García, M.F. Bertoa, C. Calero, A. Vallecillo, F. Ruíz,
M. Piattini, and M. Genero, “Towards a consistent terminology for software
measurement,” Information and Software Technology, Vol. 48, No. 8, 2006,
 Ø. Moen, M. Gavlen, and I. Endresen, “Internationalization of small,
computer software firms: Entry forms and market selection,” European
Journal of Marketing, Vol. 38, No. 9/10, 2004, pp. 1236–1251.
 A. Boyd, “The goals, questions, indicators, measures (GQIM) approach
to the measurement of customer satisfaction with e-commerce Web sites,” in
Aslib proceedings, Vol. 54. MCB UP Ltd, 2002, pp. 177–187.
 A. Abran, L. Buglione, and A. Sellami, “Software measurement body of
knowledge – initial validation using Vincenti’s classification of engineering
knowledge types,” in Software Measurement Conference, 2004, pp. 1–16.
 S. Alexandre, A. Renault, and N. Habra, “POWPL: A gradual
approach for software process improvement in SMEs,” in 32nd EUROMICRO
Conference on Software Engineering and Advanced Applications. IEEE, 2006,
 A.H. Lee, W.C. Chen, and C.J. Chang, “A fuzzy AHP and BSC approach
for evaluating performance of IT department in the manufacturing industry
in Taiwan,” Expert Systems with Applications, Vol. 34, No. 1, 2008, pp.
 C. Wang, E. Walker, and J. Redmond, “Explaining the lack of strategic
planning in SMEs: The importance of owner motivation,” International
Journal of Organisational Behaviour, Vol. 12, No. 1, 2007, pp. 1–16.
 M. Hudson, A. Smart, and M. Bourne, “Theory and practice in SME
performance measurement systems,” International Journal of Operations &
Production Management, Vol. 21, No. 8, 2001, pp. 1096–1115.
 Systems and software engineering – Systems and
software Quality Requirements and Evaluation (SQuaRE) – Quality measure
elements, International Organization for Standardization Standard ISO/IEC
 “Capability maturity model integration (CMMI)
(continuous representation),” Carnegie Mellon University,
Tech. Rep. ICMU/SEI-2002-TR-011, 2002. [Online].
(staged representation),” Carnegie Mellon University,
Tech. Rep. CMU/SEI-2002-TR-012, SEI, 2002. [Online].
 F.G. Wilkie, D. McFall, and F. McCaffery, “An evaluation of
CMMI process areas for small-to medium-sized software development
organisations,” Software Process: Improvement and Practice, Vol. 10, No. 2,
2005, pp. 189–201.
 T. Tahir and C. Gencel, “A structured goal based measurement
framework enabling traceability and prioritization,” in 6th International
Conference on Emerging Technologies (ICET). IEEE, 2010, pp. 282–286.
 C. Gencel, K. Petersen, A.A. Mughal, and M.I. Iqbal, “A decision
support framework for metrics selection in goal-based measurement
programs: GQM-DSFMS,” Journal of Systems and Software, Vol. 86,
No. 12, 2013, pp. 3091–3108.
Measurement (ESEM). IEEE, 2011, pp. 49–57.
 P. Carbone, L. Buglione, L. Mari, and D. Petri, “A comparison between
foundations of metrology and software measurement,” IEEE Transactions on
Instrumentation and Measurement, Vol. 57, No. 2, 2008, pp. 235–241.
 M. Díaz-Ley, F. García, and M. Piattini, “Software measurement
programs in SMEs–defining software indicators: A methodological
framework,” Product-Focused Software Process Improvement, 2007, pp.
 M. Diaz-Ley, F. García, and M. Piattini, “MIS-PyME software
measurement maturity model-supporting the definition of software
measurement programs,” Product-Focused Software Process Improvement,
2008, pp. 19–33.
 A. Tosun, A. Bener, and B. Turhan, “Implementation of a software
quality improvement project in an SME: A before and after comparison,”
in 35th Euromicro Conference on Software Engineering and Advanced
Applications. IEEE, 2009, pp. 203–209.
 E. Amrina and S.M. Yusof, “A proposed manufacturing performance
measures for small and medium-sized enterprises (SMEs),” in Proceedings
of the 10th Asia Pacific Industrial Engineering and Management Systems
(APIEMS) Conference, 2009, pp. 623–629.
 J.A.C.M. Villalón, G.C. Agustín, T.S.F. Gilabert, A.D.A. Seco, L.G.
Sánchez, and M.P. Cota, “Experiences in the application of software process
improvement in SMEs,” Software Quality Journal, Vol. 10, No. 3, 2002, pp.
 F. McCaffery, M. Pikkarainen, and I. Richardson, “AHAA–Agile,
hybrid assessment method for automotive, safety critical SMEs,” in
Proceedings of the 30th International Conference on Software Engineering.
ACM, 2008, pp. 551–560.
 F.T. Shah, S. Shamail, and N. Ahmad Akhtar, “Lean quality
improvement model for quality practices in software industry in Pakistan,”
Journal of Software: Evolution and Process, Vol. 27, No. 4, 2015, pp.
 S. Bibi, I. Stamelos, G. Gerolimos, and V. Kollias, “BBN based
approach for improving the software development process of an SME –
A case study,” Journal of Software: Evolution and Process, Vol. 22, No. 2,
 E. Caballero, J.A. Calvo-Manzano, and T. San Feliu, “Introducing
scrum in a very small enterprise: A productivity and quality analysis,”
Systems, Software and Service Process Improvement, 2011, pp. 215–224.
 C.G. von Wangenheim, T. Punter, and A. Anacleto, “Software
measurement for small and medium enterprises,” in Proceeding 7th
International Conference on Empirical Assessment in Software Engineering
 M. Sulayman, C. Urquhart, E. Mendes, and S. Seidel, “Software
process improvement success factors for small and medium Web companies:
A qualitative study,” Information and Software Technology, Vol. 54, No. 5,
2012, pp. 479–500.
 N. Ohsugi, K. Fushida, N. Inoguchi, H. Arai, H. Yamanaka, T. Niwa,
M. Fujinuki, M. Tomura, and T. Kitani, “Using trac for empirical data
collection and analysis in developing small and medium-sized enterprise
systems,” in ACM/IEEE International Symposium on Empirical Software
Engineering and Measurement (ESEM). IEEE, 2015, pp. 1–9.
 H.M. Haddad and D.E. Meredith, “Instituting software metrics in small
organizations: A practical approach,” in Eighth International Conference
on Information Technology: New Generations (ITNG). IEEE, 2011, pp.
 A. Potter, P. Childerhouse, R. Banomyong,
and N. Supatn, “Developing a supply chain performance tool for smes in
thailand,” Supply Chain Management: An International Journal, Vol. 16,
No. 1, 2011, pp. 20–31.
 A. Janes, V. Lenarduzzi, and A.C. Stan, “A continuous software quality
monitoring approach for small and medium enterprises,” in Proceedings of the
8th ACM/SPEC on International Conference on Performance Engineering
Companion. ACM, 2017, pp. 97–100.
H. Ayed, N. Habra, and B. Vanderose, “AM-QuICk: A measurement-based
framework for Agile methods customisation,” in Software Measurement and
the 2013 Eighth International Conference on Software Process and Product
Measurement (IWSM-MENSURA). IEEE, 2013, pp. 71–80.
 M. Díaz, F. García, and M. Piattini, “Defining, performing and
maintaining software measurement programs: State of the art,” in IV
Simposio Internacional de Sistemas de Informacióne, 2006, p. 13.
 J.C. Ruiz, Z.B. Osorio, J. Mejia, M. Muñoz, A.M. Ch, B.A. Olivares
et al., “Definition of a hybrid measurement process for the models ISO/IEC
15504 – ISO/IEC 12207:2008 and CMMI Dev 1.3 in SMEs,” in Electronics,
Robotics and Automotive Mechanics Conference (CERMA). IEEE, 2011, pp.
An experience,” in Ninth Electronics, Robotics and Automotive Mechanics
Conference (CERMA). IEEE, 2012, pp. 141–146.
 F.J. Pino, F. Garcia, and M. Piattini, “Key processes to start software
process improvement in small companies,” in Proceedings of the 2009 ACM
symposium on Applied Computing. ACM, 2009, pp. 509–516.
 R. Chalmeta, S. Palomero, and M. Matilla, “Methodology to develop a
performance measurement system in small and medium-sized enterprises,”
International Journal of Computer Integrated Manufacturing, Vol. 25,
No. 8, 2012, pp. 716–740.
 D. Homchuenchom, C. Piyabunditkul, H. Lichter, and T. Anwar,
“SPIALS: A light-weight software process improvement self-assessment
tool,” in 5th Malaysian Conference in Software Engineering (MySEC). IEEE,
2011, pp. 195–199.