| 2026 | |
| [1] | , "Disambiguating Software Property Descriptions: A Solution Proposal and Evaluation", In e-Informatica Software Engineering Journal, vol. 20, no. 1, pp. 260101, 2026.
DOI: 10.37190/e-Inf260101. Download article (PDF)Get article BibTeX file |
Authors
Efi Papatheocharous, Séverine Sentilles, Kai Petersen, Federico Ciccozzi
Abstract
Context: The state-of-the-art and practice on software quality is growing constantly and presents several challenges. The ever-growing body of knowledge on the topic, obfuscates the situation further, the lack of explicit structure makes it difficult to identify which properties exist today and how they can be evaluated, two critical aspects to increase software quality.
Objective: A step to disambiguate software properties descriptions is made via a Property Model Ontology (PMO) and steps towards the evaluation of the structure are carried out together with experts. The objectives of this paper are: 1) present in detail the PMO, 2) describe the research process used to develop and evaluate the PMO, and, 3) exemplify the usage of the PMO through real instantiations obtained from practitioners and researchers.
Method: Expert interviews and qualitative research methods are used to evaluate the PMO and develop a proof of concept.
Results: The PMO consists of concepts describing extra-functional properties (EFPs) and their evaluation methods, i.e., how to measure the properties. The PMO is instantiated in a modelling environment through a metamodel and an online web-content management system.
Conclusions: Consensus on the definition and structure of EFPs is achieved and a common understanding on how they can be reused in practice.
Keywords
Extra-functional properties, Software engineering, Modelling, Ontology, Software quality, Evaluation
References
1. N. Schneidewind, Standard for a software quality metrics methodology, IEEE Std. 1061-1998, 1998.
2. T. Olsson, S. Sentilles, and E. Papatheocharous, “A systematic literature review of empirical research on quality requirements,” Requirements Engineering, Vol. 27, No. 2, 2022, pp. 249–271.
3. J. Mylopoulos and L. Chung, “Representing and reasoning with non-functional requirements: A retrospective,” IEEE Transactions on Software Engineering, 2025.
4. L. Chung, B.A. Nixon, E. Yu, and J. Mylopoulos, Non-functional requirements in software engineering, Vol. 5. Springer Science and Business Media, 2012.
5. B. Regnell, R.B. Svensson, and T. Olsson, “Supporting roadmapping of quality requirements,” IEEE Software, Vol. 25, No. 2, 2008, pp. 42–47.
6. E. Yu, “Towards modelling and reasoning support for early-phase requirements engineering,” in International Symposium on Requirements Engineering, 1997, pp. 226–235.
7. X. Franch, J.C.S. do Prado Leite, G. Mussbacher, J. Mylopoulos, and A. Perini, Social Modeling Using the i* Framework. Springer, 2024.
8. I.G. Ndukwe, S.A. Licorish, A. Tahir, and S.G. MacDonell, “How have views on software quality differed over time? Research and practice viewpoints,” Journal of Systems and Software, Vol. 195, 2023, p. 111524.
9. S. Sentilles, E. Papatheocharous, F. Ciccozzi, and K. Petersen, “A property model ontology,” in Software Engineering and Advanced Applications (SEAA). IEEE, 2016, pp. 165–172.
10. S. Sentilles, Managing Extra-Functional Properties in Component-Based Development of Embedded Systems, Ph.D. dissertation, Mälardalen University, Västerås, Sweden, 2012.
11. Software engineering – Product quality, ISO/IEC Std. 9126, 2001.
12. S.T. Albin, The art of software architecture: design methods and techniques, Vol. 9. John Wiley and Sons, 2003.
13. J.A. Mccall, P.K. Richards, and G.F. Walters, “Factors in Software Quality. Volume I. Concepts and Definitions of Software Quality.” GENERAL ELECTRIC CO SUNNYVALE CALIF, Technical Report ADA049014, 1977. [Online]. http://stinet.dtic.mil/oai/oai?verb=getRecord∖&metadataPrefix=html∖&identifier=ADA049014
14. B.W. Boehm, J.R. Brown, and M. Lipow, “Quantitative evaluation of software quality,” in Proceedings of the 2nd international conference on Software engineering, ICSE ’76. Los Alamitos, CA, USA: IEEE Computer Society Press, 1976, pp. 592–605. [Online]. http://dl.acm.org/citation.cfm?id=800253.807736
15. Systems and Software Engineering. Systems and Software Quality Requirements and Evaluation (SQuaRE). Product Quality Model, ISO/IEC Std. 25 010:2023.
16. J.C. Laprie, “Dependable computing and fault-tolerance,” Digest of Papers FTCS-15, 1985, pp. 2–11.
17. J. de AG Saraiva, M.S. De França, S.C. Soares, J. Fernando Filho, and R.M. de Souza, “Classifying metrics for assessing object-oriented software maintainability: A family of metrics’ catalogs,” Journal of Systems and Software, Vol. 103, 2015, pp. 85–101.
18. I. Crnkovic, M. Larsson, and O. Preiss, “Concerning predictability in dependable component-based systems: Classification of quality attributes,” 2005.
19. S.S. Thapar, P. Singh, and S. Rani, “Challenges to development of standard software quality model,” International Journal of Computer Applications, Vol. 49, No. 10, 2012.
20. C. Izurieta, D. Reimanis, E. O’Donoghue, K. Liyanage, A.R.M. Muneza et al., “A generalized approach to the operationalization of Software Quality Models,” PeerJ Computer Science, Vol. 10, 2024, p. e2357.
21. I. Crnkovic, S. Sentilles, A. Vulgarakis, and M.R. Chaudron, “A classification framework for software component models,” IEEE Transactions on Software Engineering, Vol. 37, No. 5, 2011, pp. 593–615.
22. M. Glinz, “On non-functional requirements,” in 15th IEEE International Requirements Engineering Conference, 2007, pp. 21–26.
23. F. Deissenboeck, E. Juergens, K. Lochmann, and S. Wagner, “Software quality models: Purposes, usage scenarios and requirements,” in 2009 ICSE Workshop on Software Quality. IEEE, 2009, pp. 9–14.
24. N. Yılmaz and A.K. Tarhan, “Meta-models for software quality and its evaluation: A systematic literature review,” in International Workshop on Software Measurement and the 15th International Conference on Software Process and Product Measurement, Mexico, 2020.
25. M. Broy, F. Deissenboeck, and M. Pizka, “Demystifying maintainability,” in Proceedings of the 2006 International Workshop on Software Quality. ACM, 2006, pp. 21–26.
26. B. Kitchenham, “What’s up with software metrics?–a preliminary mapping study,” Journal of Systems and Software, Vol. 83, No. 1, 2010, pp. 37–51.
27. F. Zhang, A. Mockus, Y. Zou, F. Khomh, and A.E. Hassan, “How does context affect the distribution of software maintainability metrics?” in 2013 IEEE International Conference on Software Maintenance. IEEE, 2013, pp. 350–359.
28. B. Kitchenham and S. Pfleeger, “Software quality: The elusive target,” IEEE Software, Vol. 12, No. 9, 1996.
29. J. Eckhardt, A. Vogelsang, and D.M. Fernández, “Are ”non-functional” requirements really non-functional? an investigation of non-functional requirements in practice,” in 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE). IEEE, 2016, pp. 832–842.
30. N.D. Anh, D.S. Cruzes, R. Conradi, M. Höst, X. Franch et al., “Collaborative resolution of requirements mismatches when adopting open source components,” in International Working Conference on Requirements Engineering: Foundation for Software Quality. Springer, 2012, pp. 77–93.
31. J. Saraiva, S. Soares, and F. Castor, “Towards a catalog of object-oriented software maintainability metrics,” in 2013 4th International Workshop on Emerging Trends in Software Metrics (WETSoM). IEEE, 2013, pp. 84–87.
32. C. Gencel, R. Heldal, and K. Lind, “On the relationship between different size measures in the software life cycle,” in 2009 16th Asia-Pacific Software Engineering Conference. IEEE, 2009, pp. 19–26.
33. C. Wohlin, E. Papatheocharous, J. Carlson, K. Petersen, E. Alégroth et al., “Towards evidence-based decision-making for identification and usage of assets in composite software: A research roadmap,” Journal of Software: Evolution and Process, Vol. 33, No. 6, 2021, p. e2345.
34. B. Henderson-Sellers, “Bridging metamodels and ontologies in software engineering,” Journal of Systems and Software, Vol. 84, No. 2, 2011, pp. 301–313.
35. G. Stănescu and S.V. Oprea, “Recent trends and insights in semantic web and ontology-driven knowledge representation across disciplines using topic modeling,” Electronics, Vol. 14, No. 7, 2025, p. 1313.
36. C. Antoniou and N. Bassiliades, “A tool for requirements engineering using ontologies and boilerplates,” Automated Software Engineering, Vol. 31, No. 1, 2024, p. 5.
37. N. Guarino, Formal ontology in information systems: Proceedings of the first international conference (FOIS’98), June 6-8, Trento, Italy, Vol. 46. IOS press, 1998.
38. D. Fensel, Ontologies: A silver bullet for knowledge management and electronic-commerce. Berlin: Spring-Verlag, 2004.
39. G. Van Heijst, A.T. Schreiber, and B.J. Wielinga, “Using explicit ontologies in KBS development,” International Journal of Human-Computer Studies, Vol. 46, No. 2-3, 1997, pp. 183–292.
40. G. Guizzardi, M. Lopes, F. Baião, and R. Falbo, “On the importance of truly ontological distinctions for ontology representation languages: An industrial case study in the domain of oil and gas,” in Enterprise, Business-Process and Information Systems Modeling. Springer, 2009, pp. 224–236.
41. G. Guizzardi, F. Baião, M. Lopes, and R. Falbo, “The role of foundational ontologies for domain ontology engineering: An industrial case study in the domain of oil and gas exploration and production,” International Journal of Information System Modeling and Design (IJISMD), Vol. 1, No. 2, 2010, pp. 1–22.
42. G. Guizzardi, “On ontology, ontologies, conceptualizations, modeling languages, and (meta) models,” Frontiers in Artificial Intelligence and Applications, Vol. 155, 2007, p. 18.
43. G. Guizzardi, Ontological foundations for structural conceptual models, Ph.D. dissertation, University of Twente, 2005. [Online]. https://research.utwente.nl/en/publications/ontological-foundations-for-structural-conceptual-models/
44. A. Gangemi, N. Guarino, C. Masolo, A. Oltramari, and L. Schneider, “Sweetening ontologies with dolce,” in International Conference on Knowledge Engineering and Knowledge Management. Springer, 2002, pp. 166–181.
45. M.F. Bertoa, A. Vallecillo, and F. García, “An ontology for software measurement,” in Ontologies for software engineering and software technology. Springer, 2006, pp. 175–196.
46. J.R. Hilera and L. Fernández-Sanz, “Developing domain-ontologies to improve sofware engineering knowledge,” in International Conference on Software Engineering Advances (ICSEA). IEEE, 2010, pp. 380–383.
47. M.J. Blas, S. Gonnet, and H. Leone, “An ontology to document a quality scheme specification of a software product,” Expert Systems, Vol. 34, No. 5, 2017, p. e12213.
48. N. Anquetil, K.M. de Oliveira, and M.G. Dias, “Software maintenance ontology,” in Ontologies for Software Engineering and Software Technology. Springer, 2006, pp. 153–173.
49. O. Lassila and D. McGuinness, “The role of frame-based representation on the semantic web,” Linköping Electronic Articles in Computer and Information Science, Vol. 6, No. 5, 2001, p. 2001.
50. N.F. Noy, D.L. McGuinness et al., “Ontology development 101: A guide to creating your first ontology,” 2001.
51. W.A. Conklin, “Software assurance: The need for definitions,” in International Conference on System Sciences (HICSS). IEEE, 2011, pp. 1–7.
52. S. Sentilles, E. Papatheocharous, and F. Ciccozzi, “What do we know about software security evaluation? A preliminary study,” in 6th International Workshop on Quantitative Approaches to Software Quality, APSEC, 2018.
53. S. Henninger and P. Ashokkumar, “An ontology-based metamodel for software patterns,” University of Nebraska – Lincoln, Tech. Rep. TR-UNL-CSE-2006-00005, 2006.
54. F. Béhé, S. Galland, N. Gaud, C. Nicolle, and A. Koukam, “An ontology-based metamodel for multiagent-based simulations,” Simulation Modelling Practice and Theory, Vol. 40, 2014, pp. 64–85.
55. S. Sentilles, E. Papatheocharous, F. Ciccozzi, and K. Petersen, “Property models for the automotive domain,” Mälardalen University, Tech. Rep., 2016. [Online]. http://www.es.mdh.se/publications/4295-
56. J. Brank, M. Grobelnik, and D. Mladenić, “A survey of ontology evaluation techniques,” in Proceedings of the Conference on Data Mining and Data Warehouses, 2005, pp. 166–170.
57. A. Burton-Jones, V.C. Storey, V. Sugumaran, and P. Ahluwalia, “A semiotic metrics suite for assessing the quality of ontologies,” Data and Knowledge Engineering, Vol. 55, No. 1, 2005, pp. 84–102.
58. T. Gorschek, P. Garre, S. Larsson, and C. Wohlin, “A model for technology transfer in practice,” IEEE Software, Vol. 23, No. 6, 2006, pp. 88–95. [Online]. https://doi.org/10.1109/MS.2006.147
59. S. Sentilles, E. Papatheocharous, and F. Ciccozzi, “What do we know about software security evaluation? a preliminary study,” in 6th International Workshop on Quantitative Approaches to Software Quality, 2018. [Online]. http://www.es.mdh.se/publications/5277-
60. J. Maxwell, “Understanding and validity in qualitative research,” Harvard Educational Review, Vol. 62, No. 3, 1992, pp. 279–301.
61. K. Petersen and C. Gencel, “Worldviews, research methods, and their relationship to validity in empirical software engineering research,” in Joint Conference of the 23rd International Workshop on Software Measurement and the Eighth International Conference on Software Process and Product Measurement (IWSM-MENSURA). IEEE, 2013, pp. 81–89.
62. J. Axelsson, “Evolutionary architecting of embedded automotive product lines: An industrial case study,” in European Conference on Software Architecture, 2009, pp. 101–110.
63. H. Gustavsson and U. Eklund, “Architecting automotive product lines: Industrial practice,” in Software Product Lines: Going Beyond. Springer, 2010, pp. 92–105.
64. R.A. McGee, U. Eklund, and M. Lundin, “Stakeholder identification and quality attribute prioritization for a global Vehicle Control System,” in Proceedings of the Fourth European Conference on Software Architecture: Companion Volume. ACM, 2010, pp. 43–48.
65. U. Eklund and J. Bosch, “Architecture for embedded open software ecosystems,” Journal of Systems and Software, Vol. 92, 2014, pp. 128–142.
66. M. Broy, “Challenges in automotive software engineering,” in Proceedings of the 28th International Conference on Software Engineering, ICSE ’06. New York, NY, USA: ACM, 2006, pp. 33–42.
67. A. Pretschner, M. Broy, I.H. Kruger, and T. Stauner, “Software engineering for automotive systems: A roadmap,” in Future of Software Engineering, FOSE ’07. Washington, DC, USA: IEEE Computer Society, 2007, pp. 55–71.
68. R. Etemaadi, K. Lind, R. Heldal, and M.R. Chaudron, “Quality-driven optimization of system architecture: Industrial case study on an automotive sub-system,” Journal of Systems and Software, Vol. 86, No. 10, 2013, pp. 2559–2573.
69. C. Ebert and C. Jones, “Embedded software: Facts, figures, and future,” Computer, Vol. 42, No. 4, 2009, pp. 42–52.
70. B.W. Boehm, Software Engineering Economics, 1st ed. Prentice Hall PTR, 1981.
71. J. Li, G. Ruhe, A. Al-Emran, and M.M. Richter, “A flexible method for software effort estimation by analogy,” Empirical Software Engineering, Vol. 12, No. 1, 2007, pp. 65–106.
72. M. Jørgensen, “A review of studies on expert estimation of software development effort,” Journal of Systems and Software, Vol. 70, No. 1–2, 2004, pp. 37–60.
73. S.L. Pfleeger, F. Wu, and R. Lewis, “Software cost estimation and sizing methods: issues, and guidelines, Volume 269, Rand Comporation,” http://www.rand.org/content/dam/rand/pubs/monographs/2005/RAND/MG269.pdf, 2005.
74. V. Chapurlat, B. Kamsu-Foguem, and F. Prunet, “Enterprise model verification and validation: An approach.” Annual Reviews in Control, Vol. 27, No. 2, 2003, pp. 185–197.
75. R.S. Pressman, A manager’s guide to software engineering. McGraw-Hill, Inc., 1993.
76. R. Broek, J. Gorman, O. Haugen, G. Melby, B. Moller-Pedersen et al., “Quality by construction exemplified by TIMe-the integrated methodology,” Telektronikk, Vol. 95, No. 1, 1999, pp. 73–82.
77. R. Morgan, Component library retrieval using property models, Ph.D. dissertation, Durham University, 1991.
78. M. Belguidoum and F. Dagnat, “Dependability in software component deployment,” in Dependability of Computer Systems, 2007, pp. 223–230.
79. A. Billig, S. Busse, A. Leicher, and J.G. Süß, “Platform independent model transformation based on triple,” in Middleware 2004. Springer, 2004, pp. 493–511.
80. V.A. Paun, Precise and Adaptable Worst-Case Execution Time Estimation in Hard Real-Time Systems, Ph.D. dissertation, Ecole Doctorale Polytechnique, 2014.
81. I. Wenzel, R. Kirner, B. Rieder, and P. Puschner, “Measurement-based worst-case execution time analysis,” in Third IEEE Workshop on Software Technologies for Future Embedded and Ubiquitous Systems (SEUS’05), 2005, pp. 7–10.
82. A. Ermedahl, A modular tool architecture for worst-case execution time analysis, Ph.D. dissertation, Acta Universitatis Upsaliensis, 2003.
83. S. Sentilles, F. Ciccozzi, and E. Papatheocharous, “PROMOpedia: A web-content management-based encyclopedia of software property models,” in 40th International Conference on Software Engineering – Demonstrations track, ICSE, 2018. [Online]. http://www.es.mdh.se/publications/5086-