| 2026 | |
| [1] | , "A Defect Classification Framework for AI-Based Software Systems (AIODC)", In e-Informatica Software Engineering Journal, vol. 20, no. 1, pp. 260102, 2026.
DOI: 10.37190/e-Inf260102. Download article (PDF)Get article BibTeX file |
Authors
Mohammed Alannsary
Abstract
Context: Artificial Intelligence (AI) is increasingly integrated into critical domains, making defect analysis essential to ensure system quality and reliability. Current defect classification frameworks do not adequately address the unique properties of AI systems.
Objective: This paper proposes AIODC, a defect classification framework inspired by the Orthogonal Defect Classification (ODC) that incorporates AI-specific characteristics.
Method: The framework extends ODC by introducing three new attributes – Data, Learning, and Thinking – and adds a ŞCatastrophic” severity level to account for risks associated with AI. Additionally, it modifies impact mapping utilizing AI/AIP quality models. The methodology was validated through a case study that examined 42 actual Keras defects.
Results: This study demonstrated the feasibility of modifying ODC for AI systems to classify its defects. The case study indicated that defects occurring during the Learning phase are the most prevalent and were significantly linked to high severity, whereas defects in the Thinking phase primarily impact trustworthiness and accuracy.
Conclusions: The results affirm the practicality and significance of AIODC in identifying high-risk defect categories, thus facilitating more focused and effective quality assurance strategies in AI-driven software systems.
Keywords
Software testing, analysis and verification, Software Quality, AI and knowledge based software engineering
References
1. M.O. Alannsary, Quality improvement of SaaS (Software as a Service) in the Cloud , Ph.D. dissertation, Southern Methodist University, 2016.
2. J. Tian, Software quality engineering: testing, quality assurance, and quantifiable improvement . John Wiley & Sons, 2005.
3. P.J. Denning, “What is software quality?” Communications of the ACM , Vol. 35, No. 1, 1992, pp. 13–15.
4. R. Chillarege, I.S. Bhandari, J.K. Chaar, M.J. Halliday, D.S. Moebus et al., “Orthogonal defect classification-a concept for in-process measurements,” IEEE Transactions on software Engineering , Vol. 18, No. 11, 1992, pp. 943–956.
5. L. Ma and J. Tian, “Web error classification and analysis for reliability improvement,” Journal of Systems and Software , Vol. 80, No. 6, 2007, pp. 795–804.
6. M. Alannsary and J. Tian, “Cloud-odc: Defect classification and analysis for the cloud,” in Proceedings of the International Conference on Software Engineering Research and Practice (SERP) . The Steering Committee of The World Congress in Computer Science, Computer …, 2015, p. 71.
7. S. Martínez-Fernández, J. Bogner, X. Franch, M. Oriol, J. Siebert et al., “Software engineering for ai-based systems: a survey,” ACM Transactions on Software Engineering and Methodology (TOSEM) , Vol. 31, No. 2, 2022, pp. 1–59.
8. S.C.G. Varma, “Bridging the gap between traditional software engineering and modern ai development practices,” International Journal of Emerging Trends in Computer Science and Information Technology , Vol. 4, No. 3, 2023, pp. 32–40.
9. B. Gezici and A.K. Tarhan, “Systematic literature review on software quality for ai-based software,” Empirical Software Engineering , Vol. 27, No. 3, 2022, p. 66.
10. M.A. Ali, N.K. Yap, A.A.A. Ghani, H. Zulzalil, N.I. Admodisastro et al., “A systematic mapping of quality models for ai systems, software and components,” Applied Sciences , Vol. 12, No. 17, 2022, p. 8700.
11. V. Kharchenko, H. Fesenko, and O. Illiashenko, “Quality models for artificial intelligence systems: characteristic-based approach, development and application,” Sensors , Vol. 22, No. 13, 2022, p. 4865.
12. V. Lenarduzzi, F. Lomio, S. Moreschini, D. Taibi, and D.A. Tamburri, “Software quality for ai: Where we are now?” in Software Quality: Future Perspectives on Software Engineering Quality: 13th International Conference, SWQD 2021, Vienna, Austria, January 19–21, 2021, Proceedings 13 . Springer, 2021, pp. 43–53.
13. L. Bailey, The Impact of AI on Software Development , Ph.D. dissertation, Doctoral dissertation, Worcester Polytechnic Institute, 2024.
14. H. Wang and L. Yuan, “Software engineering defect detection and classification system based on artificial intelligence,” Nonlinear Engineering , Vol. 11, No. 1, 2022, pp. 380–386.
15. M. Pandit, D. Gupta, D. Anand, N. Goyal, H.M. Aljahdali et al., “Towards design and feasibility analysis of depaas: Ai based global unified software defect prediction framework,” Applied Sciences , Vol. 12, No. 1, 2022, p. 493.
16. A. Tosun, A. Bener, and R. Kale, “Ai-based software defect predictors: Applications and benefits in a case study,” in Proceedings of the AAAI conference on artificial intelligence , Vol. 24, No. 2, 2010, pp. 1748–1755.
17. J. Pachouly, S. Ahirrao, K. Kotecha, G. Selvachandran, and A. Abraham, “A systematic literature review on software defect prediction using artificial intelligence: Datasets, data validation methods, approaches, and tools,” Engineering Applications of Artificial Intelligence , Vol. 111, 2022, p. 104773.
18. R.G. Hussain, K.C. Yow, and M. Gori, “Leveraging an enhanced codebert-based model for multiclass software defect prediction via defect classification,” IEEE Access , 2025.
19. H.V. Pandhare, “Future of software test automation using ai/ml,” International Journal Of Engineering And Computer Science , Vol. 13, No. 05, 2025.
20. X. Dong, J. Wang, and Y. Liang, “A novel ensemble classifier selection method for software defect prediction,” IEEE Access , 2025.
21. S. Stradowski and L. Madeyski, ““your ai is impressive, but my code does not have any bugs” managing false positives in industrial contexts,” Science of Computer Programming , 2025, p. 103320.
22. X. Chen, X. Hu, Y. Huang, H. Jiang, W. Ji et al., “Deep learning-based software engineering: progress, challenges, and opportunities,” Science China Information Sciences , Vol. 68, No. 1, 2025, pp. 1–88.
23. A. Güzel and A. Egesoy, “Development of an ai-driven model for advancing software engineering practices,” International Journal of Innovative Research in Computer Science and Technology , Vol. 13, No. 1, 2025, pp. 1–11.
24. B. Vinayagasundaram and S. Srivatsa, “Software quality in artificial intelligence system,” Information Technology Journal , Vol. 6, No. 6, 2007, pp. 835–842.
25. C. Tao, J. Gao, and T. Wang, “Testing and quality validation for ai software–perspectives, issues, and practices,” IEEE Access , Vol. 7, 2019, pp. 120 164–120 175.
26. M.M. Morovati, A. Nikanjam, F. Khomh, and Z.M. Jiang, “Bugs in machine learning-based systems: a faultload benchmark,” Empirical Software Engineering , Vol. 28, No. 3, 2023, p. 62.
27. T. Zhang, D. Han, V. Vinayakarao, I.C. Irsan, B. Xu et al., “Duplicate bug report detection: How far are we?” ACM Transactions on Software Engineering and Methodology , Vol. 32, No. 4, 2023, pp. 1–32.
28. R.A. Radice and R.W. Phillips, Software engineering: An industrial approach. Vol. 1 . Prentice-Hall, Inc., 1988.
29. V. Lenarduzzi and M. Isomursu, “Ai living lab: Quality assurance for ai-based health systems,” in 2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN) , 2023, pp. 86–87.
30. International Organization for Standardization, Medical devices — Application of risk management to medical devices , Std. ISO 14 971:2019, 2019.
31. D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman et al., “Concrete problems in ai safety,” arXiv preprint arXiv:1606.06565 , 2016.