| 2021 | |
| [1] | , "Mining Non-Functional Requirements using Machine Learning Techniques", In e-Informatica Software Engineering Journal, vol. 15, no. 1, pp. 85–114, 2021.
DOI: 10.37190/e-Inf210105. Download article (PDF)Get article BibTeX file |
Authors
Rajni Jindal, Ruchika Malhotra, Abha Jain, Ankita Bansal
Abstract
Background: Non-Functional Requirements (NFR) have a direct impact on the architecture of the system, thus it is essential to identify NFRs in the initial phases of software development.
Aim: The work is based on extraction of relevant keywords from NFR descriptions by employing text mining steps and thereafter classifying these descriptions into one of the nine types of NFRs.
Method: For each NFR type, keywords are extracted from a set of pre-categorized specifications using Information-Gain measure. Then models using 8 Machine Learning (ML) techniques are developed for classification of NFR descriptions. A set of 15 projects (containing 326 NFR descriptions) developed by MS students at DePaul University are used to evaluate the models.
Results: The study analyzes the performance of ML models in terms of classification and misclassification rate to determine the best model for predicting each type NFR descriptions. The Naïve Bayes model has performed best in predicting “maintainability” and “availability” type of NFRs.
Conclusion: The NFR descriptions should be analyzed and mapped into their corresponding NFR types during the initial phases. The authors conducted cost benefit analysis to appreciate the advantage of using the proposed models.
Keywords
requirement engineering, text mining, non-functional requirements, machine learning, receiver operating characteristics
References
1. M. Glinz, “On non-functional requirements,” in Proceedings of the 15th IEEE International Requirements Engineering Conference . Delhi, India: IEEE, 2010, pp. 21–26.
2. M. Glinz, “Rethinking the notion of non-functional requirements,” in Proceedings of the 3rd world congress for software quality , Munich, Germany, 2005, pp. 55–64.
3. L. Chung, B. Nixon, E. Yu, and J. Mylopoulos, Non-Functional Requirements in Software Engineering . KluwerAcademic, 2000.
4. S. Amasaki and P. Leelaprute, “The effects of vectorization methods on non-functional requirements classification,” in 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) . Prague, Czech Republic: IEEE, 2018, pp. 175–182.
5. V. Sharma, R. Ramnani, and S. Sengupta, “A framework for identifying and analysing non-functional requirements from text,” in Proceedings of the 4th International Workshop on Twin Peaks of Requirements and Architecture . New York, USA: ACM, 2014, pp. 1–8.
6. L. Hakim and S. Rochimah, “Oversampling imbalance data: Case study on functional and non functional requirement,” in Electrical Power, Electronics, Communications, Controls and Informatics Seminar (EECCIS) . Batu, East Java, Indonesia: IEEE, 2018, pp. 315–319.
7. D. Méndez-Fernández, S. Wagner, M. Kalinowski, M. Felderer, P. Mafra, A. Vetro, T. Conte, M. Christiansson, D. Greer, C. Lassenius, T. Mannisto, M. Nayebi, M. Oivo, B. Penzenstadler, D. Pfahl, R. Prikladnicki, G. Ruhe, A. Schekelmann, S. Sen, R. Spinola, A. Tuzcu, J. de la Vara, and R.Wieringa, “Naming the pain in requirements engineering,” Empirical software engineering , Vol. 22, No. 5, 2017, pp. 2298–2338.
8. R. Svensson, M. Host, and B. Regnell, “Managing quality requirements: A systematic review,” in 36th EUROMICRO Conference on Software Engineering and Advanced Applications . Lille, France: IEEE, 2010, pp. 261–268.
9. J. Cleland-Huang, R. Settimi, X. Zou, and P. Solc, “Automated classification of non-functional requirements,” Requirements Engineering , Vol. 12, No. 2, 2007, pp. 103–120.
10. I. Hussain, L. Kosseim, and O. Ormandjieva, “Using linguistic knowledge to classify non-functional requirements in SRS documents,” in International Conference on Application of Natural Language to Information Systems . London, UK: Springer, 2008, pp. 287–298.
11. G. Gokyer, S. Cetin, C. Sener, and M. Yondem, “Non-functional requirements to architectural concerns: ML and NLP at crossroads,” in Proceedings of the 2008 the 3rd international conference on software engineering advances . Sliema, Malta: IEEE, 2008, pp. 400–406.
12. A. Rashwan, O. Ormandjieva, and R. Witte, “Ontology-based classification of non-functional requirements in software specifications: A new corpus and svm-based classifier,” in 37th Annual Computer Software and Applications Conference . Kyoto, Japan: IEEE, 2013, pp. 381–386.
13. T. Nguyen, J. Grundy, and M. Almorsy, “Rule-based extraction of goal-use case models from text,” in Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering . Bergamo, Italy: ACM, 2015, pp. 591–601.
14. A. Casamayor, D. Godoy, and M. Campo, “Identification of non-functional requirements in textual specifications: A semi-supervised learning approach,” Information and Software Technology , Vol. 52, No. 4, 2010, pp. 436–445.
15. W. Zhang, Y. Yang, Q. Wang, and F. Shu, “An empirical study on classification of non-functional requirements,” in Twenty-Third International Conference on Software Engineering and Knowledge Engineering , Miami Beach, USA, 2011, pp. 444–449.
16. J. Slankas and L. Williams, “Automated extraction of non-functional requirements in available documentation,” in 1st International Workshop on Natural Language Analysis in Software Engineering . San Francisco, USA: IEEE, 2013, pp. 9–16.
17. P. Singh, D. Singh, and A. Sharma, “Rule-based system for automated classification of non-functional requirements from requirement specifications,” in International conference on Advances in Computing, Communications and Informatics . Jaipur, India: IEEE, 2016, pp. 620–626.
18. Z. Kurtanovic and W. Maalej, “Automatically classifying functional and non-functional requirements using supervised machine learning,” in 25th International Requirements Engineering Conference Workshops . Lisbon, Portugal: IEEE, 2017, pp. 490–495.
19. A. Hindle, N.A. Ernst, M.W. Godfrey, and J. Mylopoulos, “Automated topic naming,” Empirical Software Engineering , Vol. 18, No. 6, 2013, pp. 1125–1155.
20. D. Ott, “Automatic requirement categorization of large natural language specifications at Mercedes-Benz for review improvements,” in International Working Conference on Requirements Engineering: Foundation for Software Quality . Essen, Germany: Springer, 2013, pp. 50–64.
21. M. Lu and P. Liang, “Automatic classification of non-functional requirements from augmented app user reviews,” in 21st International Conference on Evaluation and Assessment in Software Engineering (EASE) . Karlskrona, Sweden: ACM, 2017, pp. 344–353.
22. R. Deocadez, R. Harrison, and D. Rodriguez, “Automatically classifying requirements from app stores: A preliminary study,” in 25th International Requirements Engineering Conference Workshops . Lisbon, Portugal: IEEE, 2017, pp. 367–371.
23. C. Li, L. Huang, J. Ge, B. Luo, and V. Ng, “Automatically classifying user requests in crowdsourcing requirements engineering,” Journal of Systems and Software , Vol. 138, 2018, pp. 108–123.
24. J. Zou, L. Xu, M. Yang, X. Zhang, and D. Yang, “Towards comprehending the non-functional requirements through developers’ eyes: An exploration of stack overflow using topic analysis,” Information and Software Technology , Vol. 84, 2017, pp. 19–32.
25. E. Knauss, S. Houmb, K. Schneider, S. Islam, and J. Jürjens, “Supporting requirements engineers in recognising security issues,” in International Working Conference on Requirements Engineering: Foundation for Software Quality . Essen, Germany: Springer, 2011, pp. 4–18.
26. M. Riaz, J. King, J. Slankas, and L.Williams, “Hidden in plain sight: Automatically identifying security requirements from natural language artifacts,” in International Conference on Requirements Engineering Conference . Karlskrona, Sweden: IEEE, 2014, pp. 183–192.
27. R. Jindal, R. Malhotra, and A. Jain, “Automated classification of security requirements,” in 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI) . Jaipur, India: IEEE, 2016, pp. 2027–2033.
28. R. Malhotra, A. Chug, A. Hayrapetian, and R. Raje, “Analyzing and evaluating security features in software requirements,” in Innovation and Challenges in Cyber Security . Noida, India: IEEE, 2016, pp. 26–30.
29. E. Knauss and D. Ott, “(semi-) automatic categorization of natural language requirements,” in International Working Conference on Requirements Engineering: Foundation for Software Quality . Essen, Germany: Springer, 2014, pp. 39–54.
30. R. Maiti and F. Mitropoulos, “Capturing, eliciting, predicting and prioritizing (CEPP) non-functional requirements metadata during the early stages of agile software development,” in Proceedings of the SoutheastCon . Fort Lauderdale, USA: IEEE, 2015, pp. 1–8.
31. A. Mahmoud and G. Williams, “Detecting, classifying, and tracing non-functional software requirements,” Requirements Engineering , Vol. 21, No. 3, 2016, pp. 357–381.
32. Z. Abad, O. Karras, P. Ghazi, M. Glinz, G. Ruhe, and K. Schneider, “What works better? A study of classifying requirements,” in 25th International Requirements Engineering Conference Workshops . Lisbon, Portugal: IEEE, 2017, pp. 496–501.
33. L. Toth and L. Vidacs, “Study of various classifiers for identification and classification of non-functional requirements,” in International Conference on Computational Science and Its Applications . Springer, 2018, pp. 492–503.
34. S. Uddin, A. Khan, M. Hossain, and M. Moni, “Comparing different supervised machine learning algorithms for disease prediction,” BMC Medical Informatics and Decision Making , Vol. 19, No. 281, 2019, pp. 1–16.
35. Y. Dengju, J. Yang, and X. Zhan, “A novel method for disease prediction: Hybrid of random forest and multivariate adaptive regression splines,” Journal of Computers , Vol. 8, No. 1, 2013, pp. 170–177.
36. C. Sinclair, L. Pierce, and S. Matzner, “An application of machine learning to network intrusion detection,” in Proceedings of 15th Annual Computer Security Applications Conference (ACSAC’99) . Phoenix, AZ, USA: IEEE, 1999, pp. 371–377.
37. E. Aleskerov, B. Freisleben, and B. Rao, “Cardwatch: A neural network based database mining system for credit card fraud detection,” in Proceedings of the IEEE/IAFE Computational Intelligence for Financial Engineering (CIFEr) . New York, NY, USA: IEEE, 1997, pp. 220–226.
38. S. Rathore and S. Kumar, “An empirical study of some software fault prediction techniques for the number of faults prediction,” Soft Computing , Vol. 21, 2016, pp. 7417–7434.
39. A. Okutan and O. Yildiz, “Software defect prediction using bayesian networks,” Empirical Software Engineering , Vol. 19, No. 1, 2014, pp. 154–181.
40. T. Patil and S. Sherekar, “Performance analysis of Naïve Bayes and J48 classification algorithm for data classification,” International Journal of Computer Science and Applications , Vol. 6, No. 2, 2013, pp. 256–261.
41. J. Qinlan, C4.5: Programs for machine Learning , 1st ed. San Mateo, CA: Morgan Kaufmann Publishers, 1993.
42. M. Danham and S. Sridhar, Data mining: Introductory Advanced Topics , 1st ed. Person Education, 2006.
43. Y. Freund and R. Schapire, “A short introduction to boosting,” Journal of Japanese Society for Artificial Intelligence , Vol. 14, No. 5, 1999, pp. 771–780.
44. B. Widro and M. Lehr, “30 years of adaptive neural networks: Perceptron, madaline, and backpropagation,” Proceedings of the IEEE , Vol. 78, No. 9, 1990, pp. 1415–1442.
45. D. Broomhead and D. Lowe, “Multivariable functional interpolation and adaptive networks,” Complex Systems , Vol. 2, 1998, pp. 321–355.
46. S. Eyheramendy, D. Lewis, and D. Madigan, “On the Naive Bayes model for text categorization,” in Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics , 2002.
47. A. McCallum and K. Nigam, “A comparison of event models for Naive Bayes text classification,” in Learning for Text Categorization , M. Sahami, Ed. AAAI Press, 1998, pp. 41–48.
48. R. Malhotra, Empirical Research in Software Engineering – Concepts, Analysis and Applications , 1st ed. India: CRC Press, 2015.
49. M. Stone, “Cross-validatory choice and assessment of statistical predictions,” Journal of the Royal Statistical Society , Vol. 36, No. 2, 1974, pp. 111–147.
50. Y. Jiang, B. Cukic, and Y. Ma, “Techniques for evaluating fault prediction models,” Empirical Software Engineering , Vol. 13, No. 15, 2008, pp. 561–595.
51. H. He and E. Garcia, “Learning from imbalanced data,” IEEE Transactions on Knowledge and Data Engineering , Vol. 21, No. 9, 2009, pp. 1263–1284.
52. T. Menzies, A. Dekhtyar, J. Distefance, and J. Greenwald, “Problems with precision: A response to “comments on ‘data mining static code attributes to learn defect predictors’”,” IEEE Transactions on Software Engineering , Vol. 33, No. 9, 2007, pp. 637–640.
53. F. Provost and T. Fawcett, “Robust classification for imprecise environments,” Machine Learning , Vol. 42, 2001, pp. 203–231.
54. R. Shatnawi, “Improving software fault-prediction for imbalanced data,” in Proc. of International Conf. on Innovations in Information Technology . Abu Dhabi, United Arab Emirates: IEEE, 2012, pp. 54–59.
55. T. Fawcett, “An introduction to ROC analysis,” Pattern Recogn Lett , Vol. 27, No. 8, 2006, pp. 861–874.
56. F. Sebastiani, “Machine learning in automated text categorization,” ACM Computing Surveys , Vol. 34, No. 1, 2002.
57. T. Menzies and A. Marcus, “Automated severity assessment of software defect reports,” in IEEE International Conference on Software Maintenance (ICSM) . Beijing, China: IEEE, 2008, pp. 346–355.
58. E. Arisholm and L. Briand, “Predicting fault-prone components in a Java legacy system,” in Proceedings of ACM/IEEE international symposium on empirical software engineering . IEEE, 2006, pp. 8–17.
59. R. Malhotra and M. Khanna, “An exploratory study for software change prediction in object-oriented systems using hybridized techniques,” Autom Softw Eng , Vol. 24, No. 3, 2017, pp. 673–717.
60. L. Briand, J. Wust, and J. Daly, “Exploring the relationship between design measures and software quality in object-oriented systems,” J Syst Softw , Vol. 51, No. 3, 2000, pp. 245–273.
61. L. Briand, J. Wust, and H. Lounis, “Replicated case studies for investigating quality factors in object oriented designs,” Empir Softw Eng J , Vol. 6, No. 1, 2001, pp. 11–58.
62. R. Malhotra and M. Khanna, “Threats to validity in search-based predictive modelling for software engineering,” IET Software , Vol. 12, No. 4, 2018, pp. 293–305.
63. A. Dean, D. Voss, and D. Draguljic, Design and analysis of experiments , 10th ed. New York: Springer, 1999.
64. Y. Zhou, H. Leung, and B. Xu, “Examining the potentially confounding effect of class size on the associations between object-oriented metrics and change-proneness,” IEEE Transactions on Software Engineering , Vol. 35, No. 5, 2009, pp. 607–623.
65. R. Harrison, S. Counsell, and R. Nithi, “Experimental assessment of the effect of inheritance on the maintainability of object-oriented systems,” Journal of Systems and Software , Vol. 52, No. 2–3, 2000, pp. 173–179.