| 2024 | |
| [1] | , "Measuring End-user Developers’ Episodic Experience of a Low-code Development Platform", In e-Informatica Software Engineering Journal, vol. 18, no. 1, pp. 240105, 2024.
DOI: 10.37190/e-Inf240105. Download article (PDF)Get article BibTeX file |
Authors
Dongmei Gao, Fabian Fagerholm
Abstract
Background: As low-code development platforms (LCDPs) are becoming a trend, understanding how end-user developers think and feel as they work with such platforms is important. Particularly, assessing experiences during episodes of use can contribute to overall experience throughout long-term use.
Aim: This paper aims to understand end-user developers’ episodic experience when they are building an application on a low-code platform and to provide guidance on how such experiences can be measured.
Method: We designed the Episodic Developer Experience Questionnaire for LCDPs based on prior literature and refined it through expert Delphi sessions. The instrument contains 10 individual experience items, capturing various aspects of episodic experience. We further validated it through remote online tests on an LCDP.
Results: The results showed significant differences in the relationships between items describing aspects of overall experience and items describing perceptions of tool quality and task difficulty. Programming expertise also affected end-user developers’ episodic experience.
Conclusion: The study illustrates the design of questionnaire-based experience assessment in the context of development and identifies the importance of separating personal experience from assessment of tasks and tools since tool quality and task difficulty do not necessarily influence experience straightforwardly.
Keywords
developer experience, episodic experience, low-code development platforms, experience measurement, software engineering, human-computer interaction
References
1. F. Fagerholm and J. Münch, “Developer experience: Concept and definition,” in International Conference on Software and System Process (ICSSP). IEEE, 2012, pp. 73–77.
2. F. Fagerholm, Software developer experience: Case studies in lean-agile and open source environments, Ph.D. dissertation, University of Helsinki, Faculty of Science, Department of Computer Science, Helsinki, Finland, 2015.
3. D. Graziotin, X. Wang, and P. Abrahamsson, “Are happy developers more productive?” in International Conference on Product Focused Software Process Improvement. Springer, 2013, pp. 50–64.
4. M.A. Storey, T. Zimmermann, C. Bird, J. Czerwonka, B. Murphy et al., “Towards a theory of software developer job satisfaction and perceived productivity,” IEEE Transactions on Software Engineering, Vol. 47, No. 10, 2019, pp. 2125–2142.
5. N. Baddoo, T. Hall, and D. Jagielska, “Software developer motivation in a high maturity company: a case study,” Software process: improvement and practice, Vol. 11, No. 3, 2006, pp. 219–228.
6. D. Graziotin, F. Fagerholm, X. Wang, and P. Abrahamsson, “What happens when software developers are (un)happy,” Journal of Systems and Software, Vol. 140, 2018, pp. 32–47.
7. H. Lieberman, F. Paternò, M. Klann, and V. Wulf, “End-user development: An emerging paradigm,” in End user development. Springer, 2006, pp. 1–8.
8. Y. Luo, P. Liang, C. Wang, M. Shahin, and J. Zhan, “Characteristics and challenges of low-code development: The practitioners’ perspective,” in Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). ACM, 2021, pp. 1–11.
9. M. Tisi, J.M. Mottu, D.S. Kolovos, J. De Lara, E.M. Guerra et al., “Lowcomote: Training the next generation of experts in scalable low-code engineering platforms,” in STAF 2019 Co-Located Events Joint Proceedings: 1st Junior Researcher Community Event, 2nd International Workshop on Model-Driven Engineering for Design-Runtime Interaction in Complex Systems, and 1st Research Project Showcase Workshop co-located with Software Technologies: Applications and Foundations (STAF 2019), 2019.
10. J. Cabot, “Positioning of the low-code movement within the field of model-driven engineering,” in Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings, MODELS ’20. ACM, 2020, pp. 1–3.
11. A. Sahay, A. Indamutsa, D. Di Ruscio, and A. Pierantonio, “Supporting the understanding and comparison of low-code development platforms,” in 2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE, 2020, pp. 171–178.
12. F. Khorram, J.M. Mottu, and G. Sunyé, “Challenges & opportunities in low-code testing,” in Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings, MODELS ’20. ACM, 2020, pp. 1–10.
13. H. Lee and Y. Pan, “Evaluation of the nomological validity of cognitive, emotional, and behavioral factors for the measurement of developer experience,” Applied Sciences, Vol. 11, No. 17, 2021, p. 7805.
14. E. Karapanos, J. Zimmerman, J. Forlizzi, and J.B. Martens, “User experience over time: an initial framework,” in Proceedings of the SIGCHI conference on human factors in computing systems, 2009, pp. 729–738.
15. B. Callary, P. Werthner, and P. Trudel, “How meaningful episodic experiences influence the process of becoming an experienced coach,” Qualitative research in sport, exercise and health, Vol. 4, No. 3, 2012, pp. 420–438.
16. P. Marti and I. Iacono, “Anticipated, momentary, episodic, remembered: the many facets of user experience,” in 2016 federated conference on computer science and information systems (fedcsis). IEEE, 2016, pp. 1647–1655.
17. M. Greiler, M.A. Storey, and A. Noda, “An actionable framework for understanding and improving developer experience,” IEEE Transactions on Software Engineering, 2022.
18. K. Kuusinen, “Are software developers just users of development tools? assessing developer experience of a graphical user interface designer,” in Human-Centered and Error-Resilient Systems Development. Springer, 2016, pp. 215–233.
19. K. Kuusinen, “Software developers as users: Developer experience of a cross-platform integrated development environment,” in International Conference on Product-Focused Software Process Improvement. Springer, 2015, pp. 546–552.
20. T. Kaltio and A. Kinnula, “Deploying the defined sw process,” Software Process: Improvement and Practice, Vol. 5, No. 1, 2000, pp. 65–83.
21. D. Graziotin, F. Fagerholm, X. Wang, and P. Abrahamsson, “On the unhappiness of software developers,” in Proceedings of the 21st international conference on evaluation and assessment in software engineering, 2017, pp. 324–333.
22. K.R. Linberg, “Software developer perceptions about software project failure: a case study,” Journal of Systems and Software, Vol. 49, No. 2-3, 1999, pp. 177–192.
23. A. Bobkowska, “On explaining intuitiveness of software engineering techniques with user experience concepts,” in Proceedings of the International Conference on Multimedia, Interaction, Design and Innovation, 2013, pp. 1–8.
24. F. Fagerholm, M. Felderer, D. Fucci, M. Unterkalmsteiner, B. Marculescu et al., “Cognition in software engineering: A taxonomy and survey of a half-century of research,” ACM Comput. Surv., Vol. 54, No. 11s, 2022.
25. M. Sánchez-Gordón and R. Colomo-Palacios, “Taking the emotional pulse of software engineering — a systematic literature review of empirical studies,” Information and Software Technology, Vol. 115, 2019, pp. 23–43.
26. G. Fischer, D. Fogli, and A. Piccinno, “Revisiting and broadening the meta-design framework for end-user development,” in New perspectives in end-user development. Springer, 2017, pp. 61–97.
27. B.R. Barricelli, F. Cassano, D. Fogli, and A. Piccinno, “End-user development, end-user programming and end-user software engineering: A systematic mapping study,” Journal of Systems and Software, Vol. 149, 2019, pp. 101–137.
28. W.W. Cotterman and K. Kumar, “User cube: a taxonomy of end users,” Communications of the ACM, Vol. 32, No. 11, 1989, pp. 1313–1320.
29. M. Hirzel, “Low-code programming models,” Communications of the ACM, Vol. 66, No. 10, 2023, pp. 76–85.
30. C. Richardson, J.R. Rymer, C. Mines, A. Cullen, and D. Whittaker, “New development platforms emerge for customer-facing applications,” Forrester, Cambridge MA USA 15, Tech. Rep. 16, Jun 2014.
31. R. Waszkowski, “Low-code platform for automating business processes in manufacturing,” IFAC-PapersOnLine, Vol. 52, No. 10, 2019, pp. 376–381.
32. N. Dalkey and O. Helmer, “An experimental application of the delphi method to the use of experts,” Management science, Vol. 9, No. 3, 1963, pp. 458–467.
33. A. Nylund, “A multivocal literature review on developer experience,” 2020, aalto University.
34. S.A. Jackson, A.J. Martin, and R.C. Eklund, “Long and short measures of flow: The construct validity of the fss-2, dfs-2, and new brief counterparts,” Journal of Sport and Exercise Psychology, Vol. 30, No. 5, 2008, pp. 561–587.
35. R.M. Ryan, “Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory.” Journal of personality and social psychology, Vol. 43, No. 3, 1982, p. 450.
36. M. Hassenzahl, S. Diefenbach, and A. Göritz, “Needs, affect, and interactive products–facets of user experience,” Interacting with computers, Vol. 22, No. 5, 2010, pp. 353–362.
37. D. Alwin, “Feeling thermometers versus 7-point scales: Which are better?” Sociological Methods & Research, Vol. 25, No. 3, 1997, pp. 318–340.
38. H. Taherdoost, “What is the best response scale for survey and questionnaire design; review of different lengths of rating scale / attitude scale / likert scale,” International Journal of Academic Research in Management, Vol. 8, No. 1, 2019, pp. 1–10.
39. D. Gao and F. Fagerholm, “Supplementary materials for the paper ”Measuring End-user Developers’ Episodic Experience of a Low-code Development Platform — A Preliminary Study”,” https://doi.org/10.5281/zenodo.7515833, 2023.
40. M. Lynn, “Determination and quantification of content validity,” Nursing Research, Vol. 35, 1986, pp. 382–386.
41. D.V. Cicchetti, “Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology.” Psychological assessment, Vol. 6, No. 4, 1994, p. 284.
42. T.K. Koo and M.Y. Li, “A guideline of selecting and reporting intraclass correlation coefficients for reliability research,” Journal of chiropractic medicine, Vol. 15, No. 2, 2016, pp. 155–163.
43. B. Laugwitz, T. Held, and M. Schrepp, “Construction and evaluation of a user experience questionnaire,” in Symposium of the Austrian HCI and usability engineering group. Springer, 2008, pp. 63–76.
44. D.A. Redelmeier and D. Kahneman, “Patients’ memories of painful medical treatments: real-time and retrospective evaluations of two minimally invasive procedures,” Pain, Vol. 66, No. 1, 1996, pp. 3–8.
45. J. Connell, J. Carlton, A. Grundy, E. Taylor Buck, A.D. Keetharuth et al., “The importance of content and face validity in instrument development: lessons learnt from service users when developing the recovering quality of life measure (reqol),” Quality of Life Research, Vol. 27, 2018, pp. 1893–1902.