Investigating the Effectiveness of Feedback-Driven Exercises on Deadlock Detection Skills in Conceptual Modelling
摘要
Conceptual modelling (CM) has gained prominence in software development, especially with the rise of low-code/no-code development techniques rooted in model-driven development (MDD), which simplify coding complexity and bridge the gap between business and IT. However, these approaches are not without issues, as errors in models can lead to unreliable, error-prone software. One particular challenge for modellers is that of potential deadlock situations which can obstruct the system’s components and make it unable to finish the assigned tasks. This paper reports a study that aimed to explore how an Automated Feedback System (AFS) can assist novice modellers in identifying and rectifying possible deadlock situations in conceptual models. The study consisted of a two-group posttest-only experimental design, allowing to assess the impact of an AFS-enhanced approach on novice modellers’ ability to identify and correct deadlock-related errors in the context of a CM course. Unfortunately, the experiment did not conclusively demonstrate that the AFS-enhanced approach significantly improves novice modellers’ performance. Nonetheless, after performing additional post-hoc statistical tests on the behavioural data collected from the participants during the CM course prior to the experiment, we were able to conclude that this negative result was in part due to an imbalanced experimental group division. Subsequently, we redivided the participants based on the self-reported usage of a model-simulation tool during the experiment, which did allow us to find evidence for one of our hypotheses. We conclude the paper by proposing several improvements to the used experimental design as well as the implications for teaching conceptual modelling based on the already found results.