Paper on GLM evaluation accepted at ICWL2009
Last year, we performed an evaluation of the Graphical Learning Modeller (GLM) with twenty instructors at the University of Vienna. The results of this evaluation were described in an article we submitted to the International Conference on Web-based Learning (ICWL) 2009. The article was accepted and will thus be published in a forthcoming Lecture Notes in Computer Science book.
The article describes the GLM functionalities, i.e. how it is used to design units of learning that are conformant to the IMS Learning Design specification. It then describes the study performed at the University of Vienna. Results showed that almost all instructors were able to successfully produce units of learning with the GLM after only a limited introduction. We were especially surprised that 70% of the test users included level B elements in their units of learning -- a concept that was not even included in the short introduction. No other IMS LD editor has been reported to achieve such success rates in level B usage. Problematic regarding the GLM was that the instructors did not easily understand the concepts of the software (and perhaps the concepts of the IMS LD specification). One of the major problems may be that the editors to create units of learning (such as the GLM) are commonly separate from the runtime environment, where the unit of learning will be interpreted/played like in a learning management system. For unit of learning designers it is hard to imagine what consequences their actions in the editing environment will have in the runtime environment and whether the unit of learning works in the way that they want it to work.