Step 5: Evaluate course effectiveness

The final step is to evaluate whether the course guided learners to the learning outcomes defined initially. The results of this evaluation will help identify problems with the course design and allow for adjustments to improve course effectiveness in future iterations. If time and resources permit, it is good practice to pilot the first versions of your course and allow for incorporation of quick feedback and modifications shortly after launch.

Therefore, the evaluation needs to be actionable, i.e. it needs to be able to inform decisions.

For longer courses with a full curriculum, it can be straightforward to define reliable metrics for course effectiveness. By way of example, course evaluation for a full semester of student seminars (Wiljes and Cimiano 2019) can be built on the study requirements that students must meet in order to receive credit points. Writing an individual DMP as a seminar paper provides a sound basis for evaluating whether students have acquired the knowledge, skills and abilities as defined by the learning outcomes. In addition, this allows you to identify problems with specific topics and narrow them down to the specific methods (i.e. learning experiences) that were used. To give an example, if 'metadata' is presented as the topic of a talk and the final evaluation of students' DMPs reveals that they are not able to apply the content of the talk properly, another teaching method should be trialled instead. You could, for instance, provide students with a specific metadata standard and have them work out on their own how to apply it. Biernacka et al. (2020) provide examples of teaching methods for a wide variety of RDM/FAIR topics.

With shorter courses, e.g. a 4-hour workshop, evaluating course effectiveness is more challenging. We recommend leaving enough time for students to write down and ask questions. A lively discussion is generally a good sign that students are progressing.

To some extent, the metrics provided in step 4 to assess learners' progress can also be applied to evaluate overall course effectiveness. However, you should note that these metrics may also need to be improved upon iteratively.

Conducting an anonymous survey on student satisfaction can complement an evaluation of course effectiveness. However, this should be interpreted with care because student satisfaction may be influenced by factors other than successful learning (Denson et al. 2010). In addition, students are biased in evaluating how their own skills and knowledge improve (Dunning et al. 2004; Karpen 2018).

---