The Great Library is a game-based learning (GBL) environment that was developed by Delft University of Technology staff together with OrbitGames developers as part of a project to improve the peer review process in a MSc course on research design. This project was granted by SURF, a Dutch government funded organization for education and research.

@en

At Polytechnics design & engineering students are taught about state-of-the-art technical knowledge. Students become qualified engineers and learn to innovate artifacts related to their domain. Not taught is how to develop new engineering knowledge within a multidisciplinary context of stakeholders, companies and regulations. In short, students don't learn to innovate technology. What is taught today is the result of a technological innovation of yesterday. This is not sufficient for industry to innovatively deal with society's grand challenges. The paper describes a project that aims to educate all TU Delft graduate students in the verb of innovating technology, that is, the development of new technologies from inventions in the labs to full- fledged application in business. Such along three dimensions: technical, human and business. The educational portfolio consists of three modules in line with growth along Bloom's taxonomy and online materials on theoretical backbones. All modules apply the notion of technological innovation journeys (Tijo's). Tijo's are rich descriptions of the developmental journey of new technology and are based on inventions from the university's own labs. @en

Peer reviewing is an approach often used in digital (higher) education as it represents a possibility to provide feedback to large student groups. However, especially in digital learning contexts, students feel being disconnected from the anonymous group of learners, leading to low-quality feedback. When feedback is open, and counterparts are known, this knowledge can lead to biases when commenting on each other’s work. We developed a game-based learning environment called ‘The Great Library’ that we used to analyse the effect of avatars as game element to limit the effects of both fully anonymous and fully open feedback. Students and teachers using the new game-based learning environment were asked to provide feedback on their experiences with the learning environment and the use of avatars. The responses were compared to the same course provided without game elements. The qualitative data analysis reveals that the introduction of avatars still lacks acceptance on the side of the students, yet that they feel a sense of community in the GBL. Open and direct feedback is still used and appreciated by the students. The anonymity of the peer reviews resulting from the use of the avatars represents a challenge for the student-teacher interaction. The environment and set-up of the course in a whole still needs some adjustment in order to fully support the learning process of the students. @en

Peer review is increasingly used in higher education, also at Delft University of Technology. Unfortunately, the potential of student-student feedback often appears to be undervalued, making it less effective. The quality of the reviews is not as good and they are less constructive than expected. Moreover, not all students take giving and or receiving feedback serious. Since 2005 the peer review process in the course Preparation Master Thesis has been continuously developing with the aim to increase the contribution of constructive feedback to the learning process. In the course students learn the essential academic skills to write a research proposal. More than 1200 students have participated so far. Our objective is to support students in developing a suitable and feasible research design. For intermediary feedback on drafts and formative assessment, peer review (PR) is an essential element of this course. In the first years of the course the PR procedure was similar as well known in academic and other professional circles: students received collegial informal peer review of their ideas in class, and a more formal written peer review of the full draft of the deliverable. From 2012 the course was also offered fully online. Without class meetings, and far less student-teacher interaction the peer review procedure became more critical for the students’ learning experience, since informal interaction was hardly arranged. Continuous evaluation led to adaptation of the peer review process in the course. In this paper we reflect on the improvements that we have made over the years: - to consider feedback seriously: rebut and asses the reviews - to learn how to give constructive feedback: introduce a repeated review process, evaluating the reviews and assessing the rebuttal - to structure feedback: using an assessment form, later a rubric - to reduce peer pressure: using an anonymous review - to improve trust, confidence and comfort: experiments with peer groups Based on the most recent lessons learned we will discuss our latest project integrating the peer review process in a game-based learning environment. Based on the data and experience from the previous course runs we assume that game incentives will improve learning results as students are encouraged to engage in the interactive peer review process and further their professional behaviour giving constructive feedback. We aim to make this into an adaptable learning format for any programme that wishes to adopt a similar approach to increase the learning experience and effect of peer review. @en

Research into complex socio-technical systems requires a multi-disciplinary approach in which the role of actors, public values, and technical innovation are integrated. The research is aimed at systems design and the development of institutional rules for the operation of the technical innovation, taking multiple actor interests and public values into account. This multidisciplinary approach entails that research designs range from grounded theory approaches towards discrete event simulations; and from case study analysis towards agent based modelling. Consequently, the academic toolbox of research methods is wide and knowledge and creativity is required to develop appropriate research designs. Therefore, within our master programmes on complex socio-technical systems, our challenge was to make our students more aware of the competing and complementing theories, research methods, and design approaches that are at the base of complex socio-technical systems design and analysis. Teaching academic research skills offers the master student the means to develop a suitable and feasible research design for their master thesis project. Ultimately, these skills enable graduated students to critically assess research performed by third parties during their professional careers. The course we developed goes beyond the teaching of single research methods, which is often the core in dedicated courses on research methods. Our objective is to support students in developing a suitable and feasible research design in a multidisciplinary research context. This research design is the basis for their master thesis project. In the course we teach students to discover a scientific knowledge gap on the basis of a literature review, to formulate the contribution they wish to make to fill (part of) this gap, to formulate a well focused main research question and related sub questions and to choose an appropriate set of research methods that logically leads towards reaching the research deliverable and research objective. In addition, the data requirements, risks, and data analysis method(s) are part of the overall research design. Finally, an informative abstract, research flow diagram, bibliography, and a time schedule with research activities need to be added to the research proposal. In this case history we present our experiences with the course in which so far approximately 900 students participated. Based on our experiences we are now developing a redesign for a blended learning format for the course. The latter offers a general part on research design and video material to address the wide variety of research approaches and related research methods to support the academic creativity of the master students to develop their own research design. Gamification will be used to encourage the students to engage in an interactive peer review process. Our vision is that this blended learning format can be adapted to any MSc programme that wishes to adopt a similar course to prepare their students for their master thesis project.@en