2. SOME TRENDS ON DATA-DRIVEN INNOVATION IN ENGINEERING EDUCATION
One of the most important trends in data-driven innovation for engineering education is the improvement of educational content. Educational content may include, for example, video lectures, automatic correction exercises, and other additional resources (texts, animations, simulations, etc.). Nowadays, it is possible to collect low-level data related to the interaction of each learner with each educational resource, including (Ruipérez-Valiente, Muñoz-Merino, Leony, & Delgado Kloos, 2015) when the learner starts to play a video, when the learner stops a video, number of seconds of the video watched by the learner, when the learner attempts an automatic correction exercise, number of attempts in each automatic correction exercise by the learner, learning sequence followed by the learner when moving between educational resources, etc. With this low-level data, it is possible to detect videos or exercises that are not working correctly and that need to be improved. Some high-level indicators that contribute to detecting video lectures that need to be revised are: a high number of repetitions in a fragment of a video (which typically denotes a complex explanation or an error in that fragment), and a high percentage of students who do not watch a video from the beginning to the end (which typically denotes inappropriate content for the student’s level or lack of engagement in the teacher’s explanation). Some high-level indicators that contribute to detecting automatic correction exercises that need to be revised are: a very low number of incorrect answers in formative exercises (which typically denotes very simple exercises that may cause boredom and waste students’ time), and a very high number of incorrect answers in summative exercises (which typically denotes very complex exercises that may cause frustration as students are not well prepared to solve those exercises).
Another important trend in data-driven innovation in engineering education refers to the improvement of social interactions among learners, and applies typically to courses with a very large number of students, either online or blended courses, and where teachers cannot provide personalized assistance due to the very large number of social interactions that take place in the course. Research in this line focused on characterizing the social interactions produced in a course, and on proposing methods and visualizations to help teachers make decisions about how to improve their course design. For example, there have been research studies which detected that the most appropriate tool to manage social interaction in courses with a very large number of students is the built-in forum provided by the learning platforms (Alario-Hoyos, Pérez-Sanagustín, Delgado-Kloos, Parada G., & Muñoz-Organero, 2014). Some other research studies focused on the identification of leaders within the community of learners, characterizing these leaders as the most active students in the course forum (Alario-Hoyos, Muñoz-Merino, Pérez-Sanagustín, Delgado Kloos, & Parada G., 2016). This identification of leaders is important to facilitate teachers’ work, as leaders can act as a bridge between the faculty and the rest of the students, even receiving special roles to be able to curate forum messages. Some other research studies focused on analyzing the overall class mood from social interactions, calculating the polarity of messages (positive, neutral, negative) posted by students in the course forum. The polarity of messages was calculated by applying word dictionaries and syntax rules, and the aim was to detect parts of the course in which the overall class mood was more positive or more negative to take corrective measures in the second case (Moreno-Marcos, Alario-Hoyos, Muñoz-Merino, Estévez-Ayres, & Delgado Kloos, 2018). Finally, all the data collected from social interactions can be used as input to develop chatbots or conversational agents programmed to give support to students in specific courses (Delgado Kloos, Catalán-Aguirre, Muñoz-Merino, Alario-Hoyos, 2018).
An additional relevant trend in data-driven innovation in engineering education is the study, characterization, and support of students’ development of self-regulated learning (SRL) skills. SRL skills are particularly important in engineering education due to, among other things, the complexity of the contents and the permanent retraining demanded in today’s engineers. Appropriate strategies to self-regulate each one’s learning should be applied in every learning stage (before, during, and after each learning activity), including, for instance, setting reasonable and measurable objectives (before), seeking help when necessary (during), self-reflecting on the work done and the objectives achieved (after), etc. (Alonso-Mencía, Alario-Hoyos, Maldonado-Mahauad, Estévez-Ayres, Pérez-Sanagustín, & Delgado Kloos, 2019). The study and characterization of SRL skills in engineering courses led to the conclusion that strategies related to appropriate time management are the most problematic ones for learners. Therefore, specific interventions need to be done to facilitate time management, both when designing a course and when developing support tools (Alario-Hoyos, Estévez-Ayres, Pérez-Sanagustín, Delgado Kloos, & Fernández-Panadero, 2017). It is also important to support students’ self-reflection by offering high-level visualizations based on low-level data with the aim to increase students’ awareness on the desired level to be achieved and that of their classmates, both for an entire course and for each module or part of a course (Ruipérez-Valiente, Muñoz-Merino, Leony, & Delgado Kloos, 2015).
One last, but not least, relevant trend in data-driven innovation in engineering education is the prediction of students’ behavior from previously collected data and appropriately trained machine learning models. The variables that are typically predicted through these models refer to students’ partial or final grades in a course; and also to whether a student will abandon a course or not; and, by extension, to whether a student will abandon a complete study program (bachelor’s degree or master’s degree) or not. The aim of these prediction models is to take corrective measures in order to prevent students from failing a course, or from dropping out of a course or study program. Studies on prediction in education have detected that, in general, low-level data, such as the interaction with educational content (e.g., videos and exercises), have greater predictive power than data collected from self-reported questionnaires (such as students’ intentions and motivations) (Moreno-Marcos, Alario-Hoyos, Muñoz-Merino, & Delgado Kloos, 2018). There are still important gaps in this research line, including proposing generalizable models applicable to different educational contexts and areas of knowledge, and developing predictive models and tools for real-time data collection and processing in order to improve the implementation of corrective measures.
3. SOME RISKS OF DATA-DRIVEN INNOVATION IN ENGINEERING EDUCATION
The collection and processing of data for decision-making in engineering education is not without problems. In fact, there are important risks to worry about, which have been highlighted by numerous experts on learning analytics, educational data mining, and data protection policies, (Dringus, 2012) (Khalil, Taraghi, & Ebner, 2016), as well as certain ethical considerations (Slade, & Tait, 2019). Some of these risks are briefly described in this section, although each of them would deserve an entire paper for discussion.
A first problem refers to the secure storage of collected data. Many institutional education systems are not prepared to store large amounts of data in a secure way and are likely to have breaches that can compromise important data, including students’ personal data. Relying on third-party services for data storage in the cloud can also lead to an inappropriate use of the data collected by these service providers.
A second problem refers to data privacy. It is very important to have an institutional strategy that clearly states what data needs to be collected from students and who has access to the data. Many educational institutions are not aware of all the data they collect (or can potentially collect); do not have mechanisms to control who has access to the data collected; and/ or do not have an internal policy to facilitate that the right