Learning Analytics: Extending the Learning Engagement Beyond Classroom
學習分析: 課堂以外學習參與的延續
Student thesis: Doctoral Thesis
Author(s)
Related Research Unit(s)
Detail(s)
Awarding Institution | |
---|---|
Supervisors/Advisors |
|
Award date | 18 May 2021 |
Link(s)
Permanent Link | https://scholars.cityu.edu.hk/en/theses/theses(8ef1827b-d46b-4a57-a0ed-83fc7a85af7a).html |
---|---|
Other link(s) | Links |
Abstract
Learning analytics (LA), which has been defined as “the measurement, collection, analysis, and reporting of data about learners and their contexts, for the purpose of understanding and optimizing learning and the environments in which it occurs”, has begun to receive attention from researchers in the recent decade. Although there are multiple purposes to different LA applications, most of them focus on predicting learners’ academic performance to provide teachers with early interventions in helping learners, such as adjusting the instructional design for enhancing their learning engagement in the classroom. Many researchers have proved engaged learning to be one of the critical factors affecting learning outcomes. However, the method to cultivate engaged learning has not been investigated. Therefore, the first research question of this paper is: How can learning engagement be cultivated inside the classroom?
On the other hand, higher education institutions’ expectations have evolved from developing graduates with strong subject skills that are usually measured by academic scores to cultivating graduates with job-ready skills identified as the 21st century competencies. In addition, learning outside the classroom, particularly extracurricular activities, effectively cultivates learners’ 21st century competencies. LA applications in helping learners outside the classroom are rarely discussed. Therefore, the second and third research questions of this paper are:
•Can learning engagement enhance learners’ learning outcomes outside the classroom in non-academic settings?
•Can LA help in predicting student success beyond academic performance, including job readiness and employability?
To answer the above research questions, the author follows a five-step approach, with the first two steps being the preparation work and the remaining three steps consist of three studies. The first preparation work is to conduct a literature survey review on the LA research domain to identify LA‘s trend(s). The author has refined a model from Chatti’s Model and reviewed all papers published in the journal of learning analytics from the years 2014 to 2019. Through this, five trends are identified: (1) extending in data landscape; (2) extending in subjects and beneficiaries; (3) extending the coverage of student types; (4) shifting in objectives; and (5) shifting in analysis methods.
The second preparation work is to develop a centralized repository named “Central Repository on Student Development Activities” (CRESDA) for capturing extracurricular activities to analyse learners’ 21st century competencies quantitatively and systematically. These data are to be used in the third study for assessing the engagement level outside the classroom.
In the first study, the author aims to show the relationship between engaged learning and academic performance and identify the critical factor(s) of how engaged learning could be cultivated. This study adopts the longitudinal methodology by collecting two years of learners’ behavioural data in an e-schoolbag system used in a primary school in mainland China. Through this, the researchers further identify the importance of instructional design in cultivating learning engagement in the classroom. This study addresses the part of the first research question. However, there exists a missing link between the instructional design and the engagement level. Therefore, the second study is required.
In the second study, the author aims to demonstrate how to cultivate learners’ engagement level in classrooms by promoting situational interest based on the Theory of Interest (ToI) and the Self-Determination Theory (SDT). Using the longitudinal methodology, data are collected from a 13-week engineering course at the City University of Hong Kong for 2 consecutive years in a blended learning environment. It also includes the perceived usefulness of instructional design into the conceptual model to prove the importance of instructional design in the learning process. The main path of the model is from the instructional design, leading to learning satisfaction, learning motivation, situational interest, engaged learning, and finally the academic performance. This study addresses the remaining part of the first research question.
In the third study, the author aims to show how LA can be used in non-academic settings outside the classroom to help the learning planning and reflection of learners. Data have been collected from the CRESDA in the academic years 2017/18 and 2018/19 and compared to the findings against 2016/17. The employers’ feedback of 2018/19 is used for analysing learners’ employability. This paper has analysed students’ engagement level in extracurricular activities, including differences between extracurricular activities offered and the degree of participation, differences in participation patterns in extracurricular activities by students of different disciplines, and the correlation between engagement in extracurricular activities and job-readiness. It also shows how critical participation in extracurricular activities is related to fresh graduates‘ employability by analysing learners’ participation in extracurricular activities against the employers’ feedback during a placement programme by computer science students. This study addresses the second and the third research questions.
Throughout the three studies, the author experienced different types of challenges similar to those being experienced by other researchers in previous literature. These challenges include: (i) data quality and quantity issues due to limitations of systems, capabilities in using and willingness in using those systems by different stakeholders; (ii) data standardisation and pre-processing being required due to the differences in definition, format, and measuring scales by different systems, institutions, and nations; (iii) methods of data analysis depend on many factors such as the purpose, data type, and data source; (iv) trust and commitment from different stakeholders in using the systems; and (v) security and privacy measures when handling sensitive data. These challenges limit the deployment of LA applications in scale. Corresponding recommendations are proposed and discussed in Chapter 7.
This paper contributes theoretically to different research domains, including (i) learning analytics by refining the reference model, identifying the trends, and suggesting the future research directions of LA; (ii) identifying the criticalness of instructional design in cultivating learners’ engagement level and the main path between them by combining the SDT and ToI and applying them to LA; (iii) 21st century competencies by proposing a systematic and quantitative measurement method and conducting empirical studies to verify its impact on academic performance and job-readiness. This paper also contributes practically in guiding and supporting various stakeholders, including (i) learners can receive personalized advisory services and other interventions from teachers, and adjust learning plans with data support; (ii) teachers can refine the instructional design for motivating learners for higher engagement level and spend their efforts in a more effective way by given more supports to at-risk students; (iii) advisors can understand students’ strengths and weaknesses better in order to provide personalized advisory services to each student; (iv) system designers/researchers can design better LA applications based on the LA trends; (v) institution management can provide support to student-at-risk in a more timely manner; (vi) LA developers can develop LA applications at larger-scale; and (vii) parents can understand their children’s learning progress and provide proper support to them.
On the other hand, higher education institutions’ expectations have evolved from developing graduates with strong subject skills that are usually measured by academic scores to cultivating graduates with job-ready skills identified as the 21st century competencies. In addition, learning outside the classroom, particularly extracurricular activities, effectively cultivates learners’ 21st century competencies. LA applications in helping learners outside the classroom are rarely discussed. Therefore, the second and third research questions of this paper are:
•Can learning engagement enhance learners’ learning outcomes outside the classroom in non-academic settings?
•Can LA help in predicting student success beyond academic performance, including job readiness and employability?
To answer the above research questions, the author follows a five-step approach, with the first two steps being the preparation work and the remaining three steps consist of three studies. The first preparation work is to conduct a literature survey review on the LA research domain to identify LA‘s trend(s). The author has refined a model from Chatti’s Model and reviewed all papers published in the journal of learning analytics from the years 2014 to 2019. Through this, five trends are identified: (1) extending in data landscape; (2) extending in subjects and beneficiaries; (3) extending the coverage of student types; (4) shifting in objectives; and (5) shifting in analysis methods.
The second preparation work is to develop a centralized repository named “Central Repository on Student Development Activities” (CRESDA) for capturing extracurricular activities to analyse learners’ 21st century competencies quantitatively and systematically. These data are to be used in the third study for assessing the engagement level outside the classroom.
In the first study, the author aims to show the relationship between engaged learning and academic performance and identify the critical factor(s) of how engaged learning could be cultivated. This study adopts the longitudinal methodology by collecting two years of learners’ behavioural data in an e-schoolbag system used in a primary school in mainland China. Through this, the researchers further identify the importance of instructional design in cultivating learning engagement in the classroom. This study addresses the part of the first research question. However, there exists a missing link between the instructional design and the engagement level. Therefore, the second study is required.
In the second study, the author aims to demonstrate how to cultivate learners’ engagement level in classrooms by promoting situational interest based on the Theory of Interest (ToI) and the Self-Determination Theory (SDT). Using the longitudinal methodology, data are collected from a 13-week engineering course at the City University of Hong Kong for 2 consecutive years in a blended learning environment. It also includes the perceived usefulness of instructional design into the conceptual model to prove the importance of instructional design in the learning process. The main path of the model is from the instructional design, leading to learning satisfaction, learning motivation, situational interest, engaged learning, and finally the academic performance. This study addresses the remaining part of the first research question.
In the third study, the author aims to show how LA can be used in non-academic settings outside the classroom to help the learning planning and reflection of learners. Data have been collected from the CRESDA in the academic years 2017/18 and 2018/19 and compared to the findings against 2016/17. The employers’ feedback of 2018/19 is used for analysing learners’ employability. This paper has analysed students’ engagement level in extracurricular activities, including differences between extracurricular activities offered and the degree of participation, differences in participation patterns in extracurricular activities by students of different disciplines, and the correlation between engagement in extracurricular activities and job-readiness. It also shows how critical participation in extracurricular activities is related to fresh graduates‘ employability by analysing learners’ participation in extracurricular activities against the employers’ feedback during a placement programme by computer science students. This study addresses the second and the third research questions.
Throughout the three studies, the author experienced different types of challenges similar to those being experienced by other researchers in previous literature. These challenges include: (i) data quality and quantity issues due to limitations of systems, capabilities in using and willingness in using those systems by different stakeholders; (ii) data standardisation and pre-processing being required due to the differences in definition, format, and measuring scales by different systems, institutions, and nations; (iii) methods of data analysis depend on many factors such as the purpose, data type, and data source; (iv) trust and commitment from different stakeholders in using the systems; and (v) security and privacy measures when handling sensitive data. These challenges limit the deployment of LA applications in scale. Corresponding recommendations are proposed and discussed in Chapter 7.
This paper contributes theoretically to different research domains, including (i) learning analytics by refining the reference model, identifying the trends, and suggesting the future research directions of LA; (ii) identifying the criticalness of instructional design in cultivating learners’ engagement level and the main path between them by combining the SDT and ToI and applying them to LA; (iii) 21st century competencies by proposing a systematic and quantitative measurement method and conducting empirical studies to verify its impact on academic performance and job-readiness. This paper also contributes practically in guiding and supporting various stakeholders, including (i) learners can receive personalized advisory services and other interventions from teachers, and adjust learning plans with data support; (ii) teachers can refine the instructional design for motivating learners for higher engagement level and spend their efforts in a more effective way by given more supports to at-risk students; (iii) advisors can understand students’ strengths and weaknesses better in order to provide personalized advisory services to each student; (iv) system designers/researchers can design better LA applications based on the LA trends; (v) institution management can provide support to student-at-risk in a more timely manner; (vi) LA developers can develop LA applications at larger-scale; and (vii) parents can understand their children’s learning progress and provide proper support to them.
- Learning Analytics, Engaged Learning, Extracurricular Activities, Learning Outcomes, Personal Development, 21st Century Competencies