Exploring Analytics at DIT

Patrick Walsh 
Technical Officer

Learning Analytics (LA) has been cited in four consecutive Horizon Reports from 2011-2014 as an emerging technology impacting on teaching, learning and research. It is largely an emerging field in educational research but analysis of Learning Management System (LMS) student engagement can provide a starting point for a LA initiative.

LMSs track and store vast quantities of student data. Research shows that Higher Education Institutes (HEIs) can harness the power of this data to build a better understanding of student learning. This study is an exploratory LA initiative to evaluate the inbuilt analytic features available within Blackboard Learn 9.1, the LMS solution adopted by DIT, to determine to what extent it informs academic staff on student engagement.

The study interrogated data generated through student engagement with DIT’s LMS. A sequential mixed method approach was employed. The quantitative phase involved the measurement and analysis of data gleaned from LMS reporting. Reports were extracted, anonymised and analysed for existence of interdependencies, trends, patterns and relationships. Interrogation of LMS variables was conducted on frequency of logins, student hits (click activity), grades, time spent on course and multiple choice quiz (MCQ) results to determine which variables, if any, were indicative of student success. Interviews conducted with lecturers provided the qualitative data for the study. Lecturers discussed their evaluation of these data analytic features and their thoughts on Learning Analytics.

Analysis of LMS variables established a statistically significant weakly positive correlation between hit activity, MCQ score, login activity and student examination results. These findings suggest that activity within LMS, measured by logins, hit activity and results in MCQs provide indicators of student academic performance.
Lecturers involved in the study felt the analytic features provided them with a sense of student engagement with course modules and better understanding of their student cohorts. The analytic tools also facilitated academic staff in learning about their own engagement within LMS or lack thereof with the features of LMS and may encourage them to make greater use of LMS and its analytic capabilities

Two issues to emerge included:

Data privacy
Data privacy remains a thorny issue. Openness and transparency are key factors in building student confidence in data mining exercises. Lecturers agreed that monitoring of student data, when done for the right intentions to promote learning and help identify students who may be at risk and when put to good use, trumps ethical issues.

Dispersed Information Sources
Lecturers would like to see greater integration of data from different sources e.g. Student Information Systems (SIS), library usage, class attendance records or records of previous academic history. Integrating data from different Institutional sources is a challenge faced by HEIs in attempting to build a richer profile of their student cohorts.

Learning Analytics is still in its infancy stage. It is an emerging tool which needs to evolve in terms of sophistication, popularity and effectiveness, particularly with Irish HEIs. However these embedded analytic tools offer us a lens into the student learning experience.

All talks:

If you require information about any aspect of the conference please contact: Fi Coyle: fiona.coyle@heanet.ie / +353 1 6609040