Defining “data” in conversations with students about the ethical use of learning analytics

Concise paper

Download the paper [PDF]

Abi Brooker
University of Melbourne
@abi__brooker

Linda Corrin
University of Melbourne
@lindacorrin

Negin Mirriahi
University of South Australia
@neginm

Josie Fisher
University of New England

Catch this session

Monday 4 December, 1.50pm - 2.10pm
Stream 3
Room L206

Abstract

In any conversation about the development of ethical standards for practice, it is vital that all stakeholders have a shared understanding of the main concepts in order to reach agreement. In the context of higher education and learning analytics, while many conversations are underway, it is less clear that such a shared understanding exists around the concept of “data”. In order to understand this situation more fully we conducted a study to investigate students’ perceptions of the ethical and privacy considerations related to the data that universities collect and use about them for the purposes of learning analytics. In this paper, we focus specifically on the understandings students have of the types of data that can be collected about them within the educational environment. The outcomes showed that there was a diversity of understandings, but that five main data types emerged: personal data, online activity, student feedback, academic information, and resource usage. In developing a better understanding of the ways students understand data, it can assist institutions to have more effective conversations with students about the ethical use of learning analytics.

About the authors

Abi Brooker

Dr Abi Brooker is a teaching fellow at the Melbourne School of Psychological Sciences, at the University of Melbourne where she teaches developmental psychology to undergraduate students. She is also co-leader of a national network of academic teaching staff interested in issues related to student and staff well-being. Her research interests take a lifespan developmental perspective of young adults’ experiences, especially those studying at university. Current projects include the ethical implications of learning analytics on the student experience, the consequences of students’ experiences of psychological well-being and distress, and undergraduate students’ curricular and non-curricular challenges.

Linda Corrin

Dr Linda Corrin is a Senior Lecturer in Higher Education in the Williams Centre for Learning Advancement, Faculty of Business and Economics, at the University of Melbourne. In her current role, she provides support for curriculum development, delivery and assessment to staff in the faculty. Her research interests include students' engagement with technology, learning analytics, feedback, and learning design. Currently, she is working on several large research projects exploring how learning analytics can be used to provide meaningful and timely feedback to academics and students. Linda is member of the University of Melbourne’s Learning Analytics Research Group and co-founder of the Victorian and Tasmanian Learning Analytics Network. She is also a co-coordinator of the ASCILITE Learning Analytics Special Interest Group.

Negin Mirriahi

Dr Negin Mirriahi is Senior Lecturer in the Teaching Innovation Unit at the University of South Australia. She has extensive international experience managing, implementing, and evaluating innovative educational technology in higher education and in designing fully online, blended, and open courses. Her research is in learning analytics to inform pedagogical practice, video analytics to enhance learning, technology adoption, blended and online learning, and academic staff development.

Josie Fisher

Dr Josie Fisher is an Associate Professor at the University of New England Business School. She teaches professional ethics, business ethics, corporate social responsibility and sustainability to postgraduate students. In addition, she is Chair of Teaching and Learning in the School, a senior management position that provides leadership in the design and continuous improvement of courses and units with a focus on innovative delivery. Her research has two themes: professional and business ethics, corporate social responsibility and sustainability; and learning analytics, particularly the ethical aspects of the use of analytics by universities.


Explaining learning achievement in student experience of blended learning: What can a sociomaterial perspective contribute?

Concise paper

Download the paper [PDF]

Feifei Han
The University of Sydney

Robert Ellis
The University of Sydney

Catch this session

Monday 4 December, 2.10pm - 2.30pm
Stream 2
Room R113

Abstract

Drawing on theories of student approaches to learning and sociomaterial perspectives on learning, we investigated how a combination of sociocognitive and sociomaterial variables explain variation in 365 students’ learning achievement in a first year human biology blended learning course in an Australian research intensive university. We used student experience questionnaires to measure students’ self-reports about their approaches to learning through inquiry, approaches to using online learning technologies, and their use of on-campus physical learning spaces. We also obtained observed measures of online learning technologies in terms of frequency and duration through analytics provided by a proprietary learning management system. Students’ learning achievement was evaluated using their assessment schedule comprising six assessment tasks. Correlation analyses were conducted to examine the interrelationship between approaches, use of online learning technologies, use of on-campus physical learning spaces, and achievement. Based on the correlation results, we regressed learning achievement on approaches, use of online learning technologies and physical learning spaces. The results showed that by introducing sociomaterial variables into the regression analysis, a significant proportion of learning achievement was explained over and above the explanations offered by student experience variables alone. The results highlight an important role of combining both self-report and observational data in analyses of student experiences of blended course designs.

About the authors

Feifei Han

Feifei Han currently is an educational researcher at the University of Sydney. Her current research interests comprise of three broad themes: (1) language and literacy education; (2) teaching, learning, and educational technology in higher education, and (3) educational psychology.

Robert Ellis

Robert Ellis is an Associate Professor of Education at the University of Sydney. He has been a funded chief investigative researcher for the Australian Research Council since 2005. His current project is funded up until 2019 (with Goodyear and others). Robert’s research interests focus on the student and teaching experience of e-learning, quality and innovation in higher education and learning spaces. He is author of two books and more than eighty internationally refereed publications, mostly in journal articles. He is also a coordinating editor of the Springer Journal ‘Higher Education’, and co-editor of the new book series for Springer ‘Understanding Teaching and Learning practice’. The goal of his research and scholarship is to achieve meaningful social contributions through translational research outcomes.


Improving the undergraduate science experience through an evidence-based framework for design, implementation and evaluation of flipped learning

Concise paper

Download the paper [PDF]

Yvonne Davila
University of Technology Sydney
@whydee13

Elaine Huber
University of Technology Sydney
@enm181

Jorge Reyna
University of Technology Sydney

Peter Meier
University of Technology Sydney

Catch this session

Monday 4 December,  1.30pm - 1.50pm
Stream 2
Room R113

Abstract

Flipped Learning (FL) is a student-centred pedagogical approach where new content is introduced prior to class which permits more time during class for active learning. Despite the growing body of evidence of the effectiveness of FL, many educators are reluctant to adopt this approach to teaching or are unsure of how to implement FL in their classes. Many students are uncertain of how to adapt their approaches to learning to a FL curriculum. In response to these challenges and calls for a robust framework to guide the design and implementation of FL, we developed the Flipped Teacher and Flipped Learner (FTFL) Framework based on the pedagogical literature. This paper reports on the use of our FTFL framework in the redesign of a large first year science subject from a traditional delivery to a FL delivery. We evaluated the efficacy of the redesign using a mixed methods approach with data on students’ interactions with FL activities, and student and educator experiences. Findings from two iterations of the redesign indicate successful implementation of FL through high student engagement with online and class materials, and positive feedback from students and academics. Using the FTFL framework to guide the design and integration of FL, with an emphasis on clear communication, is key to our successful FL intervention and support of student learning.

About the authors

Yvonne Davila

Dr Yvonne Davila is a Lecturer in Higher Education Learning Design in the Faculty of Science at the University of Technology Sydney. Yvonne collaborates with academics to develop curricula that motivate and enhance student learning of key scientific concepts and skills. Her work focuses on how best to use blended learning innovations to support science students in higher education, particularly in their first year of university. Yvonne has led FYE and T&L projects and is a long-term member of the UTS First Year Experience Strategy team. In 2016 Yvonne was awarded a UTS Teaching and Learning Award for her work on developing and supporting academic and professional communication skills in first year Science through an embedded, student-centred, flipped learning approach.

Elaine Huber

Elaine Huber works at the University of Technology Sydney, Australia. She is a senior lecturer in Higher Education Learning Design and works with academics in the Faculty of Science to help them innovate their teaching curriculum. She has a passion for integrating technologies through blended approaches such as the flipped classroom. Elaine is also investigating the praxis of evaluating small-scale learning and teaching projects in higher education for her PhD in Education.

Jorge Reyna

Jorge Reyna is a Lecturer in Higher Education Learning Design in the Faculty of Science at UTS. Jorge is focused on using digital media as an assessment tool to foster deep learning and digital media literacy. Additionally, his area of interest includes flipped classrooms, development of desktop recording lectures that are interactive and engaging for students and inclusive design applied to online learning.

Peter Meier

Associate Professor Peter Meier is the Associate Dean (Teaching and Learning) in the Faculty of Science, UTS. Peter has over 20 years of academic experience including subject coordination, program directorship and leadership roles at Faculty level. Peter has overseen the Faculty wide curriculum review whereby all Science subjects were redesigned to align with the university’s learning.futures initiative. Most recently, Peter led the UTS chapter of the ‘WIL in Science: Leadership for WIL’ Lighthouse Project, which developed an integrated Faculty strategy to extend work-integrated learning (WIL) activities in science and related degrees through curriculum renewal, scaling and development of individual placement programs.


Knowing when to target students with timely academic learning support: Not a minefield with data mining

Concise paper

Download the paper [PDF]

Elizabeth McCarthy
University of Southern Queensland
@elzbthmccrthy

Catch this session

Monday 4 December, 1.50pm - 2.10pm
Stream 4
Room L209

Abstract

The strategic scheduling of timely engagement opportunities with academic learning support, targeting specific student cohorts requires intentional, informed and coordinated planning. Currently these timing decisions appear to be made with a limited student focus, which considers individual course units only as opposed to having an awareness of the schedule constraints imposed by the students’ full course workload. Hence, in order to respect the full student academic workload, and maximise the quantity and quality of opportunities for students to engage with learning advisors, a means to capture and work with the composition and distribution of student full workload is needed. A data mining approach is proposed in this concise paper, where public domain information accessed from the back end HTML language of course unit information webpages is collected and consolidated in graphical form. The resulting visualisation of the students’ academic learning activities provides a quick and convenient means for academics to make informed scheduling decisions. The case study presented describes the implementation of the data mining in the context of discipline specific academic learning advisors at the University of Southern Queensland servicing three campuses under the ‘One-University’ model.

About the Author

Elizabeth McCarthy

Elizabeth McCarthy is a learning advisor, specialising in mathematics skills, and an academic in the mathematics and engineering disciplines with experience of 10 years. She is a mechatronics engineering, machine learning and mathematics enthusiast who is currently working towards her PhD project. For fun, she enjoys coding data science apps and tools to improve access to data for decision making purposes.


Me in a minute: A simple strategy for developing and showcasing personal employability

Concise paper

Download the paper [PDF]

Trina Jorre de St Jorre
Deakin University
@trinajorre

Liz Johnson
Deakin University

Gypsy O'Dea
Deakin University

Catch this session

Monday 4 December, 1.30pm - 1.50pm
Stream 5
Room C204

Abstract

Graduates require evidence of employability beyond marks and grades to differentiate themselves in the highly competitive labour market. Universities cannot guarantee employment, but they can engage students in learning and recognise achievement that is relevant to employment. Here, we share preliminary insights from interviews investigating student perceptions of an extra-curricular video strategy designed to develop and showcase graduate employability. The Me in a Minute video strategy provides students with support to film a one minute video pitch aimed at potential employers. Student perceptions of the strategy suggest that in addition to providing an individualised artefact that can be used to showcase achievement, the strategy engages students in reflection that helps them to better understand and articulate evidence of their achievements relevant to employment. Furthermore, students value the learning associated with pitching, more than the video itself.

About the authors

Trina Jorre de St Jorre

Dr Trina Jorre de St Jorre is a Lecturer in Graduate Employability at Deakin University. She is interested in pedagogies that engage and empower students and her research focus is on assuring graduate capabilities, improving employment outcomes and incorporating the student voice into curriculum development.

Liz Johnson

Liz Johnson is Pro Vice Chancellor, Teaching and Learning at Deakin University where she leads the Deakin Learning Futures, the central divisional team that supports learning and teaching. Liz is a National Teaching Fellow with research interests in work-integrated learning, curriculum renewal and building capability for learning and teaching. Liz is also Director of the Teaching and Learning Centre of the Australian Council of Deans of Science, leading a number of national projects to enhance university teaching in science.

Gypsy O'Dea

Gypsy O’Dea is a Psychology student, Writing Mentor, and Research Assistant at Deakin University. As a Writing Mentor, her focus is on strategies that empower students to become self-directed and independent learners, with a view to improving graduate outcomes. Her current research is with the Australian Temperament Project Generation 3 study, investigating intergenerational predictors of child development.


Quantext: Analysing student responses to short-answer questions

Concise paper

Download the paper [PDF]

Jenny McDonald
University of Auckland
@aggiewil

Adon Moskal
Otago Polytechnic
@AdonMoskal

Catch this session

Monday 4 December, 1.30pm - 1.50pm
Stream 4
Room L209

Abstract

We introduce a web-based tool for teachers to support the rapid analysis of student responses to short answer or mini-essay questions. Designed to support teaching in large-class settings, it aims to bring to practicing teachers analytic tools that can reveal insights in their student text data. We background development of the tool to date, briefly describe its architecture and features, and report on a bench-test evaluation. Finally, we introduce a pilot study to evaluate the tool in classrooms at three NZ universities and one polytechnic. We conclude with options for accessing the tool and outline plans for ongoing development.

About the authors

Jenny McDonald

Dr Jenny McDonald is co-developer of Quantext and a Research Associate at the Centre for Learning and Research (CLeaR), University of Auckland. Jenny is an experienced educational technologist and academic developer. She has particular research interests in natural language processing techniques for formative feedback and learning analytics. She was co-PI with A/Prof. Cathy Gunn on the recent NZ Ako-funded project, ‘Building an evidence-base for teaching and learning design using learning analytics data’.

Adon Moskal

Adon Moskal is co-developer of Quantext and a lecturer in Information Technology at Otago Polytechnic. From 2011-2016, Adon was a Professional Practice Fellow at the University of Otago where he developed software and researched educational technology. His research interests include student evaluation, academic development and learning analytics. Recently, Adon was a co-investigator and co-developer of the Student Relationship Engagement System v2 with researchers from the University of Auckland and the University of Sydney.


Social media in enabling education

Concise paper

Download the paper [PDF]

Susan Hopkins
University of Southern Queensland

Catch this session

Monday 4 December, 1.50pm - 2.10pm
Stream 5
Room C204

Abstract

This paper argues that students from rural and low socio-economic status (LSES) backgrounds, who undertake enabling education, benefit from the social, cultural and network capital which digital, narrative and connective platforms may provide in pre-tertiary teaching and learning. In particular, this paper discusses the trial of the use of the social networking site Facebook as a learning management system within an enabling tertiary preparation program designed to raise the aspirations and widen the participation of economically and geographically disadvantaged young people. It also discusses the role of new media in an approach to Tertiary Preparation which recognises that to succeed in their university study, non-traditional students need to develop not only academic skills and confidence, but the skills and confidence to survive and thrive in the broader networked digital society. The presentation includes updates, images and examples from the author’s most recent use of a closed group Facebook page to facilitate digital literacy, enculturation, engagement, socialisation and social networking among participants in the 2017 Life Literacies program for tertiary preparation students, funded through the Commonwealth Government's Higher Education Participation and Partnership Program (HEPPP) in 2017 (to improve the access, participation and success of students from communities under-represented in higher education).

About the Author

Susan Hopkins

Susan Hopkins is a Lecturer in the Open Access College at the University of Southern Queensland, Ipswich campus. Her research interests include sociological approaches to the education of marginalised and non-traditional students including incarcerated students and LSES students in enabling education.


Using threshold concepts about online teaching to support novice online teachers: Designing professional development guidelines to individually assist academic staff (“me”) and collectively guide the institution (“us”)

Full paper

Download the paper [PDF]

Maria Northcote
Avondale College of Higher Education

Kevin Gosselin
HonorHealth Research Institute, Arizona, USA

Peter Kilgour
Avondale College of Higher Education, NSW, Australia

Catherine McLoughlin
Australian Catholic University, ACT, Australia

Chris Boddey
Avondale College of Higher Education, NSW, Australia

Catch this session

Monday 4 December, 10.30am - 11am
Stream 2
Room R113

Abstract

As online learning expands across the higher education sector, individual university lecturers are required to take on roles that incorporate responsibilities for designing and teaching online courses. Their growing capacities to fulfil these roles are sometimes supported by professional development (PD) programs within their institutions while some staff engage in staff development activities outside their home institutions. These programs and activities may take place within Communities of Practice (CoPs) while others are conducted on an individual basis. While much research has been undertaken into the field of online teaching and learning, including investigations into the most useful technological tools to incorporate into the design of online courses, the design of PD curricula to support the needs of novice teachers of online courses has not been as extensively explored. This paper reports on the outcomes of an Office for Learning and Teaching (OLT) funded project which purposely set out to identify the threshold concepts about online teaching that university lecturers develop as they engage in both the individual and communal aspects of designing and teaching online courses. The paper explains how the identification of threshold concepts about online teaching informed the development of a set of curriculum guidelines for the PD of novice online teachers. Recommendations for the design of PD for individual teachers (at the “me” level) are provided along with recommendations for the institution (at the “us” level).

About the authors

Maria Northcote

Associate Professor Maria Northcote is the Director of the Centre for Advancement of the Scholarship of Teaching and Learning (CASTL) at Avondale College of Higher Education. She is an experienced higher education teacher, leader and researcher and is involved in undergraduate and postgraduate education, and professional development. Some of her research interests include threshold concepts, educational technology, online teaching and professional learning.

Kevin Gosselin

Kevin Gosselin holds a Ph.D. degree in educational psychology from Texas Tech University in Lubbock, TX in 2009. He has held academic appointments as adjunct faculty at Texas Tech University from 2008-2010, a lecturer at The University of Texas at Austin in 2010, and as Assistant Professor of Biostatistics at The University of Texas at Tyler from 2010-2013. In 2013, he was appointed as Assistant Dean for Research and Evidence Based-Practice and Associate Professor in the College of Nursing at Texas A&M Health Science Center. He is currently the Director of Academics and Biostatistics at HonorHealth Research Institute in Scottsdale, Arizona. His research involves distance education, faculty development, research methodology and applied performance psychology.

Peter Kilgour

Dr Peter Kilgour is the Director of the Christian Education Research Centre and a Senior Lecturer in the Faculty of Education, Business and Science at Avondale College of Higher Education. His research areas include teacher education, innovative learning and teaching, assessment in work integrated learning, cultural awareness and mathematics education. He is an educator of 35 years’ experience in four different countries. As a former secondary mathematics teacher, school principal, and school system CEO, he has a passion for innovative learning and has worked to implement this in the higher education setting, in online and on-campus modes. His current teaching responsibilities include multicultural education and professional development for pre-service teachers.

Catherine McLoughlin

Catherine McLoughlin is an Associate Professor with the Faculty of Education at the Australian Catholic University, Canberra. With over 30 years of experience in higher education in Europe, South East Asia, the Middle East, and Australia, she has experience and expertise in a variety of educational settings, with diverse students and across a wide range of cultural contexts. Catherine’s research focuses on e-learning, technology enabled pedagogy in higher education, curriculum design, and global trends in education and teacher professional development. Her current research interests include the use of social networking tools to support learning, networked learning in higher education and knowledge creation processes.

Chris Boddey

Chris Boddey provides eLearning support to teaching staff across the Avondale College of Higher Education campuses and lectures at the Avondale Business School. Chris has a professional background across primary, secondary and tertiary education in both Queensland and New South Wales. Chris has been involved in supporting educational technology innovation for over thirty years and has utilised his experience in education to capitalise on business opportunities in education throughout his career. He has operated a small business in the education sector for over fifteen years and has twenty years’ experience in school governance. Chris is keenly aware of the challenges associated with the changing face of twenty-first century education in a variety of educational settings. His research interests include: professional development curriculum design and delivery, facilitating authentic blended learning environments and addressing barriers to effective ICT integration in education.


Variations in coherence and engagement in students’ experience of blended learning

Full paper

Download the paper [PDF]

Feifei Han
The University of Sydney

Robert Ellis
The University of Sydney

Catch this session

Monday 4 December, 11.30am - 12pm
Stream 4
Room L209

Abstract

We report a study which examines variations in coherence and engagement of 344 first year engineering students’ blended learning experience. Using self-report and observational data sources, we demonstrate that student perceptions of the blended learning environment, academic learning outcomes, and actual engagement with the online learning activities are logically related at the variable level as shown by correlation analysis; and at the level of student groupings of similar learning experience and behaviors, as revealed by cluster, ANOVA, and 2 x 2 contingency analyses. Using self-report data, we found that when students perceived the learning activities in the f2f and online environments were coherent and integrated, they tended to be more engaged with the online learning and to perform relatively higher on the assessment tasks than students with negative perceptions. Using the observational data, students who were more engaged with the online learning tended to perceive that the online learning was well integrated with the f2f learning, that the online contributions were valuable for the whole learning experience, and achieved relatively higher than less engaged students. A 2 x 2 contingency table further revealed a logical relationship between the groupings of students based on the self-report and observational data: moderate and positive association was found between students with coherent perceptions and more engagement; and between students with fragmented perceptions and less engagement with the learning experience. The use of multiple data sources and methods enabled triangulation, strengthened analysis power, and offered a more comprehensive picture of students’ blended learning experience.

About the authors

Feifei Han

Feifei Han currently is an educational researcher at the University of Sydney. Her current research interests comprise of three broad themes: (1) language and literacy education; (2) teaching, learning, and educational technology in higher education, and (3) educational psychology.

Robert Ellis

Robert Ellis is an Associate Professor of Education at the University of Sydney. He has been a funded chief investigative researcher for the Australian Research Council since 2005. His current project is funded up until 2019 (with Goodyear and others). Robert’s research interests focus on the student and teaching experience of e-learning, quality and innovation in higher education and learning spaces. He is author of two books and more than eighty internationally refereed publications, mostly in journal articles. He is also a coordinating editor of the Springer Journal ‘Higher Education’, and co-editor of the new book series for Springer ‘Understanding Teaching and Learning practice’. The goal of his research and scholarship is to achieve meaningful social contributions through translational research outcomes.


Learning analytics: What's in it for me (the teacher) and us (myself and my students)?

Lightning round

Cathy Gunn
University of Auckland
@dr_cathy_gunn

Claire Donald
University of Auckland

Jenny McDonald
University of Auckland

Catch this session

Monday 4 December, 2.30pm - 3pm
Stream 2
Room R113

Abstract

Like many emergent trends in learning technology the potential for learning analytics to benefit teaching and learning is being explored with promising results. However adoption is a slow process and the level of impact on practice is so far disappointing (O’Brien, 2016). Our research found that institutions, researchers and teachers have different perceptions and use different language to talk about learning analytics. We will briefly discuss why this lack of common discourse is a barrier to progress, and runs the risk of ending in more failed expectations such as those described in the annual Gartner Hype Cycle Reports .
In three short presentations, we will describe examples of learning analytics tools and strategies developed to promote their adoption in practice by teachers and learning designers. A guiding principle is to produce easy to use tools that teachers can use or adapt to their own practice (Datnow & Hubbard, 2016; Ferguson et al., 2016). The tools must also serve a useful purpose, e.g. by supporting common tasks or addressing common challenges, and aligning with familiar teaching and assessment cycles.
Links will be provided to the open source tools and creative commons licensed resources produced by a nationally funded learning analytics research project in New Zealand.

About the authors

Cathy Gunn

Cathy Gunn is an Associate Professor at the Centre for Learning and Research in Higher Education (CLeaR) at the University of Auckland. She has been the Head of eLearning, Deputy Director, Acting Director and Principal Researcher, and produced more than 130 scholarly publications. She is an experienced learning technology researcher and an active contributor to international networks. She is a former President and life member of Ascilite.

Claire Donald

Claire Donald is a lecturer and learning designer at the Centre for Learning and Research in Higher Education (CLeaR) at the University of Auckland. She has 25 years’ experience as a learning designer and researcher in higher education, specifically in the fields of science and engineering education, MOOCs, teacher beliefs, learning analytics and learning design.

Jenny McDonald

Jenny McDonald is an independent researcher and a Research Associate at the Centre for Learning and Research (CLeaR), University of Auckland. Jenny has particular research interests in natural language processing techniques for formative feedback and learning analytics but she is broadly interested in the applications