A Faculty Approach to Implementing Advanced, E-Learning Dependent, Formative and Summative Assessment Practices

This chapter describes innovative approaches to E-Learning and related assessment, driven by a Faculty Teaching and Learning Technologies Committee within the Faculty of Pharmacy, Monash University, Australia. Using this group as a driver, we have caused institutional change in a Faculty that was previously quite traditional in its approach to teaching and assessment. The authors implemented a strategy for the pilot testing and broad adoption of innovative technologies, using a purpose-driven approach. They have used a range of technologies to increase the level of formative assessment that occurs during lectures to large student cohorts. They have used an audience response system to allow students to test and improve a range of cognitive skills in an “active” lecture environment; they will present an evaluation of this tool. The authors found that student perceptions of the level of feedback rose with the use of the audience response system, as did their perceived use of critical thinking skills. They further discuss the benefits and limitations of the use of audience response systems within the chapter and discuss our use of E-Learning technologies for summative assessment purposes.

Re-Assessing Validity and Reliability in the E-Learning Environment

Reliability and validity have a well-established place in the development and implementation of educational assessment devices. With the advent of electronic delivery and assessment some of the factors that influence reliability and validity have changed. In order to understand the process involved the authors have suggested that a socio-technical approach to these educational issues gives an economical explanatory system. Within this socio-technical system, the authors show that the way the students extract information from sources is changing to an extent where it is difficult to distinguish between cheating and poor quoting behavior. This has led them to postulate a new classification within validity and reliability – knowledge validity and reliability. They argue that electronic delivery and assessment have not changed their core structures, but rather require revised education and training for both staff and students.

Afterword

Many teachers commonly use assessment as the starting point of their teaching activities because they believe that assessment drives learning and teaching activities. Hence students tend to organise their learning activities around these prescribed assessment tasks. These beliefs and practices have the potential to detract from promoting effective, efficient, and engaging learning. Teachers, in using assessment tasks to orchestrate their teaching activities send out a message to their learners which minimises the importance of the learning experience. Not only does this constrain learners from taking full advantage of the designed learning experience, but with an explicit focus on assessment tasks by teachers, learners tend to adopt coping mechanisms that focus on the assessment task itself, and little else.

Validation of E-Learning Courses in Computer Science and Humanities

In order to offer a unified framework for the empirical assessment of e-learning (EL), this chapter presents findings from three studies conducted at a comprehensive technological university. The first, an archival study, centers on student performance in undergraduate computer science and humanities courses. The second study, a survey given three times within EL classes, investigates the variables of learning style, general expectation, and interaction in student performance. The third study investigates student performance on computer-mediated information literacy. Taken together, these three studies—focusing on archival, process, and performance-based techniques—suggest that a comprehensive assessment model has the potential to yield a depth of knowledge allowing shareholders to make informed decisions on the complexities of asynchronous learning in post-secondary education.

Ensuring Security and Integrity of Data for Online Assessment

Two challenges with online assessment are making sure data collected is secure and authenticating the data source. The first challenge relates to issues such as network security, robustness against attack and data management. The second is currently a significant impediment to widespread implementation of formal online assessment due to difficulties ensuring the identity of the person completing the assessment. In this chapter the authors discuss technical aspects associated with keeping data secure and the implications this has for delivering online assessment. The chapter also examines technologies that can assist with the issue of authenticating the identity of individuals completing online assessments and we provide some practical advice for those considering using online assessment tools. To conclude the chapter, the authors look at technologies likely to be available in the future and examine how these could be used to conduct online assessment that ensures data security and integrity without imposing an unreasonable burden on users.

Learning and Assessment with Virtual Worlds

This chapter provides an introduction to learning and teaching in the virtual world Second Life (SL). It focuses on the nature of the environment and the constructivist cognitive approach to learning that it supports. The authors present detailed accounts of two case studies and provide preliminary analysis of the way in which the environment helps students to achieve both explicit and implicit learning outcomes. The formal assessment for these studies allowed the content, style, narrative and working pattern to be decided by the students. They believe that this approach provides a useful stepping stone between content driven and problem-based teaching techniques. Initial results seem to indicate that students have brought in learning from other areas with a mature approach that enhances their transferable skills in group work, project management and problem based learning. The authors suggest that loosely specified assessments with suitable scaffolding, within the rich environment of Second Life, can be used to help students develop independent, self motivated learning. To support this they map criteria from problembased learning literature and link the learning experience to types of learner.

Is Learning as Effective When Studying Using a Mobile Device Compared to Other Methods?

A concern with E-Learning environments is whether students achieve superior or equivalent learning outcomes to those obtained through traditional methods. In this chapter the authors present the results of a research study comparing students’ learning outcomes with four different delivery methods - printed study material, lecture format, computers and “smart” mobile phones. The results of our study show that learning outcomes are similar when students study by using a computer, mobile phone, or lecture format, while studying with print material yields slightly superior test results. These findings are discussed in the context of the type of learning used in the study and the factors that impact on the effectiveness of using mobile phones for learning purposes, such as learning styles and attitudes to computers. The authors conclude the chapter by briefly discussing developments in mobile technologies and the opportunities they present for mobile learning.

The Validity of Group Marks as a Proxy for Individual Learning in E-Learning Settings

This chapter presents recent research on group assessment in an e-learning environment as an avenue to debate contemporary issues in the design of assessment strategies. The underpinning research measured individual students’ contributions to group processes, individual students’ influence on their peers’ topic understanding of the related curriculum content, and the influence of the overall group experience on personal learning in an e-learning environment designed to act as a catalyst for the group learning. As well, the learning objectives fundamental to the project work were tested individually as part of the final examination. Further, the authors complemented the quantitative aspects of the research with focus group interviews to determine if students perceived that the e-learning environment helped attain the group learning objectives. The authors found that e-learning does not necessarily enhance deep learning in group assignments. They also found that the attainment of group learning objectives does not translate to the attainment of the same individual learning objectives. The chapter provides comment on the relationship that may exist between students’ perceptions of the e-learning environment, the group project work and e-learning group dynamics.

Practical Strategies for Assessing the Quality of Collaborative Learner Engagement

Teachers and designers of computer-networked settings increasingly acknowledge that active learner engagement poses unique challenges, especially for instructors weaned on traditional site-based teaching, and that such engagement is essential to the progressive construction of learner knowledge. “Learner engagement” can mean several things: engagement with material, engagement with instructors, and, perhaps most important, peer engagement. Many teachers of computer-networked courses, who are quite diligent about incorporating activities and procedures to promote human interactivity, are confronted with the challenge of assessing the efficacy of their efforts. How do they discern whether the strategies and tactics woven into their “e-settings” are achieving the desired ends? This chapter outlines issues of self-assessment, including ethical questions. It lays out recommendations for self-assessment in a manner that respects student trust and confidentiality, distinguishing the demands of practical self-assessment from scholarly course research. The institutional pressures from which such assessment emerges are also examined.

Identifying Latent Classes and Differential Item Functioning in a Cohort of E-Learning Students

A differential item functioning analysis is performed on a cohort of E-Learning students undertaking a unit in computational finance. The motivation for this analysis is to identify differential item functioning based on attributes of the student cohort that are unobserved. The authors find evidence that a model containing two distinct latent classes of students is preferred, and identify those examination items that display the greatest level of differential item functioning. On reviewing the attributes of the students in each of the latent classes, and the items and categories that mostly distinguish those classes, the authors conclude that the bias associated with the differential item functioning is related to the a priori background knowledge that students bring to the unit. Based on this analysis, they recommend changes in unit instruction and examination design so as to remove this bias.