The Seven C's of Comprehensive Online Assessment

During the last 6 years, the Cisco Networking Academy™Program has delivered online curricula and over 36 million online assessments to support instructors and schools teaching computer networking skills to students. This chapter describes the context of this work and lessons learned from this endeavor. Through discussions with stakeholders concerning the central aspects of the Cisco Networking Academy Program assessment activities, seven themes have evolved, each starting with the letter C: claims, collaboration, complexity, contextualization, computation, communication, and coordination. These themes address many aspects of assessment, including design, development, delivery, and the management of assessment resources, which are all necessary to ensure a quality assessment program.

Feasibility Studies on Transitioning Assessment Programs from Paper to Pencil to CBT Delivery

A wide range of decision points is described in determining if, how, and when an assessment program should be transitioned from paper and pencil administration to computer-based delivery. The key factors in making effective strategic policy decisions regarding a transition are described on the basis of experiences gained in conducting feasibility studies over the past decade. The feasibility study is conceptualized as a five-phase, partially sequential process in which information learned in one phase can trigger reinterpretation of previous understandings and impact directions taken in subsequent phases. No claim is made that the conduct of a feasibility study will guarantee a seamless transition; instead, a strategic roadmap is provided for navigating bumps in the transition road.

A Computer-Adaptive Mathematics Test Accomodates Third Grade Students with Special Needs in the Pacific Northwest

Assessment of student ability is often clouded by the interaction between content knowledge and prerequisite access skills. These ancillary skills can influence the students’ level of engagement with the test material by limiting their ability to access the test information or respond to the questions. In the case of mathematics, for example, reading and writing are viewed as access skills in that a student is required to be proficient in these skills in order to succeed on the multiple choice mathematics items (Helwig, Rozek-Tedesco, Tindal, Heath, & Almond, 1999). Students with deficiencies in these access skills are unable to demonstrate their knowledge and skills in the construct under investigation by the test (Elliott, Kratochwill, & McKevitt, 2001). To compensate for these access barriers, test accommodations are provided that change the manner in which the test items are delivered, the setting in which the test is taken, the timing of the test, and/or the administration procedures employed during testing. In a traditional paper and pencil test, these changes to the test are only possible by retrofitting existing materials. In an online computerized testing platform, however, accommodations can be embedded within the design and delivery of the items, thereby creating a seamless testing environment that integrates the necessary support structures to provide all students with an equal opportunity to succeed. The purpose of this chapter is to highlight the process we used to create an accommodated mathematics test for third grade students in an online environment.

Using Web-Enabled Technology in a Performance-Based Accreditation Environment

This chapter describes two online systems that support the transition from instructor-based to student-outcome-based education in an engineering technology context. The first system facilitates communication among faculty while establishing a baseline of current curricular practice. This data can be used to identify appropriate measurement points for various student performance outcomes. The second system is designed to collect and display student performance data related to particular educational outcomes in preparation for program accreditation. Data are collected from both students and faculty. The system is scalable, so it can be adapted to various levels of need. This data are being used to assess individual or group performance across several distributed campuses.

Online Assessment of Foreign Language Proficiency

In this chapter, the challenges involved in creating online Arabic and Russian proficiency tests by using a commercial test product are discussed. Guidelines used for item and test development are presented, along with specific challenges test developers faced in designing computerized, semiadaptable tests. Also discussed are the issues involved in delivering the tests securely over the Internet to examinees, who took them on computers in college and university language labs across the United States and abroad. The chapter concludes with a list of five important lessons that could help others who are contemplating a similar test development project.

Continuous Assessment for Improved Student Outcomes

WebCT views assessment as an activity that is integral to the full scope of the learning process. A variety of methods and strategies are available to course designers and instructors to assess student performance before, during, and after a course has taken place. WebCT provides three major categories of assessment tools (self-tests, quizzes and surveys, and assignments within these tools) and seven types of questions (multiple choice, including true/false; combination multiple choice; matching; calculated; short answer; jumbled sentence; and paragraph). The layout, design, and administration of assessments is flexible through selective release, timed assessments, and the sequencing of questions. Through examples from the WebCT Exemplary Course Project, this chapter reviews the many tools and methods available and describes the assessment, grading, and reporting capabilities of WebCT.

Moving Beyond Objective Testing in Online Assessment

Computer-aided assessment (CAA) is traditionally seen as an efficient approach to testing large numbers of students by utilizing objective questions with the emphasis firmly on measurement. This chapter describes the development of a system that also seeks to contribute to improving student learning by enhancing the quality, sophistication, and authenticity of the assessments delivered. The system supports students and tutors through the learning process by providing diagnostic and directed feedback to learners and a clear measurement of their true ability to the teacher. The chapter also looks at current work focused on developing assessment systems that can assess higher order skills and begin to blur the boundary between learning and assessment.

Electronic Tools for Online Assessments

The tools used in assessment allow instructors to communicate performance information to students. Each tool has particular things that it does well or poorly, encouraging particular kinds of assessment and discouraging others. This chapter explores the use of three particular software tools-Rubistar, an assessment database, and a document comparison feature—within a teacher education course. We comment on the tools’ affordances, role in the assessment process, and ability to help instructors model effective practices for teacher education students. We also discuss two measurement issues, construct validity and consequences of use, that pertain to tool use in this environment.

From Paper and Pencil to Computerized

Computerized testing offers many advantages over paper and pencil exams. Advantages can include increased security, greater availability of the exam, rapid scoring, and reduced administrative costs. Possible disadvantages can include the need for computer hardware and software, computer glitches, and computer anxiety. This chapter discusses the advantages and disadvantages of computerized testing followed by a description of the experience of a state-based dental board as they transitioned from a paper and pencil exam to a computerized exam.

Online Assessment Distribution Models for Testing Programs

This chapter discusses three distribution models for online assessments and their characteristics. Since 1999, Pearson VUE has been assisting clients in achieving their goals of providing globally distributed computerized assessments for high-stakes certification and licensure purposes. We have identified three major distribution models for the delivery of online assessments: (a) controlled, (b) global, and (c) ubiquitous. Each of these models is appropriate for some types of online assessments but not others. The characteristics of each model interact with the features of the online assessment program in important ways to either advance or impede the goals of the assessment. Operational program experience shows that these strengths and weaknesses must be analyzed and considered with the assessment goals as part of a good decision-making process.