scholarly journals Policy in Practice: Teachers’ Conceptualizations of L2 English Oral Proficiency as Operationalized in High-Stakes Test Assessment

Languages ◽  
2021 ◽  
Vol 6 (4) ◽  
pp. 204
Author(s):  
Liliann Byman Frisén ◽  
Pia Sundqvist ◽  
Erica Sandlund
Keyword(s):  
Oral Proficiency ◽  
Analysis Data ◽  
Local Context ◽  
Assessment Practice ◽  
High Stakes ◽  
Scoring Rubrics ◽  
Assessment Guidelines ◽  
English Speaking ◽  
Scoring Rubric ◽  
Policy Transformations

Assessment of foreign/second language (L2) oral proficiency is known to be complex and influenced by the local context. In Sweden, extensive assessment guidelines for the National English Speaking Test (NEST) are offered to teachers, who act as raters of their own students’ performances on this high-stakes L2 English oral proficiency (OP) test. Despite guidelines, teachers commonly construct their own NEST scoring rubric. The present study aims to unveil teachers-as-raters’ conceptualizations, as these emerge from the self-made scoring rubrics, and possible transformations of policy. Data consist of 20 teacher-generated scoring rubrics used for assessing NEST (years 6 and 9). Rubrics were collected via personal networks and online teacher membership groups. Employing content analysis, data were analysed qualitatively to examine (i) what OP sub-skills were in focus for assessment, (ii) how sub-skills were conceptualized, and (iii) scoring rubric design. Results showed that the content and design of rubrics were heavily influenced by the official assessment guidelines, which led to broad consensus about what to assess—but not about how to assess. Lack of consensus was particularly salient for interactive skills. Analysis of policy transformations revealed that teachers’ self-made templates, in fact, lead to an analytic rather than a holistic assessment practice.

Using a Scoring Rubric to Assess the Writing of Bioethics Students

2016 ◽  
Vol 25 (2) ◽  
pp. 301-311
Author(s):  
HUGH A. STODDARD ◽  
CORY A. LABRECQUE ◽  
TOBY SCHONFELD
Keyword(s):  
Performance Assessments ◽  
Theory And Practice ◽  
Cut Scores ◽  
High Stakes ◽  
Skill Sets ◽  
Scoring Rubrics ◽  
Comprehensive Examination ◽  
Scoring Rubric ◽  
Rubric Development ◽  
Laborious Work

Abstract:Educators in bioethics have struggled to find valid and reliable assessments that transcend the “reproduction of knowledge” to target more important skill sets. This manuscript reports on the process of developing and grading a minimal-competence comprehensive examination in a bioethics master’s degree program. We describe educational theory and practice for the creation and deployment of scoring rubrics for high-stakes performance assessments that reduce scoring inconsistencies. The rubric development process can also benefit the program by building consensus among stakeholders regarding program goals and student outcomes.We describe the Structure of the Observed Learning Outcome taxonomy as a mechanism for rubric design and provide an example of how we applied that taxonomy to define pass/fail cut scores. Details about domains of assessment and writing descriptors of performance are also presented. Despite the laborious work required to create a scoring rubric, we found the effort to be worthwhile for our program.

scholarly journals Does Spelling Still Matter—and If So, How Should It Be Taught? Perspectives from Contemporary and Historical Research

2021 ◽  
Author(s):  
Steven C. Pan ◽  
Timothy C. Rickard ◽  
Robert A. Bjork
Keyword(s):  
Approaches To Teaching ◽  
Traditional Methods ◽  
High Stakes ◽  
Early 21St Century ◽  
Relevant Evidence ◽  
Spelling Instruction ◽  
English Speaking ◽  
The Subject ◽  
Traditional Approaches

AbstractA century ago, spelling skills were highly valued and widely taught in schools using traditional methods, such as weekly lists, drill exercises, and low- and high-stakes spelling tests. That approach was featured in best-selling textbooks such as the Horn-Ashbaugh Speller of 1920. In the early 21st century, however, skepticism as to the importance of spelling has grown, some schools have deemphasized or abandoned spelling instruction altogether, and there has been a proliferation of non-traditional approaches to teaching spelling. These trends invite a reevaluation of the role of spelling in modern English-speaking societies and whether the subject should be explicitly taught (and if so, what are research-supported methods for doing so). In this article, we examine the literature to address whether spelling skills are still important enough to be taught, summarize relevant evidence, and argue that a comparison of common approaches to spelling instruction in the early 20th century versus more recent approaches provides some valuable insights. We also discuss the value of explicit spelling instruction and highlight potentially effective ways to implement such instruction, including the use of spelling tests. Overall, our goals are to better characterize the role of spelling skills in today’s society and to identify several pedagogical approaches—some derived from traditional methods and others that are more recent—that hold promise for developing such skills in efficient and effective ways.

Interpreting testing and assessment: A state-of-the-art review

2021 ◽  
pp. 026553222110361
Author(s):  
Chao Han
Keyword(s):  
State Of The Art ◽  
Professional Certification ◽  
Automatic Assessment ◽  
Prospective Students ◽  
Assessment Practice ◽  
High Stakes ◽  
Future Directions ◽  
Assessment Tasks ◽  
Interpreter Education ◽  
Selection Of

Over the past decade, testing and assessing spoken-language interpreting has garnered an increasing amount of attention from stakeholders in interpreter education, professional certification, and interpreting research. This is because in these fields assessment results provide a critical evidential basis for high-stakes decisions, such as the selection of prospective students, the certification of interpreters, and the confirmation/refutation of research hypotheses. However, few reviews exist providing a comprehensive mapping of relevant practice and research. The present article therefore aims to offer a state-of-the-art review, summarizing the existing literature and discovering potential lacunae. In particular, the article first provides an overview of interpreting ability/competence and relevant research, followed by main testing and assessment practice (e.g., assessment tasks, assessment criteria, scoring methods, specificities of scoring operationalization), with a focus on operational diversity and psychometric properties. Second, the review describes a limited yet steadily growing body of empirical research that examines rater-mediated interpreting assessment, and casts light on automatic assessment as an emerging research topic. Third, the review discusses epistemological, psychometric, and practical challenges facing interpreting testers. Finally, it identifies future directions that could address the challenges arising from fast-changing pedagogical, educational, and professional landscapes.

scholarly journals A Study for the Development of Automated Essay Scoring (AES) in Malaysian English Test Environment

2019 ◽  
Vol 9 (1) ◽  
Author(s):  
Wee Sian Wong ◽  
Chih How Bong
Keyword(s):  
Reliability And Validity ◽  
Theoretical Aspect ◽  
Local Context ◽  
Test Environment ◽  
Automated Essay Scoring ◽  
Scoring Rubrics ◽  
Essay Scoring ◽  
Assessment Environment ◽  
Future Work ◽  
Assessment Context

Automated Essay Scoring (AES) is the use of specialized computer programs to assign grades to essays written in an educational assessment context. It is developed to overcome time, cost, and reliability issues in writing assessment. Most of the contemporary AES are “western” proprietary product, designed for native English speakers, where the source code is not made available to public and the assessment criteria may tend to be associated with the scoring rubrics of a particular English test context. Therefore, such AES may not be appropriate to be directly adopted in Malaysia context. There is no actual software development work found in building an AES for Malaysian English test environment. As such, this work is carried out as the study for formulating the requirement of a local AES, targeted for Malaysia's essay assessment environment. In our work, we assessed a well-known AES called LightSide for determining its suitability in our local context. We use various Machine Learning technique provided by LightSide to predict the score of Malaysian University English Test (MUET) essays; and compare its performance, i.e. the percentage of exact agreement of LightSide with the human score of the essays. Besides, we review and discuss the theoretical aspect of the AES, i.e. its state-of-the-art, reliability and validity requirement. The finding in this paper will be used as the basis of our future work in developing a local AES, namely Intelligent Essay Grader (IEG), for Malaysian English test environment.

scholarly journals Improving L2 Oral Accuracy and Grammatical Range Through Self-Assessment of Video Speech Drafts

2018 ◽  
Vol 20 (2) ◽  
pp. 127-142
Author(s):  
Martin Javier Caicedo Pereira ◽  
Jhonny Alexander Lozano Bermúdez ◽  
Luis Alfonso Vanegas Medina
Keyword(s):  
Action Research ◽  
Foreign Language ◽  
University Students ◽  
Oral Language ◽  
Research Study ◽  
Self Assessment ◽  
Self Evaluation ◽  
Scoring Rubrics ◽  
Assessment Techniques ◽  
English Speaking

This action research study examines the effect of self-evaluation on grammatical range and grammar accuracy on the English speaking performance of 27 foreign language university and pre-university students enrolled in three different language centers, in three different cities in Colombia. Participants were asked to record themselves answering different IELTS-type tasks for four times during a 6-week period and score and reflect towards their performance using IELTS-type analytical scoring rubrics and journals. Researchers used journals to register impressions, thoughts, and judgments observed during the process. The findings led to conclude that learners highly benefit from applying self-assessment techniques using videos of their production and a language benchmark to compare with in the improvement of their oral language accuracy and grammatical range.

Who do you hang out with?

2021 ◽  
Vol 6 (1) ◽  
Author(s):  
Yushan Xie ◽  
Yongcan Liu
Keyword(s):  
Study Abroad ◽  
Social Interactions ◽  
Mixed Methods Research ◽  
Oral Proficiency ◽  
Dense Networks ◽  
L2 Development ◽  
Underlying Network ◽  
English Speaking ◽  
Group Relationships ◽  
The Relationship

Abstract Social interactions that take place during study abroad in linguistically rich settings are important for oral proficiency development; however, few studies have explicitly examined the underlying network structure of students’ social experiences in study abroad contexts and its role in oral proficiency development. This mixed-methods research examined the relationship between self-reported social network structures and self-perceived English oral proficiency gains among 88 Chinese international students at a UK university. While the majority of participants’ networks consisted of Chinese-speaking peers, most of them managed to develop strong and frequent relations with English-speaking peers. Students reported gains in oral proficiency, especially in areas related to language use that require higher levels of proficiency. Diverse networks with significant and high-frequency English-speaking relationships were more likely to be associated with L2 oral proficiency gains, while dense networks with only frequent and strong in-group relationships could potentially limit L2 development.

Failed policy attempts for measuring English speaking abilities in college entrance exams

English Today ◽  
2020 ◽  
pp. 1-7
Author(s):  
Yuko Goto Butler ◽  
Jiyoon Lee ◽  
Xiaolin Peng
Keyword(s):  
South Korea ◽  
English Education ◽  
National Level ◽  
College Entrance ◽  
High Stakes ◽  
Ability Test ◽  
Entrance Exams ◽  
College Entrance Exams ◽  
English Speaking ◽  
The Government

English as a foreign language education in East Asia has received repeated criticism for its lack of success in developing sufficient English oral proficiency among its students (Muller at al., 2014). In response to the criticism, the governments of China, Japan and South Korea attempted to include assessment of students’ speaking abilities as part of their high-stakes college entrance exams, hoping for positive washback effects in both primary- and secondary-school English education as well as on shadow education (i.e., non-formal private-sector education). These attempts often failed. In South Korea, a new test called the National English Ability Test (NEAT), which included direct assessment of students’ speaking skill among other skills, was developed in 2012. However, the government's plan to use NEAT to replace the current exam – the Korean College Scholastic Aptitude Test (KCSAT) – was quickly dropped before its implementation. In China, the government has tried to promote more communicative methods of English education through incorporating English speaking test in high-stakes tests such as the Gaokao – college admission tests – in addition to reducing the weight of English in the traditional paper-based exams. However, the policies have received heavy resistance at the regional level and have not been implemented at the national level. In Japan, the government asked universities to accept designated external proficiency tests as part of the Common Test, the existing college entrance exam, in order to make up for the exam's missing speaking component. After a mountain of criticism from test users, implementation of the plan is still pending. In this light, the aim of this paper is to discuss why these policy attempts failed. While these policy attempts occurred in three different contexts, we could see striking underlying commonalities. We argue that these policy attempts were made based on a set of beliefs separate from the reality of the stakeholders (e.g., students, parents and teachers). More specifically, the failures can be largely attributed to the governments’ monolithic view of the English language and their insufficient consideration for equity rather than equality.

scholarly journals PEMBELAJARAN KEAKSARAAN DASAR BERBASIS KEARIFAN LOKAL SEBAGAI UPAYA PEMBERANTASAN BUTA AKSARA

Jurnal Akrab ◽  
2020 ◽  
Vol 11 (2) ◽  
pp. 62-79
Author(s):  
Yusuf Mualo
Keyword(s):  
Learning Outcomes ◽  
Learning Process ◽  
Literacy Learning ◽  
Literacy Education ◽  
Disaster Mitigation ◽  
Local Context ◽  
Learning Approach ◽  
Learning Outcomes Assessment ◽  
Quantum Learning ◽  
Assessment Guidelines

Literacy education is an effort to empower people in general and illiterate people in particular to increase the power available to them by increasing literacy potential. Literacy is a prerequisite for acquiring other knowledge. Thus literacy learning using local wisdom will be in line with the local context, design, process and functionalization of learning outcomes.This learning model was piloted in the Kupang Regency NTB SKB, because it is one of the areas that is high in illiteracy. The purpose of this model is to describe the quality and effectiveness of the script development of basic literacy learning models based on local wisdom, and its tools in the process of learning disaster mitigation in basic literacy education.The product of this model in the form of teaching materials, learning guides, orthopedic guides, guidance of tutors, and learning outcomes assessment guidelines are then used as a reference for SKB institutions in the implementation of basic literacy learning based on local wisdom.The learning process of this model is carried out by the PNF Unit, namely UTPD SKB Kupang Regency by referring to SKL of basic literacy education which emphasizes the dimensions of attitudes, dimensions of knowledge and skills. The approach used in this learning process is the andragogical and quantum learning approach in the learning process, which is followed by 20 people per group, where the number of lesson hours in this learning process is 38 meetings, with 3 hours per meeting. AbstrakPendidikan keaksaraan merupakan upaya pemberdayaan bagi warga masyarakat pada umumnya dan penduduk buta aksara pada khususnya untuk meningkatkan daya yang ada pada mereka dengan meningkatkan potensi keaksaraan. Dimana keaksaraan merupakan prasyarat untuk memperoleh pengetahuan lainnya.  Dengan demikian dalam pembelajaran keaksaraan dengan menggunakan kearifan lokal akan selaras dengan kontek lokal, desain, proses dan fungsionalisasi dari hasil belajar. Model pembelajaran ini diujicobakan di SKB Kabupaten Kupang Provinsi NTT, karena Kupang merupakan salah satu daerah yang termasuk kategori tinggi buta aksara. Tujuan model pembelajaran ini adalah untuk mendeskripsikan kualitas dan kefektifan naskah pengembangan model pembelajaran keaksaraan dasar berbasis kearifan lokal. Adapun produk model ini berupa bahan ajar, panduan pembelajaran, panduan ortek, padanduan tutor, dan panduan penilaian hasil belajar yang  kemudian dijadikan acuan bagi satuan PNF dalam penyelenggaraan pembelajaran keaksaraan dasar berbsis kearifan lokal. Proses Pembelajaran model ini dilakukan oleh Satuan PNF yaitu UTPD SKB Kabupaten Kupang adalah mengacu pada SKL pendidikan keaksaraan dasar yang menekankan pada dimensi sikap, dimensi pengetahuan dan keterampilan, pendekatan yang digunakan dalam proses pembelajaran ini adalah pendekatan andragogi dan quantum learning dalam proses pembelajarannya, yang diikuti sebanyak 20 orang perkelompok, dimana jumlah jam pelajaran didalam proses pembelajaran ini adalah sebanyak 38 kali pertemuan dengan 1 kali pertemuan sebanyak 3 jam pelajaran.

Reconsidering the Use of Scoring Rubrics in Biology Instruction

2015 ◽  
Vol 77 (9) ◽  
pp. 669-675 ◽  
Author(s):  
Xiaowei Tang ◽  
Janet Coffey ◽  
Daniel M. Levin
Keyword(s):  
High School ◽  
Teaching And Learning ◽  
Science Teacher Education ◽  
High School Biology ◽  
Quality Work ◽  
Grade Levels ◽  
Close Analysis ◽  
Scoring Rubrics ◽  
Instructional Use ◽  
Scoring Rubric

Scoring rubrics are widely employed across content areas and grade levels, including in high school biology classes. Besides regular external use within accountability systems, educators also have advanced their instructional use inside classrooms. In recent years, a consensus appears to be emerging in the educational literature that instructional use of rubrics is beneficial for student learning, and numerous examples in the research and practitioner literature establish their importance in teachers’ planning, instruction, and assessment. We examine this assumption through close analysis of students’ use of a scoring rubric in a high school biology classroom. We explore how instructional use of a scoring rubric influences biology teaching and learning activities, what messages about knowledge and learning such use conveys to students, and what influence such use may have on students’ emergent understandings of what constitutes quality in biological thinking and practice. Our analysis suggests that instructional use of scoring rubrics can actually undermine the very learning it is intended to support. We discuss an alternative way to help students understand what constitutes high-quality work, and we draw implications for science teacher education.

Sign in / Sign up

Export Citation Format

Share Document