The implementation of literacy and sign language learning system for deaf children based on the augmented reality

Author(s):  
Jie Jiang ◽  
Yang Kuang
2021 ◽  
Vol 11 (8) ◽  
pp. 3439
Author(s):  
Debashis Das Chakladar ◽  
Pradeep Kumar ◽  
Shubham Mandal ◽  
Partha Pratim Roy ◽  
Masakazu Iwamura ◽  
...  

Sign language is a visual language for communication used by hearing-impaired people with the help of hand and finger movements. Indian Sign Language (ISL) is a well-developed and standard way of communication for hearing-impaired people living in India. However, other people who use spoken language always face difficulty while communicating with a hearing-impaired person due to lack of sign language knowledge. In this study, we have developed a 3D avatar-based sign language learning system that converts the input speech/text into corresponding sign movements for ISL. The system consists of three modules. Initially, the input speech is converted into an English sentence. Then, that English sentence is converted into the corresponding ISL sentence using the Natural Language Processing (NLP) technique. Finally, the motion of the 3D avatar is defined based on the ISL sentence. The translation module achieves a 10.50 SER (Sign Error Rate) score.


2020 ◽  
Vol 7 (1) ◽  
Author(s):  
Xuewang Geng ◽  
Masanori Yamada

Abstract To address the difficulties related to acquiring Japanese compound verbs, which lack the clarity of verb combinations and the opacity of compound verb meanings, we designed and developed an augmented reality (AR) learning system based on image schema and AR animations. We investigated the effects of the AR-based language learning system developed in this study on the learning performance and cognitive load of an AR learning system and paper-based image schema materials. This study also examined the correlation between learning performance and cognitive load. Learners of these two learning methods had significantly improved performance on post-tests. Especially, regarding the retention of knowledge, the AR learning system was more effective. However, there was no significant difference in the perceived cognitive loads between the two learning methods. It is also found that the learning performance of the two learning methods was related to the perceived different types of cognitive load.


2001 ◽  
Vol 22 (4) ◽  
pp. 643-647
Author(s):  
Meredith L. Rowe

The study of gesture, especially its relationship to spoken and signed languages, has become a broadly studied topic for researchers from various fields, including neuropsychology, cognitive psychology, evolutionary psychology, human development, and communication disorders. One possible reason for the wide interest in gesture is its universality. People of all ages and cultures use gestures for various purposes. Young language-learning, hearing children often use gestures alone or in combination with speech to help express themselves to their interlocutors, for example, pointing to a desired object while saying “mine.” As a more striking example, deaf children in Nicaragua who had previously been unexposed to any conventional sign language, used gestures to develop home-sign systems that eventually developed into Nicaraguan Sign Language (Kegl, Senghas, & Coppola, 1999). On the other hand, gestures are often used in situations where the underlying purpose of the gesture is less clear. For example, people who are blind from birth are nonetheless found to gesture in conversation (Iverson & Goldin-Meadow, 1997), and adults gesture frequently, and often subconsciously, during conversations with one another. Despite their omnipresence, we know relatively little about gestures' origins, their relationship to language, and, in some instances, the purposes they serve.


Author(s):  
Tito Sugiharto ◽  
Gentur Priguna

AbstractCommunity Service Activities/ Kegiatan Pengabdian Kepada Masyarakat (PKM) with the theme "Training on Application of Augmented Reality Application for Sign Language Introduction in SLB Negeri Luragung, Cirahayu Village, Luragung Subdistrict, West Java Province" has a target of Teachers and Students in schools. The problems faced by partner institutions in community service activities include: the learning process of Sign Language in SLB Negeri Luragung still uses book media as the main media in learning. Besides learning media provided at SLB Negeri Luragung also by reading utterances through lip movements combined with cued speech.In addition, the problem faced by teachers in schools for deaf students is the development of language in its function as a means of communication, both verbally and signaling. Language development and speech go together with hearing acuity. Due to limited hearing, deaf students do not occur in the process of imitation of sound, imitation process is only limited to visual imitation. Therefore we need an alternative media for learning that can help the process of visual imitation. The absence of additional alternative learning media used in SLB Negeri Luragung that can help teachers and students in helping students communicate and learn easily becomes a special goal in community service activities. Community service aims to train and implement Augmented Reality Technology as an innovation and development of sign language learning media applications in SLB Negeri Luragung. Deaf students in SLB Negeri Luragung are expected to be able to more easily and quickly understand the sign language learning process with the application of learning based on Augmented Reality.Keywords: Augmented Reality, Deaf, Learning, SLB Negeri Luragung, Sign Language


2020 ◽  
Vol 1712 ◽  
pp. 012011
Author(s):  
Lester Wong Sze Ee ◽  
Chandra Reka Ramachandiran ◽  
Rajasvaran Logeswaran

2012 ◽  
Vol 40 (2) ◽  
pp. 391-414 ◽  
Author(s):  
NAJA FERJAN RAMÍREZ ◽  
AMY M. LIEBERMAN ◽  
RACHEL I. MAYBERRY

Children typically acquire their native language naturally and spontaneously at a very young age. The emergence of early grammar can be predicted from children's vocabulary size and composition (Bateset al., 1994; Bates, Bretherton & Snyder, 1998; Bates & Goodman, 1997). One central question in language research is understanding what causes the changes in early language acquisition. Some researchers argue that the qualitative and quantitative shifts in word learning simply reflect the changing character of the child's cognitive maturity (for example, Gentner, 1982), while others argue that the trajectory of early language acquisition is driven by the child's growing familiarity with the language (Gillette, Gleitman, Gleitman & Lederer, 1999; Snedeker & Gleitman, 2004). These hypotheses are difficult to adjudicate because language acquisition in virtually all hearing children begins from birth and occurs simultaneously with cognitive development and brain maturation. The acquisition of sign languages, in contrast, is frequently delayed until older ages. In the USA, over 90% of deaf children are born to hearing parents who do not use sign language (Schein, 1989). As a result, deaf children are often exposed to sign language as a first language at a range of ages well beyond infancy (Mayberry, 2007). In rare cases, some deaf individuals are isolated from all linguistic input until adolescence when they start receiving special services and begin to learn sign language through immersion (Morford, 2003). Case studies of language acquisition in such extreme late first-language (L1) learners provide a unique opportunity to investigate first-language learning. The current study investigates three cases of young teens who are in the early stages of acquiring American Sign Language (ASL) as a first language, to determine what first-language acquisition in adolescence looks like.


Sign in / Sign up

Export Citation Format

Share Document