User Feedback Analysis of an AI-Enabled CBT Mental Health Application (Wysa) (Preprint)

BACKGROUND Digital mental health applications (apps) are rapidly becoming a common source of accessible support across the world, but their effectiveness is often influenced by limited helpfulness and engagement. There is currently a scarcity of research exploring user engagement in digital mental health applications, especially in the space of artificial intelligence (AI) guided applications. OBJECTIVE The study’s primary objective was to analyze feedback content to understand the user’s experiences of engaging with a digital mental health app. As a secondary objective, an exploratory analysis captured the types of mental health app users. METHODS This study utilized a user-led approach to understanding factors for engagement and helpfulness in digital mental health by analyzing feedback (n=7,929) reported on Google Play Store about Wysa, a mental health app (1 year period). The analysis of keywords in user feedback categorized and evaluated the reported user experience into the core domains of acceptability, usability, usefulness, and integration. The study also captured key deficits and strengths of the app, and explored salient characteristics of the types of users who benefit from accessible digital mental health support. RESULTS The analysis of user feedback found the app to be overwhelmingly positively reviewed (84.4% 5-star rating). The themes of engaging exercises, interactive interface and AI-conversational ability indicated the acceptability of the app, while the non-judgementality and ease of conversation highlighted its usability. The app’s usefulness was portrayed by themes such as improvement in mental health, convenient access and cognitive restructuring exercises. Themes of Privacy and Confidentiality underscored users’ preference for the integrated aspects of the app. Further analysis revealed 4 predominant types of individuals who shared app feedback on the store. CONCLUSIONS Users reported therapeutic elements of a comfortable, safe, and supportive environment through using the digital mental health app. Digital mental health apps may expand mental health access to those unable to access traditional forms of mental health support and treatments.

The Influence of Feedback Content and Feedback Time on Multimedia Learning Achievement of College Students and Its Mechanism

This study investigated the content of feedback (adaptive, elaborated, and knowledge of correct response, KCR) and time (Immediate and Delayed) influences on multimedia learning of college students. Students from the Northeast Normal University (N = 157) were randomly assigned to one of the six experimental conditions. We tried to explain the influence mechanisms of different feedback effects through subjectively reported motivation, cognitive load, and eye movement trajectory during the feedback period. The results showed that (1) different feedback methods in terms of feedback time and feedback content have significantly different effects on scores. Among them, scores of the immediate feedback group were significantly higher than those of the delayed feedback group, and the scores of the adaptive feedback (AF) group were significantly higher than those of the elaborated feedback (EF) group and the knowledge of the correct response feedback group. (2) Different types of feedback contents have significantly different effects on motivation. The motivation scores reported by the AF group and EF group were significantly higher than those reported by the knowledge of the correct response feedback group. (3) Different feedback methods in terms of feedback time and feedback content had significantly different effects on subjective germane cognitive load reports. Among them, the germane cognitive load score of the immediate feedback group was significantly higher than that of the delayed feedback group. The germane cognitive load scores reported by the AF group were significantly higher than those reported by the EF group and knowledge of the correct response feedback group. (4) The germane cognitive load plays a partial mediating role between the AF and post-test scores. (5) Different feedback methods in feedback time have different effects on eye movement fixation trajectory, which shows that the subjects in the immediate feedback group were significantly less than those in the delayed feedback group in fixation count and fixation time in the interest area of the stem. (6) Consistent with our hypothesis, different feedback methods in feedback content have different effects on eye movement trajectory. In summary, the results show that the AF initiated in this study has a positive effect on multimedia learning of college students; it not only provides empirical evidence for cognitive load theory but also helps educators design adaptive learning feedback according to responses of students.

Usability and user evaluation of an integrated multimedia e-learning management system

The usability of multimedia E-Learning management systems (MEMSs) is critical as it leverages institutions in the educational value chain. The paper aims to investigate the usability effect of an integrated MEMS called Towards Student-Centered Integration of Multimedia E-Learning (TSIME) used at the University of Zimbabwe. This study adopted a quantitative approach, where two self-administered questionnaires were used to gather data on a sample of students and lecturers (n = 492) across ten faculties. Design quality, interaction and feedback, content availability and ethical issues were key independent variable constructs while attitude towards use, overall satisfaction, and likelihood to use TSIME being dependent variables. The results were analysed using descriptive and inferential statistics. The structural Equation Model was used to measure how multimedia design features influence overall satisfaction and likelihood to use. The results showed that ethical issues and content availability were not influencing the use of TSIME while attitude influenced overall usage and satisfaction. Interaction and feedback, as well as design quality, have a significant effect on the likelihood to use TSIME.

Innovative Practices in Instructor E-feedback: A Case Study of E-feedback given in Three Linguistic Courses during the COVID 19 Pandemic

This study aimed to investigate the instructor’s electronic feedback practices during the COVID 19 Pandemic in terms of the nature of the content of e-feedback, the formulation, the challenges, and the multimodal nature of the instructor’s e-feedback. This study used a qualitative case study to obtain data from the instructor’s e-feedback in three linguistic courses as delivered, practiced by the single English language instructor. The instructor’s e-feedback via Blackboard and WhatsApp platform and the follow-up interview were analyzed qualitatively. The findings indicate that (1) the highest number of instructor’s e-feedback focused on global issues as compared to local issues, (2) the instructor composed his e-feedback in the form of eight main categories: explanations, suggestions, clarifications, questioning, repetitions, statements, praises, and commands, (3) the instructor used more screencasts for providing e-feedback, followed by written and audio modes respectively. The thematic analysis (4) revealed the instructor’s positive impression on providing e-feedback through these interactive modes (written, audio, and screencast) and a range of challenging issues such as students’ preference issues, technical issues, timing issues, financial and areal issues. This study is significant because it provides us with a comprehensive picture of the patterns of the feedback content, the formulation of the e-feedback, the multimodality of the instructor’s e-feedback, and the significant issues that emerged from the instructor’s e-feedback practices. However, further research should include a relative group of instructors to determine the impact of e-feedback on learners.

Innovative Practices in Instructor E-feedback: A Case Study of E-feedback given in Three Linguistic Courses during the COVID 19 Pandemic

This study aimed to investigate the instructor’s electronic feedback practices during the COVID 19 Pandemic in terms of the nature of the content of e-feedback, the formulation, the challenges, and the multimodal nature of the instructor’s e-feedback. This study used a qualitative case study to obtain data from the instructor’s e-feedback in three linguistic courses as delivered, practiced by the single English language instructor. The instructor’s e-feedback via Blackboard and WhatsApp platform and the follow-up interview were analyzed qualitatively. The findings indicate that (1) the highest number of instructor’s e-feedback focused on global issues as compared to local issues, (2) the instructor composed his e-feedback in the form of eight main categories: explanations, suggestions, clarifications, questioning, repetitions, statements, praises, and commands, (3) the instructor used more screencasts for providing e-feedback, followed by written and audio modes respectively. The thematic analysis (4) revealed the instructor’s positive impression on providing e-feedback through these interactive modes (written, audio, and screencast) and a range of challenging issues such as students’ preference issues, technical issues, timing issues, financial and areal issues. This study is significant because it provides us with a comprehensive picture of the patterns of the feedback content, the formulation of the e-feedback, the multimodality of the instructor’s e-feedback, and the significant issues that emerged from the instructor’s e-feedback practices. However, further research should include a relative group of instructors to determine the impact of e-feedback on learners.

Analysis of Supervisors' Feedback to Residents on Communicator, Collaborator, and Professional Roles During Case Discussions

ABSTRACT Background Literature examining the feedback supervisors give to residents during case discussions in the realms of communication, collaboration, and professional roles (intrinsic roles) focuses on analyses of written feedback and self-reporting. Objectives We quantified how much of the supervisors' verbal feedback time targeted residents' intrinsic roles and how well feedback time was aligned with the role targeted by each case. We analyzed the educational goals of this feedback. We assessed whether feedback content differed depending on whether the residents implied or explicitly expressed a need for particular feedback. Methods This was a mixed-methods study conducted from 2017 to 2019. We created scripted cases for radiology and internal medicine residents to present to supervisors, then analyzed the feedback given both qualitatively and quantitatively. The cases were designed to highlight the CanMEDS intrinsic roles of communicator, collaborator, and professional. Results Radiologists (n = 15) spent 22% of case discussions providing feedback on intrinsic roles (48% aligned): 28% when the case targeted the communicator role, 14% for collaborator, and 27% for professional. Internists (n = 15) spent 70% of discussions on intrinsic roles (56% aligned): 66% for communicator, 73% for collaborator, and 72% for professional. Radiologists' goals were to offer advice (66%), reflections (21%), and agreements (7%). Internists offered advice (41%), reflections (40%), and clarifying questions (10%). We saw no consistent effects when residents explicitly requested feedback on an intrinsic role. Conclusions Case discussions represent frequent opportunities for substantial feedback on intrinsic roles, largely aligned with the clinical case. Supervisors predominantly offered monologues of advice and agreements.

The types and effects of feedback received by emergency ambulance staff: protocol for a systematic mixed studies review with narrative synthesis

PurposeThe primary aim of this systematic review is to identify, describe and synthesise the published literature on the types and effects of feedback received by emergency ambulance staff. The secondary aim will be to describe the mechanisms and moderators of the effects of prehospital feedback in an organisational context.Design/methodology/approachThe application and effects of feedback for healthcare professionals, to support improved practice, is well researched within the wider healthcare domain. Within a prehospital context, research into feedback has been developing in specific areas such as automated feedback from defibrillators and debrief after simulation. However, to date there has been no systematic review published on the types and effects of feedback available to emergency ambulance staff.FindingsThis study will be a systematic mixed studies review including empirical primary research of qualitative, quantitative and mixed methods methodology published in peer-reviewed journals in English. Studies will be included if they explore the concept of feedback as defined in this review, i.e. the systematised provision of information to emergency ambulance staff regarding their performance within prehospital practice and/or patient outcomes. The search strategy will consist of three facets: ambulance staff synonyms, feedback synonyms and feedback content. The databases to be searched from inception are MEDLINE, Embase, AMED, PsycINFO, HMIC, CINAHL and Web of Science. Study quality will be appraised using the mixed methods appraisal tool (MMAT) developed by Hong et al. (2018). Data analysis will consist of narrative synthesis guided by Popay et al. (2006) following a parallel-results convergent synthesis design.Originality/valueRegistration: PROSPERO (CRD42020162600)

Defining Feedback: Understanding Students’ Perceptions of Feedback in the Introductory Communication Course

Feedback is an essential part of the teaching/learning processes. This statement is especially true in the introductory communication course where students receive feedback throughout the presentational speaking process. This paper explores how students define useful feedback based on 1,600 qualitative questionnaires that asked students about their perceptions of feedback. A thematic analysis of a randomly selected subset of 163 responses uncovered two themes: (1) feedback content characteristics (e.g., specific, constructive, praiseworthy, and purposive) and (2) process of instructor-provided feedback (e.g., iterative, timely). Based on these findings, a set of best practices for providing feedback is offered as a means to improve teaching/learning in the introductory communication course.

Postgraduate students’ perceptions of what makes for effective assessment feedback: a case study of a clinical masters course

The purpose of this study was to examine postgraduate students’ perceptions of assessment feedback. Using the Critical Incident Technique, students enrolled on a taught clinical course were asked for their perceptions of effective and ineffective examples of assessment feedback. The data were analysed using thematic analysis and nine themes emerged that capture perceptions associated with feedback content and feedback process. Students perceived effective feedback if it was specific and clear, using positive tone of language. They expressed a preference for feedback that is delivered in a standardised format, reflecting the grades given, individualised, and when the marking criteria is explicit and enables dialogue with the marker. Students perceived feedback to be ineffective when it focused on grammatical errors rather than content, when it was provided by anonymous graders and if it was too personal. Timeliness of feedback was also important to participants. Practical implications and suggestions for future research are highlighted in this paper.

UI/UX Design for Metora: A Gamification of Learning Journalism Interviewing Method

In learning, effective media is needed to deliver the material. Visual media can be more acceptable and facilitate understanding and strengthen memory. In this modern era, digital-based visual media is more effectively captured by students. Educational games in the form of gamification can provide the same effect as effective learning media. To support educational games, interface design and user experience are the first aspects seen by users. This study explored the user interface and user experience in the process of developing a gamification of a non-game subject, Journalism Interviewing Method. User interface includes colour, shape and layout the interface. User experience in game design process includes the tangible user interface, constructive and helpful feedback, content and storytelling, how to appreciate user. This study explored the user interface and user experience in the process of developing a gamification of a non-game subject, Journalism Interviewing Method.