Can item effects explain away the evidence for unconscious sound symbolism? An adversarial commentary on Heyman, Maerten, Vankrunkelsven, Voorspoels and Moors (2019)

Apparent evidence for unconscious sound symbolism is probably artifactual: Commentary on Heyman, Maerten, Vankrunkelsven, Voorspoels and Moors (in press). Sound-symbolism effects in the absence of awareness: A replication study. Psychological Science, 0956797619875482. https://doi.org/10.1177/0956797619875482

Disentangling Item and Testing Effects in Inoculation Research on Online Misinformation: Solomon Revisited

Online misinformation is a pervasive global problem. In response, psychologists have recently explored the theory of psychological inoculation: If people are preemptively exposed to a weakened version of a misinformation technique, they can build up cognitive resistance. This study addresses two unanswered methodological questions about a widely adopted online “fake news” inoculation game, Bad News. First, research in this area has often looked at pre- and post-intervention difference scores for the same items, which may imply that any observed effects are specific to the survey items themselves (item effects). Second, it is possible that using a pretest influences the outcome variable of interest, or that the pretest may interact with the intervention (testing effects). We investigate both item and testing effects in two online studies (total N = 2,159) using the Bad News game. For the item effect, we examine if inoculation effects are still observed when different items are used in the pre- and posttest. To examine the testing effect, we use a Solomon’s Three Group Design. We find that inoculation interventions are somewhat influenced by item effects, and not by testing effects. We show that inoculation interventions are effective at improving people’s ability to spot misinformation techniques and that the Bad News game does not make people more skeptical of real news. We discuss the larger relevance of these findings for evaluating real-world psychological interventions.

Promotion Optimization for Multiple Items in Supermarkets

Promotions are a critical decision for supermarket managers, who must decide the price promotions for a large number of items. Retailers often use promotions to boost the sales of the different items by leveraging the cross-item effects. We formulate the promotion optimization problem for multiple items as a nonlinear integer program. Our formulation includes several business rules as constraints. Our demand models can be estimated from data and capture the postpromotion dip effect and cross-item effects (substitution and complementarity). Because demand functions are typically nonlinear, the exact formulation is intractable. To address this issue, we propose a general class of integer programming approximations. For demand models with additive cross-item effects, we prove that it is sufficient to account for unilateral and pairwise contributions and derive parametric bounds on the performance of the approximation. We also show that the unconstrained problem can be solved efficiently via a linear program when items are substitutable and the price set has two values. For more general cases, we develop efficient rounding schemes to obtain an integer solution. We conclude by testing our method on realistic instances and convey the potential practical impact for retailers. This paper was accepted by Yinyu Ye, optimization.

Apparent evidence for unconscious sound symbolism is probably artifactual: Commentary on Heyman, Maerten, Vankrunkelsven, Voorspoels and Moors (in press).

Sound symbolism refers to the intuition that a word’s sound should match the characteristics of its referents – e.g., kiki should label something spiky – and its prevalence and systematicity provide compelling evidence for an intuitive mapping between linguistic form and meaning. Striking recent work (Hung, Styles, & Hsieh, 2017) suggests that these mappings may have an unconscious basis, such that participants can compute the fit between a word’s sound and an object’s shape when both are masked from awareness. This surprising finding replicated in the pre-registered report by Heyman, Maerten, Vankrunkelsven, Voorspoels and Moors (2019), with potentially far-reaching implications for the role of awareness in language processing (Hassin, 2013; Rabagliati, Robertson, & Carmel, 2018). However, as I demonstrate, it is an artifact of the stimuli used. Once item effects are accounted for, these data provide no evidence that sound symbolism, and language more generally, can be processed without awareness.

A systematic comparison and reliability analysis of formal measures of sentence acceptability

Understanding and measuring sentence acceptability is of fundamental importance for linguists, but although many measures for doing so have been developed, relatively little is known about some of their psychometric properties. In this paper we evaluate within- and between-participant test-retest reliability on a wide range of measures of sentence acceptability. Doing so allows us to estimate how much of the variability within each measure is due to factors including participant-level individual differences, sample size, response styles, and item effects. The measures examined include Likert scales, two versions of forced-choice judgments, magnitude estimation, and a novel measure based on Thurstonian approaches in psychophysics. We reproduce previous findings of high between-participant reliability within and across measures, and extend these results to a generally high reliability within individual items and individual people. Our results indicate that Likert scales and the Thurstonian approach produce the most stable and reliable acceptability measures and do so with smaller sample sizes than the other measures. Moreover, their agreement with each other suggests that the limitation of a discrete Likert scale does not impose a significant degree of structure on the resulting acceptability judgments.

Systematic and Random Sources of Variability in Perceptual Decision-Making: Comment on Ratcliff, Voskuilen, and McKoon (2018)

A key assumption of models of human cognition is that there is variability in information processing. Evidence accumulation models (EAMs) commonly assume two broad variabilities in information processing: within-trial variability, which is thought to reflect moment-to-moment fluctuations in perceptual processes, and between-trial variability, which is thought to reflect variability in slower-changing processes like attention, or systematic variability between the stimuli on different trials. Recently, Ratcliff, Voskuilen, and McKoon (2018) claimed to “provide direct evidence that external noise is, in fact, required to explain the data from five simple two-choice decision tasks” (p. 33), suggesting that at least some portion of the between-trial variability in information processing is due to “noise”. However, we argue that Ratcliff et al. (2018) failed to distinguish between two different potential sources of between-trial variability: random (i.e., “external noise”) and systematic (e.g., item effects). Contrary to the claims of Ratcliff et al. (2018), we show that “external noise” is not required to explain their findings, as the same trends of data can be produced when only item effects are present. Furthermore, we contend that the concept of “noise” within cognitive models merely serves as a convenience parameter for sources of variability that we know exist, but are unable to account for. Therefore, we question the usefulness of experiments aimed at testing the general existence of “random” variability, and instead suggest that future research should attempt to replace the random variability terms within cognitive models with actual explanations of the process.

An Effective Project for Teaching Repeated-Measures Designs

A project was developed to introduce the core principles of repeated-measures designs. Using the levels of processing approach to memory, students are prompted to engage in either shallow, moderate, or deep processing of 54 common nouns. An unexpected recall task then measures the number of words remembered in each condition. Data from 293 students from two universities across 16 years indicate that the exercise reliably produces significant differences across conditions. This exercise employs several methodological techniques that are used as a basis for class discussion such as counterbalancing, randomized-blocks designs, reduction of carryover and specific-item effects, and elimination of experimenter expectancy effects. In short, most of the design considerations that would be required of a repeated measures approach are represented, and students can identify their purposes and effects from having participated in the demonstration.