Uncovering variability in children's concepts and conceptual change

Capturing the structure and development of human conceptual knowledge is a challenging but fundamental task in Cognitive Science. The most prominent approach to uncovering these concepts is Multidimensional scaling (MDS), which has provided insight into the structure of human perception and conceptual knowledge. However, MDS usually requires participants to produce large numbers of similarity judgments, leading to prohibitively long experiments for most developmental research. Furthermore, MDS provides a single psychological space, tailored to a fixed set of stimuli. In contrast, we present a method that learns psychological spaces flexibly and generalizes to novel stimuli. In addition, our approach uses a simple, developmentally appropriate task, which allows for short and engaging developmental studies. We evaluate the feasibility of our approach on simulated data and find that it can uncover the true structure even when the data consists of aggregations of diverse categorizers. We then apply the method to data from the World Color Survey and find that it can discover language-specific color organization. Finally, we use the method in a novel developmental experiment and find age-dependent differences in conceptual spaces for fruit categories. These results suggest that our method is robust and widely applicable in developmental tasks with children as young as four years old.

A Quantum Geometric Model of Color Similarity Judgements

Since Tversky (1977) argued that similarity judgments violate the three metric axioms, asymmetrical similarity judgments have been offered as particularly difficult challenges for standard, geometric models of similarity, such as multidimensional scaling. According to Tversky (1977), asymmetrical similarity judgments are driven by differences in salience or extent of knowledge. However, the notion of salience has been difficult to operationalize to different kinds of stimuli, especially perceptual stimuli for which there are no apparent differences in extent of knowledge. To investigate similarity judgments between perceptual stimuli, across three experiments we collected data where individuals would rate the similarity of a pair of temporally separated color patches. We identified several violations of symmetry in the empirical results, which the conventional multidimensional scaling model cannot readily capture. Pothos et al. (2013) proposed a quantum geometric model of similarity to account for Tversky’s (1977) findings. In the present work, we developed this model to a form that can be fit to similarity judgments. We fit several variants of quantum and multidimensional scaling models to the behavioral data and concluded in favor of the quantum approach. Without further modifications of the model, the quantum model additionally predicted violations of the triangle inequality that we observed in the same data. Overall, by offering a different form of geometric representation, the quantum geometric model of similarity provides a viable alternative to multidimensional scaling for modeling similarity judgments, while still allowing a convenient, spatial illustration of similarity.

Emergent dimensions underlying human understanding of the reachable world

Near-scale, reach-relevant environments, like work desks, restaurant place settings or lab benches, are the interface of our hand-based interactions with the world. How are our conceptual representations of these environments organized? For navigable-scale scenes, global properties such as openness, depth or naturalness have been identified, but the analogous organizing principles for reach-scale environments are not known. To uncover such principles, we obtained 1.25 million odd-one-out behavioral judgments on image triplets assembled from 990 reachspace images. Images were selected to comprehensively sample the variation both between and within reachspace categories. Using data-driven modeling, we generated a 30-dimensional embedding which predicts human similarity judgments among the images. First, examination of the embedding dimensions revealed key properties that distinguish among reachspaces, relating to their structural layout, affordances, visual appearances and functional roles. Second, clustering analyses performed over the embedding revealed four distinct interpretable classes of reachspaces, with separate clusters for spaces related to food, electronics, analog activities, and storage or display. Finally, we found that the similarity structure among reachspace images was better predicted by the function of the spaces than their locations, suggesting that reachspaces are largely conceptualized in terms of the actions they are designed to support. Altogether, these results reveal the behaviorally-relevant principles that that structure our internal representations of reach-relevant environments.

How much is a cow like a meow? A novel database of human judgements of audiovisual semantic relatedness

Semantic information about objects, events, and scenes influences how humans perceive, interact with, and navigate the world. Most evidence in support of semantic influence on cognition has been garnered from research conducted with an isolated modality (e.g., vision, audition). However, the influence of semantic information has not yet been extensively studied in multisensory environments potentially because of the difficulty in quantification of semantic relatedness. Past studies have primary relied on either a simplified binary classification of semantic relatedness based on category or on algorithmic values based on text corpora rather than human perceptual experience and judgement. With the aim to accelerate research into multisensory semantics, we created a constrained audiovisual stimulus set and derived similarity ratings between items within three categories (animals, instruments, household items). A set of 140 participants provided similarity judgments between sounds and images. Participants either heard a sound (e.g., a meow) and judged which of two pictures of objects (e.g., a picture of a dog and a duck) it was more similar to, or saw a picture (e.g., a picture of a duck) and selected which of two sounds it was more similar to (e.g., a bark or a meow). Judgements were then used to calculate similarity values of any given cross-modal pair. The derived and reported similarity judgements reflect a range of semantic similarities across three categories and items, and highlight similarities and differences among similarity judgments between modalities. We make the derived similarity values available in a database format to the research community to be used as a measure of semantic relatedness in cognitive psychology experiments, enabling more robust studies of semantics in audiovisual environments.

Disentangling dimensions of animacy in human brain and behaviour

The perception of animate things is of great behavioural importance to humans. Despite the prominence of the distinct brain and behavioural responses to animate and inanimate things, however, it remains unclear which of several commonly entangled properties underlie these observations. Here, we investigate the importance of five dimensions of animacy: being alive, looking like an animal, having agency, having mobility, and being unpredictable in brain (fMRI, EEG) and behaviour (property and similarity judgments) of 19 subjects using a stimulus set of 128 images that disentangles the five dimensions (optimized by a genetic algorithm). Our results reveal a differential pattern across brain and behaviour. The living/non-living distinction (being alive) was prominent in judgments, but despite its prominence in neuroscience literature, did not explain variance in brain representations. The other dimensions of animacy explained variance in both brain and behaviour. The having agency dimension explained more variance in higher-level visual areas, consistent with higher cognitive contributions. The being unpredictable dimension instead captured representations in both lower and higher-level visual cortex, possibly because unpredictable things require attention. Animacy is multidimensional and our results show that distinct dimensions are differentially represented in human brain and behaviour.

Framing attributes in similarity judgments and intertemporal choices

Similarity models provide an alternative approach to intertemporal choice. Instead of calculating an overall value for options, decision makers compare the similarity of option attributes and make a decision based on similarity. Similarity judgments for reward amounts and time delays depend on both the numerical difference (x2-x1) and ratio (x1/x2) of quantitative values. Changing units of these attribute values (e.g., days vs. weeks) can alter the numerical difference while maintaining the ratio. For example, framing a pair of delays in the unit of weeks (1 vs. 2) or days (7 vs. 14) both result in a ratio of 1/2. Yet the numerical difference between the delays differs depending on the unit (1 for weeks and 7 for days). Here we had participants make similarity judgments and intertemporal choices with amounts framed as dollars or cents and delays framed as days or weeks. We predicted that they units of amounts and delays would influence similarity judgments which would then influence intertemporal choices. We found that participants judged amounts framed as cents as less similar than dollars, and this resulted in more patient intertemporal choices. Additionally, they judged delays framed as weeks as more similar than days, but the framing did not influence choice. These findings suggest that the units in which amounts and delays are framed can influence their similarity judgments, which can shape intertemporal choices. These unit effects may guide stakeholders in framing aspects of intertemporal choices in different units to nudge decision makers into either more impulsive or patient choice.

Proceed with Caution

It is becoming more common that the decision-makers in private and public institutions are predictive algorithmic systems, not humans. This article argues that relying on algorithmic systems is procedurally unjust in contexts involving background conditions of structural injustice. Under such nonideal conditions, algorithmic systems, if left to their own devices, cannot meet a necessary condition of procedural justice, because they fail to provide a sufficiently nuanced model of which cases count as relevantly similar. Resolving this problem requires deliberative capacities uniquely available to human agents. After exploring the limitations of existing formal algorithmic fairness strategies, the article argues that procedural justice requires that human agents relying wholly or in part on algorithmic systems proceed with caution: by avoiding doxastic negligence about algorithmic outputs, by exercising deliberative capacities when making similarity judgments, and by suspending belief and gathering additional information in light of higher-order uncertainty.

The Treachery of Images: Objects, Pictures, Words and the Role of Affordances in Similarity Judgements

Categorization is a fundamental cognitive strategy employed to ease information processing and to aid memory formation. Past research on how humans categorize objects has used images of objects as experimental stimuli. Results suggest these stimuli are categorized based on abstract linguistic concepts. Concurrently, studies in the past 10 years have found differences in the processing of images as compared to real-world objects. One proposed explanation is that these results are due to differences in the affordances of images versus objects. Using a similarity judgement paradigm, we have explored the effect of affordances in a categorization task including words (object names), images, and objects. Consistent with previous research, we found significant differences in how participants made similarity judgements of images and objects. Moreover, we found that similarity judgments using object names were much more similar to the judgments of pictures than of objects. An exploratory cluster analysis opens the possibility of framing such differences as affordance driven. These results suggest a need for more ecologically valid categorization tasks, more conservative inferences when using images as stimuli in these tasks, and the need for further exploring the role of affordances in categorization.

Improved prediction of behavioral and neural similarity spaces using pruned DNNs

Deep Neural Networks (DNNs) have become an important tool for modeling brain and behaviour. One key area of interest has been to apply these networks to model human similarity judgements. Several previous works have used the embeddings from the penultimate layer of vision DNNs and showed that a reweighting of these features improves the fit between human similarity judgments and DNNs. These studies underline the idea that these embeddings form a good basis set but lack the correct level of salience. Here we re-examined the grounds for this idea and on the contrary, we hypothesized that these embeddings, beyond forming a good basis set, also have the correct level of salience to account for similarity judgments. It is just that the huge dimensional embedding needs to be pruned to select those features relevant for the considered domain for which a similarity space is modeled. In Study 1 we supervised DNN pruning based on a subset of human similarity judgments. We found that pruning: i) improved out-of-sample prediction of human similarity judgments from DNN embeddings, ii) produced better alignment with WordNet hierarchy, and iii) retained much higher classification accuracy than reweighting. Study 2 showed that pruning by neurobiological data is highly effective in improving out-of-sample prediction of brain-derived representational dissimilarity matrices from DNN embeddings, at times fleshing out isomorphisms not otherwise observable. Pruning supervised by human brain/behavior therefore effectively identifies alignable dimensions of semantic knowledge between DNNs and humans and constitutes an effective method for understanding the organization of knowledge in neural networks.