Panoramic visual statistics shape retina-wide organization of receptive fields

Visual systems have adapted to the structure of natural stimuli. In the retina, center-surround receptive fields (RFs) of retinal ganglion cells (RGCs) appear to efficiently encode natural sensory signals. Conventionally, it has been assumed that natural scenes are isotropic and homogeneous; thus, the RF properties are expected to be uniform across the visual field. However, natural scene statistics such as luminance and contrast are not uniform and vary significantly across elevation. Here, by combining theory and novel experimental approaches, we demonstrate that this inhomogeneity is exploited by RGC RFs across the entire retina to increase the coding efficiency. We formulated three predictions derived from the efficient coding theory: (i) optimal RFs should strengthen their surround from the dimmer ground to the brighter sky, (ii) RFs should simultaneously decrease their center size and (iii) RFs centered at the horizon should have a marked surround asymmetry due to a stark contrast drop-off. To test these predictions, we developed a new method to image high-resolution RFs of thousands of RGCs in individual retinas. We found that the RF properties match theoretical predictions, and consistently change their shape from dorsal to the ventral retina, with a distinct shift in the RF surround at the horizon. These effects are observed across RGC subtypes, which were thought to represent visual space homogeneously, indicating that functional retinal streams share common adaptations to visual scenes. Our work shows that RFs of mouse RGCs exploit the non-uniform, panoramic structure of natural scenes at a previously unappreciated scale, to increase coding efficiency.

Visual Recognition of Traffic Signs in Natural Scenes Based on Improved RetinaNet

Aiming at recognizing small proportion, blurred and complex traffic sign in natural scenes, a traffic sign detection method based on RetinaNet-NeXt is proposed. First, to ensure the quality of dataset, the data were cleaned and enhanced to denoise. Secondly, a novel backbone network ResNeXt was employed to improve the detection accuracy and effection of RetinaNet. Finally, transfer learning and group normalization were adopted to accelerate our network training. Experimental results show that the precision, recall and mAP of our method, compared with the original RetinaNet, are improved by 9.08%, 9.09% and 7.32%, respectively. Our method can be effectively applied to traffic sign detection.

Efficient coding of natural scenes improves neural system identification

Neural system identification aims at learning the response function of neurons to arbitrary stimuli using experimentally recorded data, but typically does not leverage coding principles such as efficient coding of natural environments. Visual systems, however, have evolved to efficiently process input from the natural environment. Here, we present a normative network regularization for system identification models by incorporating, as a regularizer, the efficient coding hypothesis, which states that neural response properties of sensory representations are strongly shaped by the need to preserve most of the stimulus information with limited resources. Using this approach, we explored if a system identification model can be improved by sharing its convolutional filters with those of an autoencoder which aims to efficiently encode natural stimuli. To this end, we built a hybrid model to predict the responses of retinal neurons to noise stimuli. This approach did not only yield a higher performance than the stand-alone system identification model, it also produced more biologically-plausible filters. We found these results to be consistent for retinal responses to different stimuli and across model architectures. Moreover, our normatively regularized model performed particularly well in predicting responses of direction-of-motion sensitive retinal neurons. In summary, our results support the hypothesis that efficiently encoding environmental inputs can improve system identification models of early visual processing.

Edge Detection in Natural Scenes Inspired by the Speed Drawing Challenge

Edge detection is a major step in several computer vision applications. Edges define the shape of objects to be used in a recognition system, for example. In this work, we introduce an approach to edge detection inspired by a challenge for artists: the Speed Drawing Challenge. In this challenge, a person is asked to draw the same figure in different times (as 10[Formula: see text]min, 1[Formula: see text]min and 10[Formula: see text]s); at each time, different levels of details are drawn by the artist. In a short time stamp, just the major elements remain. This work proposes a new approach for producing images with different amounts of edges representing different levels of relevance. Our method uses superpixel to suppress image details, followed by Globalized Probability of Boundary (gPb) and Canny edge detection algorithms to create an image containing different number of edges. After that, an edge analysis step detects whose edges are the most relevant for the scene. The results are presented for the BSDS500 dataset and they are compared to other edge and contour detection algorithms by quantitative and qualitative means with very satisfactory results.

A mechanistically interpretable model of the retinal neural code for natural scenes with multiscale adaptive dynamics

The visual system processes stimuli over a wide range of spatiotemporal scales, with individual neurons receiving input from tens of thousands of neurons whose dynamics range from milliseconds to tens of seconds. This poses a challenge to create models that both accurately capture visual computations and are mechanistically interpretable. Here we present a model of salamander retinal ganglion cell spiking responses recorded with a multielectrode array that captures natural scene responses and slow adaptive dynamics. The model consists of a three-layer convolutional neural network (CNN) modified to include local recurrent synaptic dynamics taken from a linear-nonlinear-kinetic (LNK) model \cite{ozuysal2012linking}. We presented alternating natural scenes and uniform field white noise stimuli designed to engage slow contrast adaptation. To overcome difficulties fitting slow and fast dynamics together, we first optimized all fast spatiotemporal parameters, then separately optimized recurrent slow synaptic parameters. The resulting full model reproduces a wide range of retinal computations and is mechanistically interpretable, having internal units that correspond to retinal interneurons with biophysically modeled synapses. This model allows us to study the contribution of model units to any retinal computation, and examine how long-term adaptation changes the retinal neural code for natural scenes through selective adaptation of retinal pathways.

Visual Servoing of Unknown Objects for Family Service Robots

Aiming at satisfying the increasing demand of family service robots for housework, this paper proposes a robot visual servoing scheme based on the randomized trees to complete the visual servoing task of unknown objects in natural scenes. Here, “unknown” means that there is no prior information on object models, such as template or database of the object. Firstly, an object to be manipulated is randomly selected by user prior to the visual servoing task execution. Then, the raw image information about the object can be obtained and used to train a randomized tree classifier online. Secondly, the current image features can be computed using the well-trained classifier. Finally, the visual controller can be designed according to the error of image feature, which is defined as the difference between the desired image features and current image features. Five visual positioning of unknown objects experiments, including 2D rigid object and 3D non-rigid object, are conducted on a MOTOMAN-SV3X six degree-of-freedom (DOF) manipulator robot. Experimental results show that the proposed scheme can effectively position an unknown object in complex natural scenes, such as occlusion and illumination changes. Furthermore, the developed robot visual servoing scheme has an excellent positioning accuracy within 0.05 mm positioning error.

Visual Discomfort and Variations in Chromaticity in Art and Nature

Visual discomfort is related to the statistical regularity of visual images. The contribution of luminance contrast to visual discomfort is well understood and can be framed in terms of a theory of efficient coding of natural stimuli, and linked to metabolic demand. While color is important in our interaction with nature, the effect of color on visual discomfort has received less attention. In this study, we build on the established association between visual discomfort and differences in chromaticity across space. We average the local differences in chromaticity in an image and show that this average is a good predictor of visual discomfort from the image. It accounts for part of the variance left unexplained by variations in luminance. We show that the local chromaticity difference in uncomfortable stimuli is high compared to that typical in natural scenes, except in particular infrequent conditions such as the arrangement of colorful fruits against foliage. Overall, our study discloses a new link between visual ecology and discomfort whereby discomfort arises when adaptive perceptual mechanisms are overstimulated by specific classes of stimuli rarely found in nature.

Pareidolia in Schizophrenia and Bipolar Disorder

While there are many studies on pareidolia in healthy individuals and patients with schizophrenia, to our knowledge, there are no prior studies on pareidolia in patients with bipolar disorder. Accordingly, in this study, we, for the first time, measured pareidolia in patients with bipolar disorder (N = 50), and compared that to patients with schizophrenia (N = 50) and healthy controls (N = 50). We have used (a) the scene test, which consists of 10 blurred images of natural scenes that was previously found to produce illusory face responses and (b) the noise test which had 32 black and white images consisting of visual noise and 8 images depicting human faces; participants indicated whether a face was present on these images and to point to the location where they saw the face. Illusory responses were defined as answers when observers falsely identified objects that were not on the images in the scene task (maximum illusory score: 10), and the number of noise images in which they reported the presence of a face (maximum illusory score: 32). Further, we also calculated the total pareidolia score for each task (the sum number of images with illusory responses in the scene and noise tests). The responses were scored by two independent raters with an excellent congruence (kappa > 0.9). Our results show that schizophrenia patients scored higher on pareidolia measures than both healthy controls and patients with bipolar disorder. Our findings are agreement with prior findings on more impaired cognitive processes in schizophrenia than in bipolar patients.

Rat sensitivity to multipoint statistics is predicted by efficient coding of natural scenes

Efficient processing of sensory data requires adapting the neuronal encoding strategy to the statistics of natural stimuli. Previously, in Hermundstad et al., 2014, we showed that local multipoint correlation patterns that are most variable in natural images are also the most perceptually salient for human observers, in a way that is compatible with the efficient coding principle. Understanding the neuronal mechanisms underlying such adaptation to image statistics will require performing invasive experiments that are impossible in humans. Therefore, it is important to understand whether a similar phenomenon can be detected in animal species that allow for powerful experimental manipulations, such as rodents. Here we selected four image statistics (from single- to four-point correlations) and trained four groups of rats to discriminate between white noise patterns and binary textures containing variable intensity levels of one of such statistics. We interpreted the resulting psychometric data with an ideal observer model, finding a sharp decrease in sensitivity from two- to four-point correlations and a further decrease from four- to three-point. This ranking fully reproduces the trend we previously observed in humans, thus extending a direct demonstration of efficient coding to a species where neuronal and developmental processes can be interrogated and causally manipulated.

A Change of Scenery: Does Exposure to Images of Nature Affect Delay Discounting and Food Desirability?

Previous research suggests that exposure to nature may reduce delay discounting (the tendency to discount larger future gains in favor of smaller immediate rewards) and thereby facilitate healthier dietary intake. This pre-registered study examined the impact of online exposure to images of natural scenes on delay discounting and food preferences. It was predicted that exposure to images of natural scenes (vs. images of urban scenes) would be associated with: (i) lower delay discounting; (ii) higher desirability for fruits and vegetables (and lower desirability for more energy-dense foods); and (iii) delay discounting would mediate the effect of nature-image exposure on food desirability. Adult participants (N = 109) were recruited to an online between-subjects experiment in which they viewed a timed sequence of six images either showing natural landscape scenes or urban scenes. They then completed measures of mood, delay discounting (using a five-trial hypothetical monetary discounting task) and rated their momentary desire to eat four fruits and vegetables (F&V), and four energy-dense foods. There was no statistically significant effect of experimental condition (natural vs. urban image exposure) on delay discounting or food desirability. Bayes factors supported the null hypothesis for discounting (BF01 = 4.89), and energy-dense food desirability (BF01 = 7.21), but provided no strong evidence for either hypothesis for F&V desirability (BF01 = 0.78). These findings indicate that brief online exposure to images of nature does not affect momentary impulsivity or energy-dense food preference, whereas for preference for less-energy dense foods, the evidence was inconclusive.