Foveal vision predictively sensitizes to defining features of eye movement targets

Despite the fovea's singular importance for active human vision, the impact of large eye movements on foveal processing remains elusive. Building on findings from passive fixation tasks, we hypothesized that during the preparation of rapid eye movements (saccades), foveal processing anticipates soon-to-be fixated visual features. Using a dynamic large-field noise paradigm, we indeed demonstrate that sensitivity for defining features of a saccade target is enhanced in the pre-saccadic center of gaze. Enhancement manifested in higher Hit Rates for foveal probes with target-congruent orientation, and a sensitization to incidental, target-like orientation information in foveally presented noise. Enhancement was spatially confined to the center of gaze and its immediate vicinity. We suggest a crucial contribution of foveal processing to trans-saccadic visual continuity which has previously been overlooked: Foveal processing of saccade targets commences before the movement is executed and thereby enables a seamless transition once the center of gaze reaches the target.

Integration of allocentric and egocentric visual information in a convolutional / multilayer perceptron network model of goal-directed gaze shifts

Allocentric (landmark-centered) and egocentric (eye-centered) visual codes are fundamental for spatial cognition, navigation, and goal-directed movement. Neuroimaging and neurophysiology suggest these codes are segregated initially, but then reintegrated in frontal cortex for movement control. We created and validated a theoretical framework for this process using physiologically constrained inputs and outputs. To implement a general framework, we integrated a Convolutional Neural Network (CNN) of the visual system with a Multilayer Perceptron (MLP) model of the sensorimotor transformation. The network was trained on a task where a landmark shifted relative to the saccade target. These visual parameters were input to the CNN, the CNN output and initial gaze position to the MLP, and a decoder transformed MLP output into saccade vectors. Decoded saccade output replicated idealized training sets with various allocentric weightings, and actual monkey data where the landmark shift had a partial influence (R2 = 0.8). Furthermore, MLP output units accurately simulated prefrontal response field shifts recorded from monkeys during the same paradigm. In summary, our model replicated both the general properties of the visuomotor transformations for gaze and specific experimental results obtained during allocentric-egocentric integration, suggesting it can provide a general framework for understanding these and other complex visuomotor behaviors.

Canonical retinotopic shifts under an inverse force field explain predictive remapping

ABSTRACTSaccadic eye-movements allow us to bring visual objects of interest to high-acuity central vision. Although saccades cause large displacements of retinal images, our percept of the visual world remains stable. Predictive remapping — the ability of cells in retinotopic brain areas to transiently exhibit spatio-temporal retinotopic shifts beyond the spatial extent of their classical receptive fields — has been proposed as a primary mechanism that mediates this seamless visual percept. Despite the well documented effects of predictive remapping, no study to date has been able to provide a mechanistic account of the neural computations and architecture that actively mediate this ubiquitous phenomenon. Borne out by the spatio-temporal dynamics of peri-saccadic sensitivity to probe stimuli in human subjects, we propose a novel neurobiologically inspired phenomenological model in which the underlying peri-saccadic attentional and oculomotor signals manifest as three temporally overlapping forces that act on retinotopic brain areas. These three forces – a compressive one toward the center of gaze, a convergent one toward the saccade target and a translational one parallel to the saccade trajectory – act in an inverse force field and specify the spatio-temporal window of predictive remapping of population receptive fields.

Coding of interceptive saccades in parietal cortex of macaque monkeys

The oculomotor system can initiate remarkably accurate saccades towards moving targets (interceptive saccades) the processing of which is still under debate. The generation of these saccades requires the oculomotor centers to have information about the motion parameters of the target that then must be extrapolated to bridge the inherent processing delays. We investigated to what degree the information about motion of a saccade target is available in the lateral intra-parietal area (area LIP) of macaque monkeys for generation of accurate interceptive saccades. When a multi-layer neural network was trained based on neural discharges from area LIP around the time of saccades towards stationary targets, it was also able to predict the end points of saccades directed towards moving targets. This prediction, however, lagged behind the actual post-saccadic position of the moving target by ~ 80 ms when the whole neuronal sample of 105 neurons was used. We further found that single neurons differentially code for the motion of the target. Selecting neurons with the strongest representation of target motion reduced this lag to ~ 30 ms which represents the position of the moving target approximately at the onset of the interceptive saccade. We conclude that—similarly to recent findings from the Superior Colliculus (Goffart et al. J Neurophysiol 118(5):2890–2901)—there is a continuum of contributions of individual LIP neurons to the accuracy of interceptive saccades. A contribution of other gaze control centers (like the cerebellum or the frontal eye field) that further increase the saccadic accuracy is, however, likely.

Coding of interceptive saccades in parietal cortex of macaque monkeys

The oculomotor system can initiate remarkably accurate saccades towards moving targets (interceptive saccades) the processing of which is still under debate. The generation of these saccades requires the oculomotor centers to have information about the motion parameters of the target that then must be extrapolated to bridge the inherent processing delays. We investigated to what degree the information about motion of a saccade target is available in the lateral intra-parietal area (area LIP) of macaque monkeys for generation of accurate interceptive saccades. When a multi-layer neural network was trained based on neural discharges from area LIP around the time of saccades towards stationary targets it was also able to predict the end points of saccades directed towards moving targets. This prediction, however, lagged behind the actual post-saccadic position of the moving target by ~80 ms when the whole neuronal sample of 105 neurons was used. We further found that single neurons differentially code for the motion of the target. Selecting neurons with the strongest representation of target motion reduced this lag to ~30 ms which represents the position of the moving target approximately at the onset of the interceptive saccade. We conclude that - similarly to recent findings from the Superior Colliculus (Goffart et al., 2017) - there is a continuum of contributions of individual LIP neurons to the accuracy of interceptive saccades. A contribution of other gaze control centers (like the cerebellum or the frontal eye field) that further increase the saccadic accuracy is, however, likely.

Salience by Competitive and Recurrent Interactions: Bridging Neural Spiking and Computation

Decisions about where to move the eyes depend on neurons in Frontal Eye Field (FEF). Movement neurons in FEF accumulate salience evidence derived from FEF visual neurons to select the location of a saccade target among distractors. How visual neurons achieve this salience representation is unknown. We present a neuro-computational model of target selection called Salience by Competitive and Recurrent Interactions (SCRI), based on the Competitive Interaction model of attentional selection and decision making (Smith & Sewell, 2013). SCRI selects targets by synthesizing localization and identification information to yield a dynamically evolving representation of salience across the visual field. SCRI accounts for neural spiking of individual FEF visual neurons, explaining idiosyncratic differences in neural dynamics with specific parameters. Many visual neurons resolve the competition between search items through feedforward inhibition between signals representing different search items, some also require lateral inhibition, and many act as recurrent gates to modulate the incoming flow of information about stimulus identity. SCRI was tested further by using simulated spiking representations of visual salience as input to the Gated Accumulator Model of FEF movement neurons (Purcell et al., 2010; Purcell, Schall, Logan, & Palmeri, 2012). Predicted saccade response times fit those observed for search arrays of different set size and different target-distractor similarity, and accumulator trajectories replicated movement neuron discharge rates. These findings offer new insights into visual decision making through converging neuro-computational constraints and provide a novel computational account of the diversity of FEF visual neurons.

Vision as oculomotor reward: cognitive contributions to the dynamic control of saccadic eye movements

Humans and other primates are equipped with a foveated visual system. As a consequence, we reorient our fovea to objects and targets in the visual field that are conspicuous or that we consider relevant or worth looking at. These reorientations are achieved by means of saccadic eye movements. Where we saccade to depends on various low-level factors such as a targets’ luminance but also crucially on high-level factors like the expected reward or a targets’ relevance for perception and subsequent behavior. Here, we review recent findings how the control of saccadic eye movements is influenced by higher-level cognitive processes. We first describe the pathways by which cognitive contributions can influence the neural oculomotor circuit. Second, we summarize what saccade parameters reveal about cognitive mechanisms, particularly saccade latencies, saccade kinematics and changes in saccade gain. Finally, we review findings on what renders a saccade target valuable, as reflected in oculomotor behavior. We emphasize that foveal vision of the target after the saccade can constitute an internal reward for the visual system and that this is reflected in oculomotor dynamics that serve to quickly and accurately provide detailed foveal vision of relevant targets in the visual field.