Simultaneous Past and Current Social Interaction-aware Trajectory Prediction for Multiple Intelligent Agents in Dynamic Scenes

Trajectory prediction of multiple agents in a crowded scene is an essential component in many applications, including intelligent monitoring, autonomous robotics, and self-driving cars. Accurate agent trajectory prediction remains a significant challenge because of the complex dynamic interactions among the agents and between them and the surrounding scene. To address the challenge, we propose a decoupled attention-based spatial-temporal modeling strategy in the proposed trajectory prediction method. The past and current interactions among agents are dynamically and adaptively summarized by two separate attention-based networks and have proven powerful in improving the prediction accuracy. Moreover, it is optional in the proposed method to make use of the road map and the plan of the ego-agent for scene-compliant and accurate predictions. The road map feature is efficiently extracted by a convolutional neural network, and the features of the ego-agent’s plan is extracted by a gated recurrent network with an attention module based on the temporal characteristic. Experiments on benchmark trajectory prediction datasets demonstrate that the proposed method is effective when the ego-agent plan and the the surrounding scene information are provided and achieves state-of-the-art performance with only the observed trajectories.

Revisiting Batch Normalization for Training Low-Latency Deep Spiking Neural Networks From Scratch

Spiking Neural Networks (SNNs) have recently emerged as an alternative to deep learning owing to sparse, asynchronous and binary event (or spike) driven processing, that can yield huge energy efficiency benefits on neuromorphic hardware. However, SNNs convey temporally-varying spike activation through time that is likely to induce a large variation of forward activation and backward gradients, resulting in unstable training. To address this training issue in SNNs, we revisit Batch Normalization (BN) and propose a temporal Batch Normalization Through Time (BNTT) technique. Different from previous BN techniques with SNNs, we find that varying the BN parameters at every time-step allows the model to learn the time-varying input distribution better. Specifically, our proposed BNTT decouples the parameters in a BNTT layer along the time axis to capture the temporal dynamics of spikes. We demonstrate BNTT on CIFAR-10, CIFAR-100, Tiny-ImageNet, event-driven DVS-CIFAR10 datasets, and Sequential MNIST and show near state-of-the-art performance. We conduct comprehensive analysis on the temporal characteristic of BNTT and showcase interesting benefits toward robustness against random and adversarial noise. Further, by monitoring the learnt parameters of BNTT, we find that we can do temporal early exit. That is, we can reduce the inference latency by ~5 − 20 time-steps from the original training latency. The code has been released at https://github.com/Intelligent-Computing-Lab-Yale/BNTT-Batch-Normalization-Through-Time.

Spatial-Temporal Change Analysis for Multivariate Drought Risk Based on Bayesian Copula: Application to the Balkhash Lake Basin

In this study, a spatial-temporal Bayesian copula (SBC) method is developed through integrating spatial-temporal analysis and Bayesian copula into a general framework. SBC method can help model dependence structures of variable pairs and handle the uncertainty caused by parameter in copulas, and SBC can reveal the spatial and temporal changes of drought events. SBC is applied to the Balkhash Lake Basin (in Central Asia) to analyze spatial-temporal characteristic and drought risk in 1901-2020. Several findings can be summarized: (1) Balkhash Lake Basin suffered 53 drought events in 1901-2020, and five typical severe drought events occurred in 1916-1920, 1943-1945, 1973-1977, 1995-1998 and 2007-2009; (2) the most severe drought event lasted for 40 months (1973.10-1977.1), affecting 335,800 km2 of the study basin; (3) drought usually develops from east to west, and Ili River delta and alluvial plain has the highest frequency of drought (47.2%), following by plateau desert (28.3%) and arid grassland in north Balkhash Lake (24.5%); (4) drought shows significant seasonality in the study basin, which usually begins in spring and summer (64.2%) and ends in summer and autumn (66.0%); and drought risk of middle and lower reaches of Ili River is highest in spring and summer; (5) in Balkhash Lake Basin, multivariate characteristics (duration, severity and affected area) significantly affect drought risk; (6) the range of drought risk is [1.9%, 18.1%], [3.7%, 33.1%], [8.7%, 46.0%], [16.0%, 55.1%] and [27.6%, 59.8%] when guarantee rate is 0.99, 0.98, 0.95, 0.90 and 0.80, respectively.

Prediction of the chemical context for Buchwald-Hartwig coupling reactions

We present machine learning models for predicting the chemical context for Buchwald-Hartwig coupling reactions. Using reaction data from in-house electronic lab notebooks, we train two models: one based on single-label data and one based on multi-label data. Both models show excellent top-3 accuracy around 90%, which suggests strong predictivity. There seems to be an advantage of including multi-label data because the multi-label model shows higher accuracy and better sensitivity for the individual contexts than the single-label model. Although the models are performant, we also show that such models need to be re-trained periodically. There is a strong temporal characteristic to the usage of different contexts. Therefore, a model trained on historical data will decrease in usefulness with time as newer and better contexts emerge and replace older ones. We hypothesize that these significant transitions in the context-use will likely affect any model predicting chemical contexts trained on historical data. Consequently, training such models warrants careful planning of what data is used for training and how often the model needs to be re-trained.

Rice Mapping Using a BiLSTM-Attention Model from Multitemporal Sentinel-1 Data

Timely and accurate rice distribution information is needed to ensure the sustainable development of food production and food security. With its unique advantages, synthetic aperture radar (SAR) can monitor the rice distribution in tropical and subtropical areas under any type of weather condition. This study proposes an accurate rice extraction and mapping framework that can solve the issues of low sample production efficiency and fragmented rice plots when prior information on rice distribution is insufficient. The experiment was carried out using multitemporal Sentinel-1A Data in Zhanjiang, China. First, the temporal characteristic map was used for the visualization of rice distribution to improve the efficiency of rice sample production. Second, rice classification was carried out based on the BiLSTM-Attention model, which focuses on learning the key information of rice and non-rice in the backscattering coefficient curve and gives different types of attention to rice and non-rice features. Finally, the rice classification results were optimized based on the high-precision global land cover classification map. The experimental results showed that the classification accuracy of the proposed framework on the test dataset was 0.9351, the kappa coefficient was 0.8703, and the extracted plots maintained good integrity. Compared with the statistical data, the consistency reached 94.6%. Therefore, the framework proposed in this study can be used to extract rice distribution information accurately and efficiently.

Spatial and Temporal Characteristic Analysis of Imbalance Usage in the Hangzhou Public Bicycle System

Calculating the availability of bicycles and racks is a traditional method for detecting imbalance usage in a public bicycle system (PBS). However, for bike-sharing systems in Asian countries, which have compact layouts and larger system scales, an alternative docking station may be found within walking distance. In this paper, we proposed a synthetic and spatial-explicit approach to discover the imbalance usage by using the Hangzhou public bicycle system as an example. A spatial filter was used to remove the false-alarm docking stations and to obtain true imbalance areas of interest (AOI), where the system operation department installs more stations or increases the capacity of existing stations. In addition, sub-nearest neighbor analysis was adopted to determine the average distance between stations, resulting in an average station spacing of 190 m rather than 15.5 m, which can reflect the nonbiased service level of Hangzhou’s public bicycle systems. Our study shows that neighboring stations are taken into account when analyzing PBSs that use a staggered or face-to-face layout, and our method can reduce the number of problematic stations that need to be reallocated by about 92.81%.

Multiscale and Multitemporal Road Detection from High Resolution SAR Images Using Attention Mechanism

Road detection from images has emerged as an important way to obtain road information, thereby gaining much attention in recent years. However, most existing methods only focus on extracting road information from single temporal intensity images, which may cause a decrease in image resolution due to the use of spatial filter methods to avoid coherent speckle noises. Some newly developed methods take into account the multi-temporal information in the preprocessing stage to filter the coherent speckle noise in the SAR imagery. They ignore the temporal characteristic of road objects such as the temporal consistency for the road objects in the multitemporal SAR images that cover the same area and are taken at adjacent times, causing the limitation in detection performance. In this paper, we propose a multiscale and multitemporal network (MSMTHRNet) for road detection from SAR imagery, which contains the temporal consistency enhancement module (TCEM) and multiscale fusion module (MSFM) that are based on attention mechanism. In particular, we propose the TCEM to make full use of multitemporal information, which contains temporal attention submodule that applies attention mechanism to capture temporal contextual information. We enforce temporal consistency constraint by the TCEM to obtain the enhanced feature representations of SAR imagery that help to distinguish the real roads. Since the width of roads are various, incorporating multiscale features is a promising way to improve the results of road detection. We propose the MSFM that applies learned weights to combine predictions of different scale features. Since there is no public dataset, we build a multitemporal road detection dataset to evaluate our methods. State-of-the-art semantic segmentation network HRNetV2 is used as a baseline method to compare with MSHRNet that only has MSFM and the MSMTHRNet. The MSHRNet(TAF) whose input is the SAR image after the temporal filter is adopted to compare with our proposed MSMTHRNet. On our test dataset, MSHRNet and MSMTHRNet improve over the HRNetV2 by 2.1% and 14.19%, respectively, in the IoU metric and by 3.25% and 17.08%, respectively, in the APLS metric. MSMTHRNet improves over the MSMTHRNet(TAF) by 8.23% and 8.81% in the IoU metric and APLS metric, respectively.

Why switchbacks may be related to solar granulation

<p>Parker Solar Probe data below 0.3 AU have revealed a near-Sun magnetic field dominated by Alfvénic structures that display back and forth reversals of the radial magnetic field. They are called magnetic switchbacks, they display no electron strahl variation consistent with magnetic field foldings within the same magnetic sector, and are associated with velocity spikes during an otherwise calmer background. They are thought to originate either at the photosphere through magnetic reconnection processes, or higher up in the corona and solar wind through turbulent processes.</p><p>In this work, we analyze the spatial and temporal characteristic scales of these magnetic switchbacks. We define switchbacks as a deviation from the parker spiral direction and detect them automatically through perihelia encounters 1 to 6. We analyze the solid angle between the magnetic field and the parker spiral both over time and space. We perform a fast Fourier transformation to the obtained angle and find a periodical spatial variation with scales consistent with solar granulation. This suggests that switchbacks form near the photosphere and may be caused, or at least modulated, by solar convection.</p>

Differences in Temporal Relapse Characteristics Between Affective and Non-affective Psychotic Disorders: Longitudinal Analysis

Objective: Multiple relapses over time are common in both affective and non-affective psychotic disorders. Characterizing the temporal nature of these relapses may be crucial to understanding the underlying neurobiology of relapse.Materials and Methods: Anonymized records of patients with affective and non-affective psychotic disorders were collected from SA Mental Health Data Universe and retrospectively analyzed. To characterize the temporal characteristic of their relapses, a relapse trend score was computed using a symbolic series-based approach. A higher score suggests that relapse follows a trend and a lower score suggests relapses are random. Regression models were built to investigate if this score was significantly different between affective and non-affective psychotic disorders.Results: Logistic regression models showed a significant group difference in relapse trend score between the patient groups. For example, in patients who were hospitalized six or more times, relapse score in affective disorders were 2.6 times higher than non-affective psychotic disorders [OR 2.6, 95% CI (1.8–3.7), p < 0.001].Discussion: The results imply that the odds of a patient with affective disorder exhibiting a predictable trend in time to relapse were much higher than a patient with recurrent non-affective psychotic disorder. In other words, within recurrent non-affective psychosis group, time to relapse is random.Conclusion: This study is an initial attempt to develop a longitudinal trajectory-based approach to investigate relapse trend differences in mental health patients. Further investigations using this approach may reflect differences in underlying biological processes between illnesses.

Land Subsidence and Its Relations with Sinkhole Activity in Karapınar Region, Turkey: A Multi-Sensor InSAR Time Series Study

The Karapinar basin, located in the Central Anatolian part of Turkey, is subjected to land subsidence and sinkhole activity due to extensive groundwater withdrawal that began in the early 2000s. In this study, we use Interferometric Synthetic Aperture Radar (InSAR), Global Navigation Satellite System (GNSS), and groundwater level data to monitor and better understand the relations between groundwater extraction, land subsidence, and sinkhole formation in the Karapinar basin. The main observations used in the study are InSAR-derived subsidence velocity maps calculated from both Sentinel-1 (2014–2018) and COSMO-SkyMed (2016–2017) SAR data. Our analysis reveals broad areas of subsidence with rates exceeding 70 mm/yr. The InSAR-derived subsidence was compared with GNSS data acquired by a continuously operating GNSS station located in the study area, which show a similar rate of subsidence. The temporal characteristic of both InSAR and GNSS time series indicate a long-term subsidence signal superimposed by seasonal variability, which follows the overall groundwater level changes, with over 80% cross-correlation consistency. Our results also indicate that sinkhole activity is limited to slow subsidence areas, reflecting strong cohesion of near-surface rock layers that resist subsidence but yield to collapse in response to aquifer system deformation induced by groundwater extraction.