AIgean : An Open Framework for Deploying Machine Learning on Heterogeneous Clusters

AIgean , pronounced like the sea, is an open framework to build and deploy machine learning (ML) algorithms on a heterogeneous cluster of devices (CPUs and FPGAs). We leverage two open source projects: Galapagos , for multi-FPGA deployment, and hls4ml , for generating ML kernels synthesizable using Vivado HLS. AIgean provides a full end-to-end multi-FPGA/CPU implementation of a neural network. The user supplies a high-level neural network description, and our tool flow is responsible for the synthesizing of the individual layers, partitioning layers across different nodes, as well as the bridging and routing required for these layers to communicate. If the user is an expert in a particular domain and would like to tinker with the implementation details of the neural network, we define a flexible implementation stack for ML that includes the layers of Algorithms, Cluster Deployment & Communication, and Hardware. This allows the user to modify specific layers of abstraction without having to worry about components outside of their area of expertise, highlighting the modularity of AIgean . We demonstrate the effectiveness of AIgean with two use cases: an autoencoder, and ResNet-50 running across 10 and 12 FPGAs. AIgean leverages the FPGA’s strength in low-latency computing, as our implementations target batch-1 implementations.

Textile Production Line Monitoring System Using Wavelet-Regression Neural Network

This paper presents design and experiments for a production line monitoring system. The system is designed based on an existing production line which mapping to the smart grid standards. The Discrete wavelet transform (DWT) and regression neural network (RNN) are applied to the operation modes data analysis. DWT used to preprocess the signals to remove noise from the raw signals. The output of DWT energy distribution has given as an input to the GRNN model. The neural network GRNN architecture involves multi-layer structures. Mean Absolute Percentage Error (MAPE) loss has used in the GRNN model, which is used to forecast the time-series data. Current research results can only apply to the single production line but in future, it will used for multiple production lines.

A hybrid approach for categorizing images based on complex networks and neural networks

There are several methods for categorizing images, the most of which are statistical, geometric, model-based and structural methods. In this paper, a new method for describing images based on complex network models is presented. Each image contains a number of key points that can be identified through standard edge detection algorithms. To understand each image better, we can use these points to create a graph of the image. In order to facilitate the use of graphs, generated graphs are created in the form of a complex network of small-worlds. Complex grid features such as topological and dynamic features can be used to display image-related features. After generating this information, it normalizes them and uses them as suitable features for categorizing images. For this purpose, the generated information is given to the neural network. Based on these features and the use of neural networks, comparisons between new images are performed. The results of the article show that this method has a good performance in identifying similarities and finally categorizing them.

Textile Production Line Monitoring System Using Wavelet-Regression Neural Network

This paper presents design and experiments for a production line monitoring system. The system is designed based on an existing production line which mapping to the smart grid standards. The Discrete wavelet transform (DWT) and regression neural network (RNN) are applied to the operation modes data analysis. DWT used to preprocess the signals to remove noise from the raw signals. The output of DWT energy distribution has given as an input to the GRNN model. The neural network GRNN architecture involves multi-layer structures. Mean Absolute Percentage Error (MAPE) loss has used in the GRNN model, which is used to forecast the time-series data. Current research results can only apply to the single production line but in future, it will used for multiple production lines.

Effects of Transcranial Magneto-acoustic Electrical Stimulation on Discharge propagation in feed-forward neural network

The mammalian brain has an extremely complex, diversified and highly modular structure, and information dissemination in the modular brain network affects various brain diseases. Although a variety of neuromodulation techniques have been used to study the discharge characteristics of neural networks, the effects of transcranial magneto-acoustic electrical stimulation(TMAES) have rarely been mentioned. Based on the excitatory and inhibitory Izhikevich neuron model, we constructs a feed-forward neural network connected by electrical synapses and chemical synapses, and analyzes the firing frequency of the neural network under TMAES and magnetic stimulation and the differences in each layer types of firing patterns of neurons. The results showed that the discharge patterns of neurons in each layer were different, the discharge frequency of inhibitory neurons was higher than that of excited neurons, and the stimulation signal could be transmitted to the whole network layer.The maximum discharge frequency of neural network connected by electrical coupling can reach 0.94kHz, and the discharge frequency of neural network connected by chemical coupling is less than 0.5 kHz.With the increase of coupling degree, the discharge frequency of neurons in each network layer under TMAES is much greater than that under magnetic stimulation.When the induced current is less than 26.5μA/cm 2 , magnetic stimulation can promote the inhibitory neurons, and TMAES has a variety of regulatory effects on the inhibitory neurons in the neural network. The results show that TMAES has better regulation effect than magnetic stimulation, and the regulation effect is affected by neural network structure and stimulation parameters.

Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networks

Several image processing tasks, such as image classification and object detection, have been significantly improved using Convolutional Neural Networks (CNN). Like ResNet and EfficientNet, many architectures have achieved outstanding results in at least one dataset by the time of their creation. A critical factor in training concerns the network’s regularization, which prevents the structure from overfitting. This work analyzes several regularization methods developed in the last few years, showing significant improvements for different CNN models. The works are classified into three main areas: the first one is called “data augmentation”, where all the techniques focus on performing changes in the input data. The second, named “internal changes”, which aims to describe procedures to modify the feature maps generated by the neural network or the kernels. The last one, called “label”, concerns transforming the labels of a given input. This work presents two main differences comparing to other available surveys about regularization: (i) the first concerns the papers gathered in the manuscript, which are not older than five years, and (ii) the second distinction is about reproducibility, i.e., all works refered here have their code available in public repositories or they have been directly implemented in some framework, such as TensorFlow or Torch.

Using deep learning to predict ideology from facial photographs: Expressions, beauty, and extra-facial information

Deep learning techniques can use common public data such as facial photographs to predict sensitive personal information, but little is known about what information contributes to the predictive success of these techniques. This lack of knowledge limits both the public’s ability to protect against revealing unintended information as well as the scientific utility of deep learning results. We combine convolutional neural networks, heat maps, facial expression coding, and classification of identifiable features such as masculinity and attractiveness in our study of political ideology in 3323 Danes. Predictive accuracy from the neural network was 61% in each gender. Model-predicted ideology correlated with aspects of both facial expressions (happiness vs neutrality) and morphology (specifically, attractiveness in females). Heat maps highlighted the informativeness of areas both on and off the face, pointing to methodological refinements and the need for future research to better understand the significance of certain facial areas.

Robust sideslip angle observer of commercial vehicles based on cornering stiffness estimation using neural network

Accurate estimation of sideslip angle is essential for vehicle stability control. For commercial vehicles, the estimation of sideslip angle is challenging due to severe load transfer and tire nonlinearity. This paper presents a robust sideslip angle observer of commercial vehicles based on identification of tire cornering stiffness. Since tire cornering stiffness of commercial vehicles is greatly affected by tire force and road adhesion coefficient, it cannot be treated as a constant. To estimate the cornering stiffness in real time, the neural network model constructed by Levenberg-Marquardt backpropagation (LMBP) algorithm is employed. LMBP is a fast convergent supervised learning algorithm, which combines the steepest descent method and gauss-newton method, and is widely used in system parameter estimation. LMBP does not rely on the mathematical model of the actual system when building the neural network. Therefore, when the mathematical model is difficult to establish, LMBP can play a very good role. Considering the complexity of tire modeling, this study adopted LMBP algorithm to estimate tire cornering stiffness, which have simplified the tire model and improved the estimation accuracy. Combined with neural network, A time-varying Kalman filter (TVKF) is designed to observe the sideslip angle of commercial vehicles. To validate the feasibility of the proposed estimation algorithm, multiple driving maneuvers under different road surface friction have been carried out. The test results show that the proposed method has better accuracy than the existing algorithm, and it’s robust over a wide range of driving conditions.

An Explainable Intelligent Framework for Anomaly Mitigation in Cyber-Physical Inverter-based Systems

An explainable intelligent framework for cyber anomaly mitigation of cyber-physical inverter-based systems is presented.<div><br></div><div>Smart inverter-based microgrids essentially constitute an extensive communication layer that makes them vulnerable to cyber anomalies. The distributed cooperative controllers implemented at the secondary control level of such systems exchange information among physical nodes using the cyber layer to meet the control objectives. The cyber anomalies targeting the communication network may distort the normal operation therefore, an effective cyber anomaly mitigation technique using an artificial neural network (ANN) is proposed in this paper. The intelligent anomaly mitigation control is modeled using adynamic recurrent neural network that employs a nonlinear autoregressive network with exogenous inputs. The effects of false data injection to the distributed cooperative controller at the secondary control level are considered. The training data for designing the neural network are generated by multiple simulations of the designed microgrid under various operating conditions using MATLAB/Simulink. The neural network is trained offline and tested online in the simulated microgrid. The proposed technique is applied as secondary voltage and frequency control of distributed cooperative control-based microgrid to regulate the voltage under various operating conditions. The performance of the proposed control technique is verified by injecting various types of false data injection-based cyber anomalies. The proposed ANN-based secondary controller maintained the normal operation of microgrid in the presence of cyber anomalies as demonstrated by real-time simulations on a real-time digital simulator OPAL-RT.<br></div>

A Novel Normalization Algorithm to Facilitate Pre-Assessment of Covid-19 Disease from Chest X-ray Image and its FPGA implementation

COVID-19 is still a fatal disease, which has threatened all people by affecting the human lungs. Chest X-Ray or computed tomography (CT) imaging is commonly used to make a fast and reliable medical investigation to detect the COVID-19 virus from these medical images is remarkably challenging because it is a full-time job and prone to human errors. In this paper, a new normalization algorithm that consists of Mean-Variance-Softmax-Rescale (MVSR) processes respectively is proposed to provide facilitation pre-assessment and diagnosis Covid-19 disease. In order to show the effect of MVSR normalization technique on image processing, the algorithm is applied to chest X-ray images. Therefore, the normalized X-ray images with MVSR are used to recognize via one of the neural network models as known Convolutional Neural Networks (CNNs). At the implementation stage, the MVSR algorithm is executed on MATLAB environment, then it is implemented on FPGA platform. All the arithmetic operations of the MVSR normalization are coded in VHDL with the help of fixed-point fractional number representation format. The experimental platform consists of Zynq-7000 Development Board and VGA monitor to display the both original X-ray and MVSR normalized image. The CNN model is constructed and executed using Anaconda Navigator interface with python language. Based on the results of this study, infections of Covid-19 disease can be easily diagnosed for MVSR normalized image. The proposed MVSR normalization makes the accuracy of CNN model increase from 83.01%, to 96.16% for binary class of chest X-ray images.