Control of Magnetic Manipulator Using Reinforcement Learning Based on Incrementally Adapted Local Linear Models

Reinforcement learning (RL) agents can learn to control a nonlinear system without using a model of the system. However, having a model brings benefits, mainly in terms of a reduced number of unsuccessful trials before achieving acceptable control performance. Several modelling approaches have been used in the RL domain, such as neural networks, local linear regression, or Gaussian processes. In this article, we focus on techniques that have not been used much so far: symbolic regression (SR), based on genetic programming and local modelling. Using measured data, symbolic regression yields a nonlinear, continuous-time analytic model. We benchmark two state-of-the-art methods, SNGP (single-node genetic programming) and MGGP (multigene genetic programming), against a standard incremental local regression method called RFWR (receptive field weighted regression). We have introduced modifications to the RFWR algorithm to better suit the low-dimensional continuous-time systems we are mostly dealing with. The benchmark is a nonlinear, dynamic magnetic manipulation system. The results show that using the RL framework and a suitable approximation method, it is possible to design a stable controller of such a complex system without the necessity of any haphazard learning. While all of the approximation methods were successful, MGGP achieved the best results at the cost of higher computational complexity. Index Terms–AI-based methods, local linear regression, nonlinear systems, magnetic manipulation, model learning for control, optimal control, reinforcement learning, symbolic regression.

Evaporation estimation from climatic factors

This study assessed the ability of two models, Local Linear Regression (LLR) and Artificial Neural Network (ANN) to estimate monthly potential evaporation from Pantagar, US Nagar (India) which falls under sub-humid and subtropical climatic zone. Observations of relative humidity, solar radiation, temperature, wind speed and evaporation have been used to train and test the developed models. A comparison was made between the estimates provided by the LLR model and ANN model. Results shown that the models were able to well learn the events they were trained to recognize. For ANN model the correlation coefficient for training period is 0.9311 and for testing period is 0.9236 and the value of root mean square error for training period is 1.070 and for testing period it is 0.9863. In case of LLR model the correlation coefficient for training period is 0.9746 and for testing period is 0.9273 and value of root mean square error for training period is 0.6121 and for testing period it is 1.5301.

Assessment of Soft Computing Techniques for the Prediction of Suspended Sediment Loads in Rivers

A key goal of sediment management is the quantification of suspended sediment load (SSL) in rivers. This research focused on a comparison of different means of suspended sediment estimation in rivers. This includes sediment rating curves (SRC) and soft computing techniques, i.e., local linear regression (LLR), artificial neural networks (ANN) and the wavelet-cum-ANN (WANN) method. Then, different techniques were applied to predict daily SSL at the Pirna and Magdeburg Stations of the Elbe River in Germany. By comparing the results of all the best models, it can be concluded that the soft computing techniques (LLR, ANN and WANN) better predicted the SSL than the SRC method. This is due to the fact that the former employed non-linear techniques for the data series reconstruction. The WANN models were the overall best performer. The WANN models in the testing phase showed a mean R2 of 0.92 and a PBIAS of −0.59%. Additionally, they were able to capture the suspended sediment peaks with greater accuracy. They were more successful as they captured the dynamic features of the non-linear and time-variant suspended sediment load, while other methods used simple raw data. Thus, WANN models could be an efficient technique to simulate the SSL time series because they extract key features embedded in the SSL signal.

Improved Output Gap Estimates and Forecasts Using a Local Linear Regression

The output gap, the difference between potential and actual output, has a direct impact on policy decisions, e.g., monetary policy. Estimating this gap and its further analysis remain the subject of controversial debates due to methodological problems. We propose a local polynomial regression combined with a Self-Exciting Threshold AutoRegressive (SETAR) model and its forecasting extension for a systematic output gap estimation. Furthermore, local polynomial regression is proposed for the (multivariate) OECD production function approach and its reliability is demonstrated in forecasting output growth. A comparison of the proposed gap to the Hodrick–Prescott filter as well as to estimations by experts from the FED and OECD shows a higher correlation of our output gap with those from those economic institutions. Furthermore, sometimes gaps with different magnitude and different positions above or below the trend are observed between different methods. This may cause competing policy implications which can be improved with our results.

Improving the Accuracy of Subseasonal Forecasting of China Precipitation With a Machine Learning Approach

Precipitation change, which is closely related to drought and flood disasters in China, affects billions of people every year, and the demand for subseasonal forecasting of precipitation is even more urgent. Subseasonal forecasting, which is more difficult than weather forecasting, however, has remained as a blank area in meteorological service for a long period of time. To improve the accuracy of subseasonal forecasting of China precipitation, this work introduces the machine learning method proposed by Hwang et al. in 2019 to predict the precipitation in China 2–6 weeks in advance. The authors used a non-linear regression model called local linear regression together with multitask feature election (MultiLLR) model and chosen 21 meteorological elements as candidate predictors to integrate diverse meteorological observation data. This method automatically eliminates irrelevant predictors so as to establish the forecast equations using multitask feature selection process. The experiments demonstrate that the pressure and Madden–Julian Oscillation (MJO) are the most important physical factors. The average prediction skill is 0.11 during 2011–2016, and there are seasonal differences in forecasting skills, evidenced by higher forecast skills of winter and spring seasons than summer and autumn seasons. The proposed method can provide effective and indicative guidance for the subseasonal prediction of precipitation in China. By adding another three factors, Arctic Oscillation (AO) index, Western North Pacific Monsoon (WNPM) index and Western North Pacific Subtropical High (WNPSH) index into the MultiLLR model, the authors find that AO can improve the forecast skill of China precipitation to the maximum extent from 0.11 to 0.13, followed by WNPSH. Moreover, the ensemble skill of our model and CFSv2 is 0.16. This work shows that our subseasonal prediction of China precipitation should be benefited from the MultiLLR model.

Scalar-on-function local linear regression and beyond

It is common to want to regress a scalar response on a random function. This paper presents results that advocate local linear regression based on a projection as a nonparametric approach to this problem. Our asymptotic results demonstrate that functional local linear regression outperforms its functional local constant counterpart. Beyond the estimation of the regression operator itself, local linear regression is also a useful tool for predicting the functional derivative of the regression operator, a promising mathematical object on its own. The local linear estimator of the functional derivative is shown to be consistent. For both the estimator of the regression functional and the estimator of its derivative, theoretical properties are detailed. On simulated datasets we illustrate good finite sample properties of the proposed methods. On a real data example of a single-functional index model we indicate how the functional derivative of the regression operator provides an original, fast, and widely applicable estimation method.

Find Outliers of Image Edge Consistency by Weighted Local Linear Regression with Equality Constraints

In this paper, we propose a general method to detect outliers from contaminated estimates of various image estimation applications. The method does not require any prior knowledge about the purpose, theory or hardware of the application but simply relies on the law of edge consistency between sources and estimates. The method is termed as ALRe (anchored linear residual) because it is based on the residual of weighted local linear regression with an equality constraint exerted on the measured pixel. Given a pair of source and contaminated estimate, ALRe offers per-pixel outlier likelihoods, which can be used to compose the data weights of post-refinement algorithms, improving the quality of refined estimate. ALRe has the features of asymmetry, no false positive and linear complexity. Its effectiveness is verified on four applications, four post-refinement algorithms and three datasets. It demonstrates that, with the help of ALRe, refined estimates are better in the aspects of both quality and edge consistency. The results are even comparable to model-based and hardware-based methods. Accuracy comparison on synthetic images shows that ALRe could detect outliers reliably. It is as effective as the mainstream weighted median filter at spike detection and is significantly better at bad region detection.