scholarly journals A data assimilation framework that uses the Kullback-Leibler divergence

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0256584
Author(s):  
Sam Pimentel ◽  
Youssef Qranfal
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Large Scale ◽  
Matrix Multiplication ◽  
Optimal Interpolation ◽  
Sequential Data ◽  
Physical Constraints ◽  
Large Matrix ◽  
Leibler Divergence ◽  
Similar Accuracy

The process of integrating observations into a numerical model of an evolving dynamical system, known as data assimilation, has become an essential tool in computational science. These methods, however, are computationally expensive as they typically involve large matrix multiplication and inversion. Furthermore, it is challenging to incorporate a constraint into the procedure, such as requiring a positive state vector. Here we introduce an entirely new approach to data assimilation, one that satisfies an information measure and uses the unnormalized Kullback-Leibler divergence, rather than the standard choice of Euclidean distance. Two sequential data assimilation algorithms are presented within this framework and are demonstrated numerically. These new methods are solved iteratively and do not require an adjoint. We find them to be computationally more efficient than Optimal Interpolation (3D-Var solution) and the Kalman filter whilst maintaining similar accuracy. Furthermore, these Kullback-Leibler data assimilation (KL-DA) methods naturally embed constraints, unlike Kalman filter approaches. They are ideally suited to systems that require positive valued solutions as the KL-DA guarantees this without need of transformations, projections, or any additional steps. This Kullback-Leibler framework presents an interesting new direction of development in data assimilation theory. The new techniques introduced here could be developed further and may hold potential for applications in the many disciplines that utilize data assimilation, especially where there is a need to evolve variables of large-scale systems that must obey physical constraints.

TREATMENT OF THE ERROR DUE TO UNRESOLVED SCALES IN SEQUENTIAL DATA ASSIMILATION

2011 ◽  
Vol 21 (12) ◽  
pp. 3619-3626 ◽  
Author(s):  
ALBERTO CARRASSI ◽  
STÉPHANE VANNITSEM
Keyword(s):  
Dynamical System ◽  
Kalman Filter ◽  
Data Assimilation ◽  
Large Scale ◽  
Model Error ◽  
Environmental Modeling ◽  
Sequential Data ◽  
Chaotic Dynamical System ◽  
Error Covariance ◽  
Sequential Data Assimilation

In this paper, a method to account for model error due to unresolved scales in sequential data assimilation, is proposed. An equation for the model error covariance required in the extended Kalman filter update is derived along with an approximation suitable for application with large scale dynamics typical in environmental modeling. This approach is tested in the context of a low order chaotic dynamical system. The results show that the filter skill is significantly improved by implementing the proposed scheme for the treatment of the unresolved scales.

scholarly journals Scale-Dependent Representation of the Information Content of Observations in the Global Ensemble Kalman Filter Data Assimilation

2016 ◽  
Vol 144 (8) ◽  
pp. 2927-2945
Author(s):  
Nedjeljka Žagar ◽  
Jeffrey Anderson ◽  
Nancy Collins ◽  
Timothy Hoar ◽  
Kevin Raeder ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Information Content ◽  
Ensemble Kalman Filter ◽  
Large Scale ◽  
Variance Reduction ◽  
Weather Prediction ◽  
Current Data ◽  
Model Framework ◽  
The Tropics

Abstract Global data assimilation systems for numerical weather prediction (NWP) are characterized by significant uncertainties in tropical analysis fields. Furthermore, the largest spread of global ensemble forecasts in the short range on all scales is in the tropics. The presented results suggest that these properties hold even in the perfect-model framework and the ensemble Kalman filter data assimilation with a globally homogeneous network of wind and temperature profiles. The reasons for this are discussed by using the modal analysis, which provides information about the scale dependency of analysis and forecast uncertainties and information about the efficiency of data assimilation to reduce the prior uncertainties in the balanced and inertio-gravity dynamics. The scale-dependent representation of variance reduction of the prior ensemble by the data assimilation shows that the peak efficiency of data assimilation is on the synoptic scales in the midlatitudes that are associated with quasigeostrophic dynamics. In contrast, the variance associated with the inertia–gravity modes is less successfully reduced on all scales. A smaller information content of observations on planetary scales with respect to the synoptic scales is discussed in relation to the large-scale tropical uncertainties that current data assimilation methodologies do not address successfully. In addition, it is shown that a smaller reduction of the large-scale uncertainties in the prior state for NWP in the tropics than in the midlatitudes is influenced by the applied radius for the covariance localization.

scholarly journals Data-Driven Interpolation of Sea Level Anomalies Using Analog Data Assimilation

2019 ◽  
Vol 11 (7) ◽  
pp. 858 ◽  
Author(s):  
Redouane Lguensat ◽  
Phi Huynh Viet ◽  
Miao Sun ◽  
Ge Chen ◽  
Tian Fenglin ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Sea Level ◽  
Large Scale ◽  
Empirical Orthogonal Functions ◽  
Data Driven ◽  
Optimal Interpolation ◽  
Observation System ◽  
Orthogonal Functions ◽  
Sea Level Anomalies ◽  
Recent Developments

From the recent developments of data-driven methods as a means to better exploit large-scale observation, simulation and reanalysis datasets for solving inverse problems, this study addresses the improvement of the reconstruction of higher-resolution Sea Level Anomaly (SLA) fields using analog strategies. This reconstruction is stated as an analog data assimilation issue, where the analog models rely on patch-based and Empirical Orthogonal Functions (EOF)-based representations to circumvent the curse of dimensionality. We implement an Observation System Simulation Experiment (OSSE) in the South China Sea. The reported results show the relevance of the proposed framework with a significant gain in terms of Root Mean Square Error (RMSE) for scales below 100 km. We further discuss the usefulness of the proposed analog model as a means to exploit high-resolution model simulations for the processing and analysis of current and future satellite-derived altimetric data with regard to conventional interpolation schemes, especially optimal interpolation.

scholarly journals Generation of Vertical Fine Structure by the Internal Waves with the Regard for Turbulent Viscosity and Diffusion

2021 ◽  
Vol 28 (1) ◽  
Author(s):  
S. G. Demyshev ◽  
N. A. Evstigneeva ◽  
D. V. Alekseev ◽  
O. A. Dymova ◽  
N. A. Miklashevskaya ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Potential Energy ◽  
Black Sea ◽  
Anticyclonic Eddy ◽  
Shear Instability ◽  
Optimal Interpolation ◽  
Open Boundary ◽  
The Black Sea ◽  
Coastal Circulation

Purpose. The study is aimed at evaluating effectiveness of the procedure of the observational data assimilation using the Kalman filter algorithm as compared to sequential analysis of the hydrophysical fields based on the optimal interpolation method, and at analyzing the mesoscale features of coastal circulation near the western Crimea coast and in the Sevastopol region. Methods and Results. Based on the hydrodynamic model adapted to the Black Sea coastal zone conditions including the open boundary and on the temperature and salinity data from the hydrological survey in 2007, the dynamic and energy characteristics of the Black Sea coastal circulation were calculated with high spatial resolution (horizontal grid is ~ 1.6 × 1.6 km and 30 vertical horizons). The hydrophysical fields were reconstructed using two algorithms of data assimilation: the sequential optimal interpolation and the modified Kalman filter. The kinetic energy changed mainly due to the wind action, vertical friction and the work of pressure forces; the potential energy – due to the potential energy advection and the horizontal turbulent diffusion. The following circulation features were reconstructed: the anticyclonic eddy with the radius about 15 km in the Kalamita Bay in the water upper layer, the anticyclonic eddy with the radius about 15 km between 32.2 and 32.6° E in the whole water layer, the intense current near Sevastopol and along the Crimea western coast directed to the north and northwest, and the submesoscale eddies of different signs of rotation in the upper layer. Conclusions. It is shown that having been taken into account, heterogeneity and non-isotropy of the error estimates of the temperature and salinity fields relative to the correlation function lead to qualitative and quantitative differences in the hydrodynamic fields (amplification of currents, change of the currents’ direction and eddy formations were better pronounced). At the same time, the mean square errors of the thermohaline fields’ estimates decreased. Formation of the anticyclonic eddy with the radius about 15 km in the Kalamita Bay could be related to the current shear instability. Submesoscale eddies with the diameters less than 5 km were formed when the current flowed around the coastline and the bottom topography inhomogeneities.

scholarly journals Ensemble Kalman Filtering with One-Step-Ahead Smoothing

2018 ◽  
Vol 146 (2) ◽  
pp. 561-581 ◽  
Author(s):  
Naila F. Raboudi ◽  
Boujemaa Ait-El-Fquih ◽  
Ibrahim Hoteit
Keyword(s):  
Data Assimilation ◽  
Large Scale ◽  
Sequential Data ◽  
Bayesian Filtering ◽  
Error Statistics ◽  
Background Error ◽  
One Step ◽  
The One ◽  
Lorenz 96 ◽  
Ensemble Kalman Filtering

The ensemble Kalman filter (EnKF) is widely used for sequential data assimilation. It operates as a succession of forecast and analysis steps. In realistic large-scale applications, EnKFs are implemented with small ensembles and poorly known model error statistics. This limits their representativeness of the background error covariances and, thus, their performance. This work explores the efficiency of the one-step-ahead (OSA) smoothing formulation of the Bayesian filtering problem to enhance the data assimilation performance of EnKFs. Filtering with OSA smoothing introduces an updated step with future observations, conditioning the ensemble sampling with more information. This should provide an improved background ensemble in the analysis step, which may help to mitigate the suboptimal character of EnKF-based methods. Here, the authors demonstrate the efficiency of a stochastic EnKF with OSA smoothing for state estimation. They then introduce a deterministic-like EnKF-OSA based on the singular evolutive interpolated ensemble Kalman (SEIK) filter. The authors show that the proposed SEIK-OSA outperforms both SEIK, as it efficiently exploits the data twice, and the stochastic EnKF-OSA, as it avoids observational error undersampling. They present extensive assimilation results from numerical experiments conducted with the Lorenz-96 model to demonstrate SEIK-OSA’s capabilities.

Data assimilation in stand-level forest inventories

2013 ◽  
Vol 43 (12) ◽  
pp. 1104-1113 ◽  
Author(s):  
Sarah Ehlers ◽  
Anton Grafström ◽  
Kenneth Nyström ◽  
Håkan Olsson ◽  
Göran Ståhl
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Extended Kalman Filter ◽  
Large Scale ◽  
Prediction Models ◽  
Quality Data ◽  
Relative Advantage ◽  
Sources Of Information ◽  
Short Intervals ◽  
Forest Inventories

The development of remote sensing methods through research and large-scale application nowadays makes it possible to obtain stand-level estimates of forest variables at short intervals and at low cost. This offers substantial possibilities to forestry practitioners, but it also poses challenges regarding how cost-efficient data acquisition strategies should be developed. For example, should cheap but low-quality data be acquired and discarded whenever new data become available or should investments be made in high-quality data that are continuously updated to last over a longer period of time? We suggest that the solution could be to establish data assimilation (DA) procedures linked to forest inventories to make appropriate use of data from several sources. With DA, old information is updated through growth forecasts and when new information becomes available it is assimilated with the old information; the different sources of information are made use of to the extent motivated by their accuracy. In this study we made a general assessment of the usefulness of DA in connection with stand-level forest inventories and we compared two different methodological approaches, the extended Kalman filter and the Bayesian method. Not surprisingly, the relative advantage of DA was found to be largest for cases where low-precision estimates of growing stock volume were obtained at short intervals and forecasts were made with accurate growth prediction models. The methodological comparison revealed a tendency of the extended Kalman filter to underestimate the variance of the estimates.

scholarly journals Dense CTD survey versus glider fleet sampling: comparison of the performance for regional ocean prediction West of Sardinia

2018 ◽  
Author(s):  
Jaime Hernandez-Lasheras ◽  
Baptiste Mourre
Keyword(s):  
Data Assimilation ◽  
Large Scale ◽  
Forecast Error ◽  
Model Error ◽  
Ocean Model ◽  
Optimal Interpolation ◽  
Trial Duration ◽  
Sea Trial ◽  
Underwater Gliders ◽  
Sea Level Anomalies

Abstract. The REP14-MEDsea trial carried out off the West coast of Sardinia in June 2014 provided a rich set of observations from both ship-based CTDs and a fleet of underwater gliders. We present the results of several simulations assimilating data either from CTDs or from different subsets of glider data, including up to 8 vehicles, in addition to satellite sea level anomalies, surface temperature and Argo profiles. The WMOP regional ocean model is used with a Local Mutimodel Ensemble Optimal Interpolation scheme to recursively ingest both lower-resolution large scale and dense local observations over the whole sea trial duration. Results show the capacity of the system to ingest both type of data, leading to improvements in the representation of all assimilated variables. These improvements persist during the 3-day periods separating two analysis. At the same time, the system presents some limitations in properly representing the smaller scale structures, which are smoothed out by the model error covariances provided by the ensemble. An evaluation of the forecasts using independent measurements from shipborne CTDs and a towed Scanfish deployed at the end of the sea trial shows that the simulations assimilating initial CTD data reduce the error by 30 to 40 % (according to the variable under consideration) with respect to the simulation without data assimilation. In the glider-data-assimilative experiments, the forecast error is reduced as the number of vehicles increases. The simulation assimilating CTDs outperforms the simulations assimilating data from one to four gliders. A fleet of eight gliders provides a similar performance as the 10-km spaced CTD initilization survey in these experiments, with an overall 40 % model error reduction capacity with respect to the simulation without data assimilation.

scholarly journals Coupling the K-nearest neighbors and locally weighted linear regression with ensemble Kalman filter for data-driven data assimilation

2021 ◽  
Vol 13 (1) ◽  
pp. 1395-1413
Author(s):  
Manhong Fan ◽  
Yulong Bai ◽  
Lili Wang ◽  
Lihong Tang ◽  
Lin Ding
Keyword(s):  
Kalman Filter ◽  
Linear Regression ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Nearest Neighbor ◽  
Nearest Neighbors ◽  
Forecast Model ◽  
Data Driven ◽  
Sequential Data ◽  
Weighted Linear Regression

Abstract Machine learning-based data-driven methods are increasingly being used to extract structures and essences from the ever-increasing pool of geoscience-related big data, which are often used in relation to the atmosphere, oceans, and land surfaces. This study focuses on applying a data-driven forecast model to the classical ensemble Kalman filter process to reconstruct, analyze, and elucidate the model. In this study, a nonparametric sampler from a catalog of historical datasets, namely, a nearest neighbor or analog sampler, is given by numerical simulations. Based on this catalog (sampler), the dynamics physics model is reconstructed using the K-nearest neighbors algorithm. The optimal values of the surrogate model are found, and the forecast step is performed using locally weighted linear regression. Several numerical experiments carried out using the Lorenz-63 and Lorenz-96 models demonstrate that the proposed approach performs as good as the ensemble Kalman filter for larger catalog sizes. This approach is restricted to the ensemble Kalman filter form. However, the basic strategy is not restricted to any particular version of the Kalman filter. It is found that this combined approach can outperform the generally used sequential data assimilation approach when the size of the catalog is substantially large.

Sign in / Sign up

Export Citation Format

Share Document