Accurate quantification of uncertainty in epidemic parameter estimates and predictions using stochastic compartmental models

Stochastic transmission dynamic models are needed to quantify the uncertainty in estimates and predictions during outbreaks of infectious diseases. We previously developed a calibration method for stochastic epidemic compartmental models, called Multiple Shooting for Stochastic Systems (MSS), and demonstrated its competitive performance against a number of existing state-of-the-art calibration methods. The existing MSS method, however, lacks a mechanism against filter degeneracy, a phenomenon that results in parameter posterior distributions that are weighted heavily around a single value. As such, when filter degeneracy occurs, the posterior distributions of parameter estimates will not yield reliable credible or prediction intervals for parameter estimates and predictions. In this work, we extend the MSS method by evaluating and incorporating two resampling techniques to detect and resolve filter degeneracy. Using simulation experiments, we demonstrate that an extended MSS method produces credible and prediction intervals with desired coverage in estimating key epidemic parameters (e.g. mean duration of infectiousness and R0) and short- and long-term predictions (e.g. one and three-week forecasts, timing and number of cases at the epidemic peak, and final epidemic size). Applying the extended MSS approach to a humidity-based stochastic compartmental influenza model, we were able to accurately predict influenza-like illness activity reported by U.S. Centers for Disease Control and Prevention from 10 regions as well as city-level influenza activity using real-time, city-specific Google search query data from 119 U.S. cities between 2003 and 2014.

Download Full-text

A Comparison of the Separate and Concurrent Calibration Methods for the Full-Information Bifactor model

Applied Psychological Measurement ◽

10.1177/0146621618813095 ◽

2018 ◽

Vol 43 (7) ◽

pp. 512-526

Author(s):

Kyung Yong Kim

Keyword(s):

Item Response Theory ◽

Item Response ◽

Calibration Method ◽

Bifactor Model ◽

Item Parameter ◽

Parameter Estimates ◽

Calibration Methods ◽

Item Parameter Estimates ◽

Concurrent Calibration ◽

Separate Calibration

When calibrating items using multidimensional item response theory (MIRT) models, item response theory (IRT) calibration programs typically set the probability density of latent variables to a multivariate standard normal distribution to handle three types of indeterminacies: (a) the location of the origin, (b) the unit of measurement along each coordinate axis, and (c) the orientation of the coordinate axes. However, by doing so, item parameter estimates obtained from two independent calibration runs on nonequivalent groups are on two different coordinate systems. To handle this issue and place all the item parameter estimates on a common coordinate system, a process called linking is necessary. Although various linking methods have been introduced and studied for the full MIRT model, little research has been conducted on linking methods for the bifactor model. Thus, the purpose of this study was to provide detailed descriptions of two separate calibration methods and the concurrent calibration method for the bifactor model and to compare the three linking methods through simulation. In general, the concurrent calibration method provided more accurate linking results than the two separate calibration methods, demonstrating better recovery of the item parameters, item characteristic surfaces, and expected score distribution.

Download Full-text

Calibration of models to data: a comparison of methods

10.1101/2020.12.21.423763 ◽

2020 ◽

Author(s):

Zenabu Suboi ◽

Thomas J. Hladish ◽

Wim Delva ◽

C. Marijn Hazelbag

Keyword(s):

Adaptive Sampling ◽

Goodness Of Fit ◽

Sequential Monte Carlo ◽

Sampling Methods ◽

Distance Measure ◽

Calibration Method ◽

Model Complexity ◽

Posterior Distributions ◽

Calibration Methods ◽

Computing Infrastructure

AbstractComplex models are often fitted to data using simulation-based calibration, a computationally challenging process. Several calibration methods to improve computational efficiency have been developed with no consensus on which methods perform best. We did a simulation study comparing the performance of 5 methods that differed in their Goodness-of-Fit (GOF) metrics and parameter search strategies. Posterior densities for two parameters of a simple Susceptible-Infectious-Recovered epidemic model were obtained for each calibration method under two scenarios. Scenario 1 (S1) allowed 60K model runs and provided two target statistics, whereas scenario 2 (S2) allowed 75K model runs and provided three target statistics. For both scenarios, we obtained reference posteriors against which we compare all other methods by running Rejection ABC for 5M parameter combinations, retaining the 0.1% best. We assessed performance by applying a 2D-grid to all posterior densities and quantifying the percentage overlap with the reference posterior.We considered basic and adaptive sampling calibration methods. Of the basic calibration methods, Bayesian calibration (Bc) Sampling Importance Resampling (S1: 34.8%, S2: 39.8%) outperformed Rejection Approximate Bayesian Computation (ABC) (S1: 2.3%, S2: 1.8%). Among the adaptive sampling methods, Bc Incremental Mixture Importance Sampling (S1: 72.7%, S2: 85.5%) outperformed sequential Monte Carlo ABC (AbcSmc) (S1: 53.9%, S2: 72.9%) and Sequential ABC (S1: 21.6%, S2: 62.7%).Basic methods led to sub-optimal calibration results. Methods using the surrogate Likelihood as a GOF outperformed methods using a distance measure. Adaptive sampling methods were more efficient compared to their basic counterparts and resulted in accurate posterior distributions. BcIMIS was the best performing method. When three rather than two target statistics were available, the difference in performance between the adaptive sampling methods was less pronounced. Although BcIMIS outperforms the other methods, limitations related to the target statistics and available computing infrastructure may warrant the choice of an alternative method.Author summaryAs mathematical models become more realistic, they tend to become more complex. Calibration, the process of tuning a model to better reproduce empirical data, can become dramatically more computationally intensive as model complexity increases. Researchers have responded by developing a range of more efficient, adaptive sampling calibration methods. However, the relative performance of these calibration methods remains unclear. To this end, we quantified the performance of five commonly used calibration methods. We found that adaptive sampling methods were more efficient compared to their basic counterparts and resulted in more accurate posterior distributions. We identified the best performing method, but caution that limitations related to the target statistics and available computing infrastructure may warrant the choice of one of the alternatives. Finally, we provide the code used to apply the calibration methods in our study as a primer to facilitate their application.

Download Full-text

A Multi Camera and Multi Laser Calibration Method for 3D Reconstruction of Revolution Parts

Sensors ◽

10.3390/s21030765 ◽

2021 ◽

Vol 21 (3) ◽

pp. 765

Author(s):

Hugo Álvarez ◽

Marcos Alonso ◽

Jairo R. Sánchez ◽

Alberto Izaguirre

Keyword(s):

3D Reconstruction ◽

Calibration Method ◽

Laser Plane ◽

Calibration Methods ◽

Laser Calibration ◽

Calibration Object ◽

Focus Plane ◽

Chessboard Pattern ◽

Line Patterns

This paper describes a method for calibrating multi camera and multi laser 3D triangulation systems, particularly for those using Scheimpflug adapters. Under this configuration, the focus plane of the camera is located at the laser plane, making it difficult to use traditional calibration methods, such as chessboard pattern-based strategies. Our method uses a conical calibration object whose intersections with the laser planes generate stepped line patterns that can be used to calculate the camera-laser homographies. The calibration object has been designed to calibrate scanners for revolving surfaces, but it can be easily extended to linear setups. The experiments carried out show that the proposed system has a precision of 0.1 mm.

Download Full-text

Stable Calibrations of Six-DOF Serial Robots by Using Identification Models with Equalized Singular Values

Robotica ◽

10.1017/s0263574721000229 ◽

2021 ◽

pp. 1-22

Author(s):

Zhouxiang Jiang ◽

Min Huang

Keyword(s):

Industrial Robot ◽

Singular Values ◽

Calibration Method ◽

Identification Accuracy ◽

Condition Numbers ◽

Measurement Instruments ◽

Unknown Parameters ◽

Calibration Methods ◽

Six Dof ◽

The Stability

SUMMARY In typical calibration methods (kinematic or non-kinematic) for serial industrial robot, though measurement instruments with high resolutions are adopted, measurement configurations are optimized, and redundant parameters are eliminated from identification model, calibration accuracy is still limited under measurement noise. This might be because huge gaps still exist among the singular values of typical identification Jacobians, thereby causing the identification models ill conditioned. This paper addresses such problem by using new identification models established in two steps. First, the typical models are divided into the submodels with truncated singular values. In this way, the unknown parameters corresponding to the abnormal singular values are removed, thereby reducing the condition numbers of the new submodels. However, these models might still be ill conditioned. Therefore, the second step is to further centralize the singular values of each submodel by using a matrix balance method. Afterward, all submodels are well conditioned and obtain much higher observability indices compared with those of typical models. Simulation results indicate that significant improvements in the stability of identification results and the identifiability of unknown parameters are acquired by using the new identification submodels. Experimental results indicate that the proposed calibration method increases the identification accuracy without incurring additional hardware setup costs to the typical calibration method.

Download Full-text

Vicarious Radiometric Calibration of Ocean Color Bands for FY-3D/MERSI-II at Lake Qinghai, China

Sensors ◽

10.3390/s21010139 ◽

2020 ◽

Vol 21 (1) ◽

pp. 139

Author(s):

Shengli Chen ◽

Xiaobing Zheng ◽

Xin Li ◽

Wei Wei ◽

Shenda Du ◽

...

Keyword(s):

Infrared Imaging ◽

Large Deviation ◽

Ocean Color ◽

Calibration Method ◽

Surface Measurement ◽

Radiometric Calibration ◽

Lake Qinghai ◽

Vicarious Calibration ◽

Calibration Methods ◽

Spectral Imager

To calibrate the low signal response of the ocean color (OC) bands and test the stability of the Fengyun-3D (FY-3D)/Medium Resolution Spectral Imager II (MERSI-II), an absolute radiometric calibration field test of FY-3D/MERSI-II at the Lake Qinghai Radiometric Calibration Site (RCS) was carried out in August 2018. The lake surface and atmospheric parameters were mainly measured by advanced observation instruments, and the MODerate spectral resolution atmospheric TRANsmittance algorithm and computer model (MODTRAN4.0) was used to simulate the multiple scattering radiance value at the altitude of the sensor. The results showed that the relative deviations between bands 9 and 12 are within 5.0%, while the relative deviations of bands 8, and 13 are 17.1%, and 12.0%, respectively. The precision of the calibration method was verified by calibrating the Aqua/Moderate-resolution Imaging Spectroradiometer (MODIS) and National Polar-orbiting Partnership (NPP)/Visible Infrared Imaging Radiometer (VIIRS), and the deviation of the calibration results was evaluated with the results of the Dunhuang RCS calibration and lunar calibration. The results showed that the relative deviations of NPP/VIIRS were within 7.0%, and the relative deviations of Aqua/MODIS were within 4.1% from 400 nm to 600 nm. The comparisons of three on-orbit calibration methods indicated that band 8 exhibited a large attenuation after launch and the calibration results had good consistency at the other bands except for band 13. The uncertainty value of the whole calibration system was approximately 6.3%, and the uncertainty brought by the field surface measurement reached 5.4%, which might be the main reason for the relatively large deviation of band 13. This study verifies the feasibility of the vicarious calibration method at the Lake Qinghai RCS and provides the basis and reference for the subsequent on-orbit calibration of FY-3D/MERSI-II.

Download Full-text

Cobot User Frame Calibration: Evaluation and Comparison between Positioning Repeatability Performances Achieved by Traditional and Vision-Based Methods

Robotics ◽

10.3390/robotics10010045 ◽

2021 ◽

Vol 10 (1) ◽

pp. 45

Author(s):

Roberto Pagani ◽

Cristina Nuzzi ◽

Marco Ghidelli ◽

Alberto Borboni ◽

Matteo Lancini ◽

...

Keyword(s):

Quantitative Analysis ◽

Production Line ◽

Calibration Method ◽

Fiducial Marker ◽

Positioning System ◽

Local Calibration ◽

Calibration Methods ◽

Robot Positioning ◽

Good Repeatability ◽

Calibration Time

Since cobots are designed to be flexible, they are frequently repositioned to change the production line according to the needs; hence, their working area (user frame) needs to be often calibrated. Therefore, it is important to adopt a fast and intuitive user frame calibration method that allows even non-expert users to perform the procedure effectively, reducing the possible mistakes that may arise in such contexts. The aim of this work was to quantitatively assess the performance of different user frame calibration procedures in terms of accuracy, complexity, and calibration time, to allow a reliable choice of which calibration method to adopt and the number of calibration points to use, given the requirements of the specific application. This has been done by first analyzing the performances of a Rethink Robotics Sawyer robot built-in user frame calibration method (Robot Positioning System, RPS) based on the analysis of a fiducial marker distortion obtained from the image acquired by the wrist camera. This resulted in a quantitative analysis of the limitations of this approach that only computes local calibration planes, highlighting the reduction of performances observed. Hence, the analysis focused on the comparison between two traditional calibration methods involving rigid markers to determine the best number of calibration points to adopt to achieve good repeatability performances. The analysis shows that, among the three methods, the RPS one resulted in very poor repeatability performances (1.42 mm), while the three and five points calibration methods achieve lower values (0.33 mm and 0.12 mm, respectively) which are closer to the reference repeatability (0.08 mm). Moreover, comparing the overall calibration times achieved by the three methods, it is shown that, incrementing the number of calibration points to more than five, it is not suggested since it could lead to a plateau in the performances, while increasing the overall calibration time.

Download Full-text

Extrinsic Parameters Calibration Method of Cameras with Non-Overlapping Fields of View in Airborne Remote Sensing

Remote Sensing ◽

10.3390/rs10081298 ◽

2018 ◽

Vol 10 (8) ◽

pp. 1298 ◽

Cited By ~ 3

Author(s):

Lei Yin ◽

Xiangjun Wang ◽

Yubo Ni ◽

Kai Zhou ◽

Jilong Zhang

Keyword(s):

Remote Sensing ◽

Camera Calibration ◽

Calibration Method ◽

Optimization Method ◽

Airborne Remote Sensing ◽

Camera System ◽

Translation Error ◽

Camera Systems ◽

Calibration Methods ◽

Extrinsic Parameters

Multi-camera systems are widely used in the fields of airborne remote sensing and unmanned aerial vehicle imaging. The measurement precision of these systems depends on the accuracy of the extrinsic parameters. Therefore, it is important to accurately calibrate the extrinsic parameters between the onboard cameras. Unlike conventional multi-camera calibration methods with a common field of view (FOV), multi-camera calibration without overlapping FOVs has certain difficulties. In this paper, we propose a calibration method for a multi-camera system without common FOVs, which is used on aero photogrammetry. First, the extrinsic parameters of any two cameras in a multi-camera system is calibrated, and the extrinsic matrix is optimized by the re-projection error. Then, the extrinsic parameters of each camera are unified to the system reference coordinate system by using the global optimization method. A simulation experiment and a physical verification experiment are designed for the theoretical arithmetic. The experimental results show that this method is operable. The rotation error angle of the camera’s extrinsic parameters is less than 0.001rad and the translation error is less than 0.08 mm.

Download Full-text

Constrained Multiple Planar Reconstruction for Automatic Camera Calibration of Intelligent Vehicles

Sensors ◽

10.3390/s21144643 ◽

2021 ◽

Vol 21 (14) ◽

pp. 4643

Author(s):

Sang Jun Lee ◽

Jeawoo Lee ◽

Wonju Lee ◽

Cheolhun Jang

Keyword(s):

Camera Calibration ◽

Calibration Method ◽

Image Features ◽

Intelligent Vehicles ◽

Point Problem ◽

Reconstruction Method ◽

Calibration Methods ◽

The World ◽

Extrinsic Parameters ◽

Automatic Camera Calibration

In intelligent vehicles, extrinsic camera calibration is preferable to be conducted on a regular basis to deal with unpredictable mechanical changes or variations on weight load distribution. Specifically, high-precision extrinsic parameters between the camera coordinate and the world coordinate are essential to implement high-level functions in intelligent vehicles such as distance estimation and lane departure warning. However, conventional calibration methods, which solve a Perspective-n-Point problem, require laborious work to measure the positions of 3D points in the world coordinate. To reduce this inconvenience, this paper proposes an automatic camera calibration method based on 3D reconstruction. The main contribution of this paper is a novel reconstruction method to recover 3D points on planes perpendicular to the ground. The proposed method jointly optimizes reprojection errors of image features projected from multiple planar surfaces, and finally, it significantly reduces errors in camera extrinsic parameters. Experiments were conducted in synthetic simulation and real calibration environments to demonstrate the effectiveness of the proposed method.

Download Full-text

A Self-Calibration Method for Robotic Measurement System

10.1115/imece1999-0753 ◽

1999 ◽

Author(s):

Chunhe Gong ◽

Jingxia Yuan ◽

Jun Ni

Keyword(s):

Coordinate System ◽

Measurement System ◽

Calibration Method ◽

Position Measurement ◽

Robot Calibration ◽

Robot System ◽

Self Calibration ◽

World Coordinate System ◽

Robot Accuracy ◽

Calibration Methods

Abstract Robot calibration plays an increasingly important role in manufacturing. For robot calibration on the manufacturing floor, it is desirable that the calibration technique be easy and convenient to implement. This paper presents a new self-calibration method to calibrate and compensate for robot system kinematic errors. Compared with the traditional calibration methods, this calibration method has several unique features. First, it is not necessary to apply an external measurement system to measure the robot end-effector position for the purpose of kinematic identification since the robot measurement system has a sensor as its integral part. Second, this self-calibration is based on distance measurement rather than absolute position measurement for kinematic identification; therefore the calibration of the transformation from the world coordinate system to the robot base coordinate system, known as base calibration, is not necessary. These features not only greatly facilitate the robot system calibration but also shorten the error propagation chain, therefore, increase the accuracy of parameter estimation. An integrated calibration system is designed to validate the effectiveness of this calibration method. Experimental results show that after calibration there is a significant improvement of robot accuracy over a typical robot workspace.

Download Full-text

GUESS: projecting machine learning scores to well-calibrated probability estimates for clinical decision-making

Bioinformatics ◽

10.1093/bioinformatics/bty984 ◽

2018 ◽

Vol 35 (14) ◽

pp. 2458-2465 ◽

Cited By ~ 3

Author(s):

Johanna Schwarz ◽

Dominik Heider

Keyword(s):

Machine Learning ◽

Decision Support ◽

Clinical Decision Support ◽

State Of The Art ◽

Clinical Decision Support Systems ◽

Clinical Decision ◽

Calibration Method ◽

Probability Estimates ◽

Calibration Methods ◽

Novel Method

Abstract Motivation Clinical decision support systems have been applied in numerous fields, ranging from cancer survival toward drug resistance prediction. Nevertheless, clinical decision support systems typically have a caveat: many of them are perceived as black-boxes by non-experts and, unfortunately, the obtained scores cannot usually be interpreted as class probability estimates. In probability-focused medical applications, it is not sufficient to perform well with regards to discrimination and, consequently, various calibration methods have been developed to enable probabilistic interpretation. The aims of this study were (i) to develop a tool for fast and comparative analysis of different calibration methods, (ii) to demonstrate their limitations for the use on clinical data and (iii) to introduce our novel method GUESS. Results We compared the performances of two different state-of-the-art calibration methods, namely histogram binning and Bayesian Binning in Quantiles, as well as our novel method GUESS on both, simulated and real-world datasets. GUESS demonstrated calibration performance comparable to the state-of-the-art methods and always retained accurate class discrimination. GUESS showed superior calibration performance in small datasets and therefore may be an optimal calibration method for typical clinical datasets. Moreover, we provide a framework (CalibratR) for R, which can be used to identify the most suitable calibration method for novel datasets in a timely and efficient manner. Using calibrated probability estimates instead of original classifier scores will contribute to the acceptance and dissemination of machine learning based classification models in cost-sensitive applications, such as clinical research. Availability and implementation GUESS as part of CalibratR can be downloaded at CRAN.

Download Full-text