scholarly journals Feature Pyramid Network Based Efficient Normal Estimation and Filtering for Time-of-Flight Depth Cameras

Sensors ◽  
2021 ◽  
Vol 21 (18) ◽  
pp. 6257
Author(s):  
Szilárd Molnár ◽  
Benjamin Kelényi ◽  
Levente Tamas
Keyword(s):  
Time Of Flight ◽  
Estimation Method ◽  
Depth Cameras ◽  
Filtering Method ◽  
Normal Estimation ◽  
Depth Images ◽  
Order Of Magnitude ◽  
Feature Pyramid ◽  
Public Datasets ◽  
Estimation And Filtering

In this paper, an efficient normal estimation and filtering method for depth images acquired by Time-of-Flight (ToF) cameras is proposed. The method is based on a common feature pyramid networks (FPN) architecture. The normal estimation method is called ToFNest, and the filtering method ToFClean. Both of these low-level 3D point cloud processing methods start from the 2D depth images, projecting the measured data into the 3D space and computing a task-specific loss function. Despite the simplicity, the methods prove to be efficient in terms of robustness and runtime. In order to validate the methods, extensive evaluations on public and custom datasets were performed. Compared with the state-of-the-art methods, the ToFNest and ToFClean algorithms are faster by an order of magnitude without losing precision on public datasets.

ToFNest: Efficient normal estimation for time-of-flight depth cameras

2021 ◽  
Author(s):  
Szilard Molnar ◽  
Benjamin Kelenyi ◽  
Levente Tamas
Keyword(s):  
Time Of Flight ◽  
Depth Cameras ◽  
Normal Estimation

Estimation method of calorie intake by deep learning using depth images obtained through a single camera smartphone

2021 ◽  
Author(s):  
Kazuyuki Kaneda ◽  
Tatsuya Ooba ◽  
Hideki Shimada ◽  
Osamu Shiku ◽  
Yuji Teshima
Keyword(s):  
Deep Learning ◽  
Estimation Method ◽  
Calorie Intake ◽  
Single Camera ◽  
Depth Images

Range unfolding for Time-of-Flight depth cameras

2010 ◽  
Author(s):  
Ouk Choi ◽  
Hwasup Lim ◽  
Byongmin Kang ◽  
Yong Sun Kim ◽  
Keechang Lee ◽  
...  
Keyword(s):  
Time Of Flight ◽  
Depth Cameras

scholarly journals Consistency-Check Edge Refinement for Deep Stereo Matching

2020 ◽  
Author(s):  
Fangrui Wu ◽  
Menglong Yang
Keyword(s):  
Computational Efficiency ◽  
Stereo Matching ◽  
Information Aggregation ◽  
Experimental Results ◽  
Global Information ◽  
Consistency Check ◽  
Filtering Method ◽  
Tightly Coupled ◽  
End To End ◽  
Public Datasets

Recent end-to-end CNN-based stereo matching algorithms obtain disparities through regression from a cost volume, which is formed by concatenating the features of stereo pairs. Some downsampling steps are often embedded in constructing cost volume for global information aggregation and computational efficiency. However, many edge details are hard to recover due to the imprudent upsampling process and ambiguous boundary predictions. To tackle this problem without training another edge prediction sub-network, we developed a novel tightly-coupled edge refinement pipeline composed of two modules. The first module implements a gentle upsampling process by a cascaded cost volume filtering method, aggregating global information without losing many details. On this basis, the second module concentrates on generating a disparity residual map for boundary pixels by sub-pixel disparity consistency check, to further recover the edge details. The experimental results on public datasets demonstrate the effectiveness of the proposed method.

scholarly journals Refinement of the ice absorption spectrum in the visible using radiance profile measurements in Antarctic snow

2016 ◽  
Author(s):  
Ghislain Picard ◽  
Quentin Libois ◽  
Laurent Arnaud
Keyword(s):  
Absorption Coefficient ◽  
Fiber Optics ◽  
Estimation Method ◽  
Radiative Transfer Model ◽  
Transfer Model ◽  
Snow Layer ◽  
Antarctic Snow ◽  
Order Of Magnitude ◽  
The Difference ◽  
The Antarctic

Abstract. Ice is a highly transparent material in the visible. According to the most widely used database (Warren and Brandt, 2008; IA2008), the ice absorption coefficient reaches values lower than 10−3 m−1 around 400 nm. These values were obtained from a radiance profile measured in a single snow layer at Dome C in Antarctica. We reproduced this experiment using a fiber optics inserted in the snow to record 56 profiles from which 70 homogeneous layers were identified. Applying the same estimation method on every layer yields 70 ice absorption spectra with a significant variability and overall larger than IA2008 by one order of magnitude. We devise another estimation method based on Bayesian inference. It reduces the statistical variability and confirms the higher absorption, around 2 × 10−2 m−1 near the minimum at 440 nm. We explore potential instrumental artifacts by developing a 3D radiative transfer model able to explicitly account for the presence of the fiber in the snow. The simulation results show that the radiance profile is indeed perturbed by the fiber intrusion but the error on the ice absorption estimate is not larger than a factor 2. This is insufficient to explain the difference between our new estimate and IA2008. Nevertheless, considering the number of profiles acquired for this study and other estimates from the Antarctic Muon and Neutrino Detector Array (AMANDA), we estimate that ice absorption values around 10−2 m−1 at the minimum are more likely than under 10−3 m−1. We provide a new estimate in the range 400–600 nm for future modeling of snow, cloud, and sea-ice optical properties. Most importantly we recommend that modeling studies take into account the large uncertainty of the ice absorption coefficient in the visible.

scholarly journals LARGE INFORMATION PLUS NOISE RANDOM MATRIX MODELS AND CONSISTENT SUBSPACE ESTIMATION IN LARGE SENSOR NETWORKS

2012 ◽  
Vol 01 (02) ◽  
pp. 1150006 ◽  
Author(s):  
WALID HACHEM ◽  
PHILIPPE LOUBATON ◽  
XAVIER MESTRE ◽  
JAMAL NAJIM ◽  
PASCAL VALLET
Keyword(s):  
Covariance Matrix ◽  
Estimation Method ◽  
Singular Values ◽  
Subspace Methods ◽  
Localization Function ◽  
Random Matrix Models ◽  
Order Of Magnitude ◽  
Source Signal ◽  
Subspace Estimation ◽  
Empirical Covariance

In array processing, a common problem is to estimate the angles of arrival of K deterministic sources impinging on an array of M antennas, from N observations of the source signal, corrupted by Gaussian noise. In the so-called subspace methods, the problem reduces to estimate a quadratic form (called "localization function") of a certain projection matrix related to the source signal empirical covariance matrix. The estimates of the angles of arrival are then obtained by taking the K deepest local minima of the estimated localization function. Recently, a new subspace estimation method has been proposed, in the context where the number of available samples N is of the same order of magnitude than the number of sensors M. In this context, the traditional subspace methods tend to fail because they are based on the empirical covariance matrix of the observations which is a poor estimate of the source signal covariance matrix. The new subspace method is based on a consistent estimator of the localization function in the regime where M and N tend to +∞ at the same rate. However, the consistency of the angles estimator was not addressed, and the purpose of this paper is to prove this consistency in the previous asymptotic regime. For this, we prove the property that the singular values of M × N Gaussian information plus noise matrix escape from certain intervals is an event of probability decreasing at rate [Formula: see text] for all p. A regularization trick is also introduced, which allows to confine these singular values into certain intervals and to use standard tools as Poincaré inequality to characterize any moments of the estimator. These results are believed to be of independent interest.

A Real-Time Compressive Tracking System for Amphibious Spherical Robots

2018 ◽  
pp. 390-409
Author(s):  
Shuxiang Guo ◽  
Liwei Shi
Keyword(s):  
High Speed ◽  
Tracking System ◽  
Microsoft Kinect ◽  
Depth Cameras ◽  
Compressive Tracking ◽  
Depth Images ◽  
Working Environments ◽  
Robotic Tracking ◽  
Prediction Mechanism ◽  
Amphibious Robot

Given the special working environments and application functions of the amphibious robot, an improved RGB-D visual tracking algorithm with dual trackers is proposed and implemented in this chapter. Compressive tracking (CT) was selected as the basis of the proposed algorithm to process colour images from a RGB-D camera, and a Kalman filter with a second-order motion model was added to the CT tracker to predict the state of the target, select candidate patches or samples, and reinforce the tracker's robustness to high-speed moving targets. In addition, a variance ratio features shift (VR-V) tracker with a Kalman prediction mechanism was adopted to process depth images from a RGB-D camera. A visible and infrared fusion mechanism or feedback strategy is introduced in the proposed algorithm to enhance its adaptability and robustness. To evaluate the effectiveness of the algorithm, Microsoft Kinect, which is a combination of colour and depth cameras, was adopted for use in a prototype of the robotic tracking system.

scholarly journals Robust Visual Compass Using Hybrid Features for Indoor Environments

Electronics ◽  
2019 ◽  
Vol 8 (2) ◽  
pp. 220 ◽  
Author(s):  
Ruibin Guo ◽  
Keju Peng ◽  
Dongxiang Zhou ◽  
Yunhui Liu
Keyword(s):  
Autonomous Navigation ◽  
Estimation Method ◽  
Rotation Matrix ◽  
Indoor Environments ◽  
3D Mapping ◽  
Orientation Estimation ◽  
Hybrid Features ◽  
Depth Images ◽  
Improved Accuracy ◽  
Crucial Part

Orientation estimation is a crucial part of robotics tasks such as motion control, autonomous navigation, and 3D mapping. In this paper, we propose a robust visual-based method to estimate robots’ drift-free orientation with RGB-D cameras. First, we detect and track hybrid features (i.e., plane, line, and point) from color and depth images, which provides reliable constraints even in uncharacteristic environments with low texture or no consistent lines. Then, we construct a cost function based on these features and, by minimizing this function, we obtain the accurate rotation matrix of each captured frame with respect to its reference keyframe. Furthermore, we present a vanishing direction-estimation method to extract the Manhattan World (MW) axes; by aligning the current MW axes with the global MW axes, we refine the aforementioned rotation matrix of each keyframe and achieve drift-free orientation. Experiments on public RGB-D datasets demonstrate the robustness and accuracy of the proposed algorithm for orientation estimation. In addition, we have applied our proposed visual compass to pose estimation, and the evaluation on public sequences shows improved accuracy.

scholarly journals Improvements to a long-term Rayleigh-scatter lidar temperature climatology by using an optimal estimation method

2018 ◽  
Vol 11 (11) ◽  
pp. 6043-6058 ◽  
Author(s):  
Ali Jalali ◽  
Robert J. Sica ◽  
Alexander Haefele
Keyword(s):  
Traditional Method ◽  
Middle Atmosphere ◽  
Density Measurement ◽  
Estimation Method ◽  
Optimal Estimation ◽  
Temperature Profiles ◽  
Vertical Resolution ◽  
Order Of Magnitude ◽  
Lidar Equation ◽  
Model Temperature

Abstract. Hauchecorne and Chanin (1980) developed a robust method to calculate middle-atmosphere temperature profiles using measurements from Rayleigh-scatter lidars. This traditional method has been successfully used to greatly improve our understanding of middle-atmospheric dynamics, but the method has some shortcomings regarding the calculation of systematic uncertainties and the vertical resolution of the retrieval. Sica and Haefele (2015) have shown that the optimal estimation method (OEM) addresses these shortcomings and allows temperatures to be retrieved with confidence over a greater range of heights than the traditional method. We have calculated a temperature climatology from 519 nights of Purple Crow Lidar Rayleigh-scatter measurements using an OEM. Our OEM retrieval is a first-principle retrieval in which the forward model is the lidar equation and the measurements are the level-0 count returns. It includes a quantitative determination of the top altitude of the retrieved temperature profiles, the evaluation of nine systematic plus random uncertainties, and the vertical resolution of the retrieval on a profile-by-profile basis. Our OEM retrieval allows for the vertical resolution to vary with height, extending the retrieval in altitude 5 to 10 km higher than the traditional method. It also allows the comparison of the traditional method's sensitivity to two in-principle equivalent methods of specifying the seed pressure: using a model pressure seed versus using a model temperature combined with the lidar's density measurement to calculate the seed pressure. We found that the seed pressure method is superior to using a model temperature combined with the lidar-derived density. The increased altitude capability of our OEM retrievals allows for a comparison of the Rayleigh-scatter lidar temperatures throughout the entire altitude range of the sodium lidar temperature measurements. Our OEM-derived Rayleigh temperatures are shown to have improved agreement relative to our previous comparisons using the traditional method, and the agreement of the OEM-derived temperatures is the same as the agreement between existing sodium lidar temperature climatologies. This detailed study of the calculation of the new Purple Crow Lidar temperature climatology using the OEM establishes that it is both highly advantageous and practical to reprocess existing Rayleigh-scatter lidar measurements that cover long time periods, during which time the lidar may have undergone several significant equipment upgrades, while gaining an upper limit to useful temperature retrievals equivalent to an order of magnitude increase in power-aperture product due to the use of an OEM.

Sign in / Sign up

Export Citation Format

Share Document