scholarly journals Global Monocular Indoor Positioning of a Robotic Vehicle with a Floorplan

Sensors ◽  
2019 ◽  
Vol 19 (3) ◽  
pp. 634 ◽  
Author(s):  
John Noonan ◽  
Hector Rotstein ◽  
Amir Geva ◽  
Ehud Rivlin
Keyword(s):  
Point Clouds ◽  
Indoor Positioning ◽  
Global Scale ◽  
Bundle Adjustment ◽  
Necessary Condition ◽  
Fusion Algorithm ◽  
Real World Data ◽  
Scale Invariant ◽  
Planar Structures ◽  
Robotic Vehicle

This paper presents a global monocular indoor positioning system for a robotic vehicle starting from a known pose. The proposed system does not depend on a dense 3D map, require prior environment exploration or installation, or rely on the scene remaining the same, photometrically or geometrically. The approach presents a new way of providing global positioning relying on the sparse knowledge of the building floorplan by utilizing special algorithms to resolve the unknown scale through wall–plane association. This Wall Plane Fusion algorithm presented finds correspondences between walls of the floorplan and planar structures present in the 3D point cloud. In order to extract planes from point clouds that contain scale ambiguity, the Scale Invariant Planar RANSAC (SIPR) algorithm was developed. The best wall–plane correspondence is used as an external constraint to a custom Bundle Adjustment optimization which refines the motion estimation solution and enforces a global scale solution. A necessary condition is that only one wall needs to be in view. The feasibility of using the algorithms is tested with synthetic and real-world data; extensive testing is performed in an indoor simulation environment using the Unreal Engine and Microsoft Airsim. The system performs consistently across all three types of data. The tests presented in this paper show that the standard deviation of the error did not exceed 6 cm.

A novel tracking system for human following robots with fusion of MMW radar and monocular vision

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Yipeng Zhu ◽  
Tao Wang ◽  
Shiqiang Zhu
Keyword(s):  
Sensor Fusion ◽  
Tracking System ◽  
Cost Effective ◽  
Point Clouds ◽  
Human Detection ◽  
Multimodal Fusion ◽  
Target Person ◽  
Fusion Algorithm ◽  
Content Type ◽  
Tracking Method

Purpose This paper aims to develop a robust person tracking method for human following robots. The tracking system adopts the multimodal fusion results of millimeter wave (MMW) radars and monocular cameras for perception. A prototype of human following robot is developed and evaluated by using the proposed tracking system. Design/methodology/approach Limited by angular resolution, point clouds from MMW radars are too sparse to form features for human detection. Monocular cameras can provide semantic information for objects in view, but cannot provide spatial locations. Considering the complementarity of the two sensors, a sensor fusion algorithm based on multimodal data combination is proposed to identify and localize the target person under challenging conditions. In addition, a closed-loop controller is designed for the robot to follow the target person with expected distance. Findings A series of experiments under different circumstances are carried out to validate the fusion-based tracking method. Experimental results show that the average tracking errors are around 0.1 m. It is also found that the robot can handle different situations and overcome short-term interference, continually track and follow the target person. Originality/value This paper proposed a robust tracking system with the fusion of MMW radars and cameras. Interference such as occlusion and overlapping are well handled with the help of the velocity information from the radars. Compared to other state-of-the-art plans, the sensor fusion method is cost-effective and requires no additional tags with people. Its stable performance shows good application prospects in human following robots.

scholarly journals Hierarchical Regularization of Building Boundaries in Noisy Aerial Laser Scanning and Photogrammetric Point Clouds

2018 ◽  
Vol 10 (12) ◽  
pp. 1996 ◽  
Author(s):  
Linfu Xie ◽  
Qing Zhu ◽  
Han Hu ◽  
Bo Wu ◽  
Yuan Li ◽  
...  
Keyword(s):  
Laser Scanning ◽  
Point Clouds ◽  
Normal Vector ◽  
Aerial Laser Scanning ◽  
Second Stage ◽  
Boundary Points ◽  
Planar Structures ◽  
Markov Random ◽  
Two Stages ◽  
Method Show

Aerial laser scanning or photogrammetric point clouds are often noisy at building boundaries. In order to produce regularized polygons from such noisy point clouds, this study proposes a hierarchical regularization method for the boundary points. Beginning with detected planar structures from raw point clouds, two stages of regularization are employed. In the first stage, the boundary points of an individual plane are consolidated locally by shifting them along their refined normal vector to resist noise, and then grouped into piecewise smooth segments. In the second stage, global regularities among different segments from different planes are softly enforced through a labeling process, in which the same label represents parallel or orthogonal segments. This is formulated as a Markov random field and solved efficiently via graph cut. The performance of the proposed method is evaluated for extracting 2D footprints and 3D polygons of buildings in metropolitan area. The results reveal that the proposed method is superior to the state-of-art methods both qualitatively and quantitatively in compactness. The simplified polygons could fit the original boundary points with an average residuals of 0.2 m, and in the meantime reduce up to 90% complexities of the edges. The satisfactory performances of the proposed method show a promising potential for 3D reconstruction of polygonal models from noisy point clouds.

scholarly journals Intact Planar Abstraction of Buildings via Global Normal Refinement from Noisy Oblique Photogrammetric Point Clouds

2018 ◽  
Vol 7 (11) ◽  
pp. 431 ◽  
Author(s):  
Qing Zhu ◽  
Feng Wang ◽  
Han Hu ◽  
Yulin Ding ◽  
Jiali Xie ◽  
...  
Keyword(s):  
Global Optimization ◽  
Three Dimensional ◽  
Region Growing ◽  
Point Clouds ◽  
Spatial Relations ◽  
Global Scale ◽  
Surface Boundary ◽  
Planar Surfaces ◽  
Normal Vectors ◽  
Almost All

Oblique photogrammetric point clouds are currently one of the major data sources for the three-dimensional level-of-detail reconstruction of buildings. However, they are severely noise-laden and pose serious problems for the effective and automatic surface extraction of buildings. In addition, conventional methods generally use normal vectors estimated in a local neighborhood, which are liable to be affected by noise, leading to inferior results in successive building reconstruction. In this paper, we propose an intact planar abstraction method for buildings, which explicitly handles noise by integrating information in a larger context through global optimization. The information propagates hierarchically from a local to global scale through the following steps: first, based on voxel cloud connectivity segmentation, single points are clustered into supervoxels that are enforced to not cross the surface boundary; second, each supervoxel is expanded to nearby supervoxels through the maximal support region, which strictly enforces planarity; third, the relationships established by the maximal support regions are injected into a global optimization, which reorients the local normal vectors to be more consistent in a larger context; finally, the intact planar surfaces are obtained by region growing using robust normal and point connectivity in the established spatial relations. Experiments on the photogrammetric point clouds obtained from oblique images showed that the proposed method is effective in reducing the influence of noise and retrieving almost all of the major planar structures of the examined buildings.

scholarly journals 3D Object Detection Using Scale Invariant and Feature Reweighting Networks

2019 ◽  
Vol 33 ◽  
pp. 9267-9274 ◽  
Author(s):  
Xin Zhao ◽  
Zhe Liu ◽  
Ruolan Hu ◽  
Kaiqi Huang
Keyword(s):  
Object Detection ◽  
Network Architecture ◽  
Point Clouds ◽  
Scale Invariant ◽  
3D Object ◽  
Outdoor Scenes ◽  
Indoor Scenes ◽  
Bounding Boxes ◽  
The One ◽  
3D Object Detection

3D object detection plays an important role in a large number of real-world applications. It requires us to estimate the localizations and the orientations of 3D objects in real scenes. In this paper, we present a new network architecture which focuses on utilizing the front view images and frustum point clouds to generate 3D detection results. On the one hand, a PointSIFT module is utilized to improve the performance of 3D segmentation. It can capture the information from different orientations in space and the robustness to different scale shapes. On the other hand, our network obtains the useful features and suppresses the features with less information by a SENet module. This module reweights channel features and estimates the 3D bounding boxes more effectively. Our method is evaluated on both KITTI dataset for outdoor scenes and SUN-RGBD dataset for indoor scenes. The experimental results illustrate that our method achieves better performance than the state-of-the-art methods especially when point clouds are highly sparse.

scholarly journals LiDAR-Aided Interior Orientation Parameters Refinement Strategy for Consumer-Grade Cameras Onboard UAV Remote Sensing Systems

2020 ◽  
Vol 12 (14) ◽  
pp. 2268
Author(s):  
Tian Zhou ◽  
Seyyed Meghdad Hasheminasab ◽  
Radhika Ravi ◽  
Ayman Habib
Keyword(s):  
Precision Agriculture ◽  
Point Cloud ◽  
Point Clouds ◽  
Bundle Adjustment ◽  
Flying Height ◽  
Control Points ◽  
Object Space ◽  
Area Of Interest ◽  
Refinement Strategy ◽  
Orientation Parameters

Unmanned aerial vehicles (UAVs) are quickly emerging as a popular platform for 3D reconstruction/modeling in various applications such as precision agriculture, coastal monitoring, and emergency management. For such applications, LiDAR and frame cameras are the two most commonly used sensors for 3D mapping of the object space. For example, point clouds for the area of interest can be directly derived from LiDAR sensors onboard UAVs equipped with integrated global navigation satellite systems and inertial navigation systems (GNSS/INS). Imagery-based mapping, on the other hand, is considered to be a cost-effective and practical option and is often conducted by generating point clouds and orthophotos using structure from motion (SfM) techniques. Mapping with photogrammetric approaches requires accurate camera interior orientation parameters (IOPs), especially when direct georeferencing is utilized. Most state-of-the-art approaches for determining/refining camera IOPs depend on ground control points (GCPs). However, establishing GCPs is expensive and labor-intensive, and more importantly, the distribution and number of GCPs are usually less than optimal to provide adequate control for determining and/or refining camera IOPs. Moreover, consumer-grade cameras with unstable IOPs have been widely used for mapping applications. Therefore, in such scenarios, where frequent camera calibration or IOP refinement is required, GCP-based approaches are impractical. To eliminate the need for GCPs, this study uses LiDAR data as a reference surface to perform in situ refinement of camera IOPs. The proposed refinement strategy is conducted in three main steps. An image-based sparse point cloud is first generated via a GNSS/INS-assisted SfM strategy. Then, LiDAR points corresponding to the resultant image-based sparse point cloud are identified through an iterative plane fitting approach and are referred to as LiDAR control points (LCPs). Finally, IOPs of the utilized camera are refined through a GNSS/INS-assisted bundle adjustment procedure using LCPs. Seven datasets over two study sites with a variety of geomorphic features are used to evaluate the performance of the developed strategy. The results illustrate the ability of the proposed approach to achieve an object space absolute accuracy of 3–5 cm (i.e., 5–10 times the ground sampling distance) at a 41 m flying height.

scholarly journals Individual Tree Segmentation Method Based on Mobile Backpack LiDAR Point Clouds

Sensors ◽  
2021 ◽  
Vol 21 (18) ◽  
pp. 6007
Author(s):  
Lino Comesaña-Cebral ◽  
Joaquín Martínez-Sánchez ◽  
Henrique Lorenzo ◽  
Pedro Arias
Keyword(s):  
Forest Management ◽  
Data Collection ◽  
Laser Scanning ◽  
Point Clouds ◽  
Real World Data ◽  
High Detection Rate ◽  
Individual Tree ◽  
Competition Analysis ◽  
Dbscan Clustering ◽  
Growth Features

Individual tree (IT) segmentation is crucial for forest management, supporting forest inventory, biomass monitoring or tree competition analysis. Light detection and ranging (LiDAR) is a prominent technology in this context, outperforming competing technologies. Aerial laser scanning (ALS) is frequently used for forest documentation, showing good point densities at the tree-top surface. Even though under-canopy data collection is possible with multi-echo ALS, the number of points for regions near the ground in leafy forests drops drastically, and, as a result, terrestrial laser scanners (TLS) may be required to obtain reliable information about tree trunks or under-growth features. In this work, an IT extraction method for terrestrial backpack LiDAR data is presented. The method is based on DBSCAN clustering and cylinder voxelization of the volume, showing a high detection rate (∼90%) for tree locations obtained from point clouds, and low commission and submission errors (accuracy over 93%). The method includes a sensibility assessment to calculate the optimal input parameters and adapt the workflow to real-world data. This approach shows that forest management can benefit from IT segmentation, using a handheld TLS to improve data collection productivity.

scholarly journals INDOOR POSITIONING AND NAVIGATION BASED ON CONTROL SPHERECAL PANORAMIC IMAGES

2016 ◽  
Vol XLI-B6 ◽  
pp. 251-258 ◽  
Author(s):  
Tsung-Che Huang ◽  
Yi-Hsing Tseng
Keyword(s):  
Indoor Positioning ◽  
Image Features ◽  
Bundle Adjustment ◽  
Target Space ◽  
Mobile Mapping ◽  
Technology Roadmap ◽  
Mapping Technology ◽  
Panoramic Images ◽  
Novel Method ◽  
Indoor And Outdoor

Continuous indoor and outdoor positioning and navigation is the goal to achieve in the field of mobile mapping technology. However, accuracy of positioning and navigation will be largely degraded in indoor or occluded areas, due to receiving weak or less GNSS signals. Targeting the need of high accuracy indoor and outdoor positioning and navigation for mobile mapping applications, the objective of this study is to develop a novel method of indoor positioning and navigation with the use of spherical panoramic image (SPI). Two steps are planned in the technology roadmap. First, establishing a control SPI database that contains a good number of well-distributed control SPIs pre-acquired in the target space. A control SPI means an SPI with known exterior orientation parameters, which can be solved with a network bundle adjustment of SPIs. Having a control SPI database, the target space will be ready to provide the service of positioning and navigation. Secondly, the position and orientation of a newly taken SPI can be solved by using overlapped SPIs searched from the control SPI database. The method of matching SPIs and finding conjugate image features will be developed and tested. Two experiments will be planned and conducted in this paper to test the feasibility and validate the test results of the proposed methods. Analysis of appropriate number and distribution of needed control SPIs will also be included in the experiments with respect to different test cases.

Sign in / Sign up

Export Citation Format

Share Document