scholarly journals BLAINDER—A Blender AI Add-On for Generation of Semantically Labeled Depth-Sensing Data

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2144
Author(s):  
Stefan Reitmann ◽  
Lorenzo Neumann ◽  
Bernhard Jung
Keyword(s):  
Point Clouds ◽  
Training Data ◽  
Sensor Data ◽  
Generation Process ◽  
Data Generation ◽  
Depth Sensor ◽  
Depth Sensing ◽  
Depth Sensors ◽  
3D Point Clouds ◽  
Wide Range

Common Machine-Learning (ML) approaches for scene classification require a large amount of training data. However, for classification of depth sensor data, in contrast to image data, relatively few databases are publicly available and manual generation of semantically labeled 3D point clouds is an even more time-consuming task. To simplify the training data generation process for a wide range of domains, we have developed the BLAINDER add-on package for the open-source 3D modeling software Blender, which enables a largely automated generation of semantically annotated point-cloud data in virtual 3D environments. In this paper, we focus on classical depth-sensing techniques Light Detection and Ranging (LiDAR) and Sound Navigation and Ranging (Sonar). Within the BLAINDER add-on, different depth sensors can be loaded from presets, customized sensors can be implemented and different environmental conditions (e.g., influence of rain, dust) can be simulated. The semantically labeled data can be exported to various 2D and 3D formats and are thus optimized for different ML applications and visualizations. In addition, semantically labeled images can be exported using the rendering functionalities of Blender.

Decoupled Iterative Deep Sensor Fusion for 3D Semantic Segmentation

2021 ◽  
Vol 15 (03) ◽  
pp. 293-312
Author(s):  
Fabian Duerr ◽  
Hendrik Weigel ◽  
Jürgen Beyerer
Keyword(s):  
Autonomous Vehicles ◽  
Network Architecture ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Sensor Data ◽  
Range Image ◽  
Fusion Strategy ◽  
3D Point Clouds ◽  
3D Environment ◽  
Wide Range

One of the key tasks for autonomous vehicles or robots is a robust perception of their 3D environment, which is why autonomous vehicles or robots are equipped with a wide range of different sensors. Building upon a robust sensor setup, understanding and interpreting their 3D environment is the next important step. Semantic segmentation of 3D sensor data, e.g. point clouds, provides valuable information for this task and is often seen as key enabler for 3D scene understanding. This work presents an iterative deep fusion architecture for semantic segmentation of 3D point clouds, which builds upon a range image representation of the point clouds and additionally exploits camera features to increase accuracy and robustness. In contrast to other approaches, which fuse lidar and camera features once, the proposed fusion strategy iteratively combines and refines lidar and camera features at different scales inside the network architecture. Additionally, the proposed approach can deal with camera failure as well as jointly predict lidar and camera segmentation. We demonstrate the benefits of the presented iterative deep fusion approach on two challenging datasets, outperforming all range image-based lidar and fusion approaches. An in-depth evaluation underlines the effectiveness of the proposed fusion strategy and the potential of camera features for 3D semantic segmentation.

scholarly journals 3D MODEL FOR INDOOR SPACES USING DEPTH SENSOR

2019 ◽  
Vol XLII-4/W16 ◽  
pp. 471-479
Author(s):  
N. F. Mukhtar ◽  
S. Azri ◽  
U. Ujang ◽  
M. G. Cuétara ◽  
G. M. Retortillo ◽  
...  
Keyword(s):  
Data Processing ◽  
Data Acquisition ◽  
High Speed ◽  
3D Model ◽  
High Mobility ◽  
Point Clouds ◽  
Sensor Data ◽  
Depth Sensor ◽  
3D Point Clouds ◽  
Indoor Spaces

Abstract. In recent years, 3D model for indoor spaces have become highly demanded in the development of technology. Many approaches to 3D visualisation and modelling especially for indoor environment was developed such as laser scanner, photogrammetry, computer vision, image and many more. However, most of the technique relies on the experience of the operator to get the best result. Besides that, the equipment is quite expensive and time-consuming in terms of processing. This paper focuses on the data acquisition and visualisation of a 3D model for an indoor space by using a depth sensor. In this study, EyesMap3D Pro by Ecapture is used to collect 3D data of the indoor spaces. The EyesMap3D Pro depth sensor is able to generate 3D point clouds in high speed and high mobility due to the portability and light weight of the device. However, more attention must be paid on data acquisition, data processing, visualizing, and evaluation of the depth sensor data. Hence, this paper will discuss the data processing from extracting features from 3D point clouds to 3D indoor models. Afterwards, the evaluation on the 3D models is made to ensure the suitability in indoor model and indoor mapping application. In this study, the 3D model was exported to 3D GIS-ready format for displaying and storing more information of the indoor spaces.

scholarly journals HYBRID ACQUISITION OF HIGH QUALITY TRAINING DATA FOR SEMANTIC SEGMENTATION OF 3D POINT CLOUDS USING CROWD-BASED ACTIVE LEARNING

2020 ◽  
Vol V-2-2020 ◽  
pp. 501-508
Author(s):  
M. Kölle ◽  
V. Walter ◽  
S. Schmohl ◽  
U. Soergel
Keyword(s):  
Active Learning ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Human Interaction ◽  
Training Data ◽  
Semantic Interpretation ◽  
Training Dataset ◽  
3D Point Clouds ◽  
Semantic Labeling ◽  
3D Cnn

Abstract. Automated semantic interpretation of 3D point clouds is crucial for many tasks in the domain of geospatial data analysis. For this purpose, labeled training data is required, which has often to be provided manually by experts. One approach to minimize effort in terms of costs of human interaction is Active Learning (AL). The aim is to process only the subset of an unlabeled dataset that is particularly helpful with respect to class separation. Here a machine identifies informative instances which are then labeled by humans, thereby increasing the performance of the machine. In order to completely avoid involvement of an expert, this time-consuming annotation can be resolved via crowdsourcing. Therefore, we propose an approach combining AL with paid crowdsourcing. Although incorporating human interaction, our method can run fully automatically, so that only an unlabeled dataset and a fixed financial budget for the payment of the crowdworkers need to be provided. We conduct multiple iteration steps of the AL process on the ISPRS Vaihingen 3D Semantic Labeling benchmark dataset (V3D) and especially evaluate the performance of the crowd when labeling 3D points. We prove our concept by using labels derived from our crowd-based AL method for classifying the test dataset. The analysis outlines that by labeling only 0:4% of the training dataset by the crowd and spending less than 145 $, both our trained Random Forest and sparse 3D CNN classifier differ in Overall Accuracy by less than 3 percentage points compared to the same classifiers trained on the complete V3D training set.

Efficient nonparametric belief propagation for pose estimation and manipulation of articulated objects

2019 ◽  
Vol 4 (30) ◽  
pp. eaaw4523 ◽  
Author(s):  
Karthik Desingh ◽  
Shiyang Lu ◽  
Anthony Opipari ◽  
Odest Chadwicke Jenkins
Keyword(s):  
Pose Estimation ◽  
Message Passing ◽  
Belief Propagation ◽  
Sensor Data ◽  
Depth Sensor ◽  
Continuous Space ◽  
Articulated Objects ◽  
Message Passing Algorithm ◽  
Wide Range ◽  
Object Part

Robots working in human environments often encounter a wide range of articulated objects, such as tools, cabinets, and other jointed objects. Such articulated objects can take an infinite number of possible poses, as a point in a potentially high-dimensional continuous space. A robot must perceive this continuous pose to manipulate the object to a desired pose. This problem of perception and manipulation of articulated objects remains a challenge due to its high dimensionality and multimodal uncertainty. Here, we describe a factored approach to estimate the poses of articulated objects using an efficient approach to nonparametric belief propagation. We consider inputs as geometrical models with articulation constraints and observed RGBD (red, green, blue, and depth) sensor data. The described framework produces object-part pose beliefs iteratively. The problem is formulated as a pairwise Markov random field (MRF), where each hidden node (continuous pose variable) is an observed object-part’s pose and the edges denote the articulation constraints between the parts. We describe articulated pose estimation by a “pull” message passing algorithm for nonparametric belief propagation (PMPNBP) and evaluate its convergence properties over scenes with articulated objects. Robot experiments are provided to demonstrate the necessity of maintaining beliefs to perform goal-driven manipulation tasks.

scholarly journals Registration of spatio-temporal point clouds of plants for phenotyping

PLoS ONE ◽  
2021 ◽  
Vol 16 (2) ◽  
pp. e0247243
Author(s):  
Nived Chebrolu ◽  
Federico Magistri ◽  
Thomas Läbe ◽  
Cyrill Stachniss
Keyword(s):  
Plant Traits ◽  
Scale Up ◽  
Rigid Motion ◽  
Point Clouds ◽  
Plant Performance ◽  
Sensor Data ◽  
Plant Phenotyping ◽  
Compact Representation ◽  
3D Point Clouds ◽  
Over Time

Plant phenotyping is a central task in crop science and plant breeding. It involves measuring plant traits to describe the anatomy and physiology of plants and is used for deriving traits and evaluating plant performance. Traditional methods for phenotyping are often time-consuming operations involving substantial manual labor. The availability of 3D sensor data of plants obtained from laser scanners or modern depth cameras offers the potential to automate several of these phenotyping tasks. This automation can scale up the phenotyping measurements and evaluations that have to be performed to a larger number of plant samples and at a finer spatial and temporal resolution. In this paper, we investigate the problem of registering 3D point clouds of the plants over time and space. This means that we determine correspondences between point clouds of plants taken at different points in time and register them using a new, non-rigid registration approach. This approach has the potential to form the backbone for phenotyping applications aimed at tracking the traits of plants over time. The registration task involves finding data associations between measurements taken at different times while the plants grow and change their appearance, allowing 3D models taken at different points in time to be compared with each other. Registering plants over time is challenging due to its anisotropic growth, changing topology, and non-rigid motion in between the time of the measurements. Thus, we propose a novel approach that first extracts a compact representation of the plant in the form of a skeleton that encodes both topology and semantic information, and then use this skeletal structure to determine correspondences over time and drive the registration process. Through this approach, we can tackle the data association problem for the time-series point cloud data of plants effectively. We tested our approach on different datasets acquired over time and successfully registered the 3D plant point clouds recorded with a laser scanner. We demonstrate that our method allows for developing systems for automated temporal plant-trait analysis by tracking plant traits at an organ level.

scholarly journals Transfer Learning Strategies for Anomaly Detection in IoT Vibration Data

2021 ◽  
Author(s):  
Clemens Heistracher ◽  
Anahid Jalali ◽  
Indu Strobl ◽  
Axel Suendermann ◽  
Sebastian Meixner ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Learning Strategies ◽  
Transfer Learning ◽  
Training Data ◽  
Sensor Data ◽  
Classification Models ◽  
Vibration Data ◽  
Starting Point ◽  
Wide Range ◽  
Anomaly Classification

<div>Abstract—An increasing number of industrial assets are equipped with IoT sensor platforms and the industry now expects data-driven maintenance strategies with minimal deployment costs. However, gathering labeled training data for supervised tasks such as anomaly detection is costly and often difficult to implement in operational environments. Therefore, this work aims to design and implement a solution that reduces the required amount of data for training anomaly classification models on time series sensor data and thereby brings down the overall deployment effort of IoT anomaly detection sensors. We set up several in-lab experiments using three peristaltic pumps and investigated approaches for transferring trained anomaly detection models across assets of the same type. Our experiments achieved promising effectiveness and provide initial evidence that transfer learning could be a suitable strategy for using pretrained anomaly classification models across industrial assets of the same type with minimal prior labeling and training effort. This work could serve as a starting point for more general, pretrained sensor data embeddings, applicable to a wide range of assets.</div>

scholarly journals Subjects, Trials, and Levels: Statistical Power in Conjoint Experiments

2020 ◽  
Author(s):  
Alberto Stefanelli ◽  
Martin Lukac
Keyword(s):  
Power Analysis ◽  
Statistical Power ◽  
Effect Sizes ◽  
Large Sample Size ◽  
Extensive Literature ◽  
Generation Process ◽  
Data Generation ◽  
Experimental Conditions ◽  
Wide Range ◽  
Decision Making Processes

Conjoint analysis is an experimental technique that has become quite popular to understand people's decisions in multi-dimensional decision-making processes. Despite the importance of power analysis for experimental techniques, current literature has largely disregarded statistical power considerations when designing conjoint experiments. The main goal of this article is to provide researchers and practitioners with a practical tool to calculate the statistical power of conjoint experiments. To this end, we first conducted an extensive literature review to understand how conjoint experiments are designed and gauge the plausible effect sizes discovered in the literature. Second, we formulate a data generating model that is sufficiently flexible to accommodate a wide range of conjoint designs and hypothesized effects. Third, we present the results of an extensive series of simulation experiments based on the previously formulated data generation process. Our results show that---even with relatively large sample size and the number of trials---conjoint experiments are not suited to draw inferences for experiments with large numbers of experimental conditions and relatively small effect sizes. Specifically, Type S and Type M errors are especially pronounced for experimental designs with relatively small effective sample sizes (&lt; 3000) or a high number of levels (&gt; 15) that find small but statistically significant effects (&lt; 0.03). The proposed online tool based on the simulation results can be used by researchers to perform power analysis of their designs and hence achieve adequate design for future conjoint experiments.

scholarly journals Intensity Thresholding and Deep Learning Based Lane Marking Extraction and Lane Width Estimation from Mobile Light Detection and Ranging (LiDAR) Point Clouds

2020 ◽  
Vol 12 (9) ◽  
pp. 1379 ◽  
Author(s):  
Yi-Ting Cheng ◽  
Ankit Patel ◽  
Chenglu Wen ◽  
Darcy Bullock ◽  
Ayman Habib
Keyword(s):  
Deep Learning ◽  
Learning Strategies ◽  
Point Clouds ◽  
Concrete Pavement ◽  
Training Data ◽  
Light Detection And Ranging ◽  
Light Detection ◽  
Lane Marking ◽  
Wide Range ◽  
Lane Width

Lane markings are one of the essential elements of road information, which is useful for a wide range of transportation applications. Several studies have been conducted to extract lane markings through intensity thresholding of Light Detection and Ranging (LiDAR) point clouds acquired by mobile mapping systems (MMS). This paper proposes an intensity thresholding strategy using unsupervised intensity normalization and a deep learning strategy using automatically labeled training data for lane marking extraction. For comparative evaluation, original intensity thresholding and deep learning using manually established labels strategies are also implemented. A pavement surface-based assessment of lane marking extraction by the four strategies is conducted in asphalt and concrete pavement areas covered by MMS equipped with multiple LiDAR scanners. Additionally, the extracted lane markings are used for lane width estimation and reporting lane marking gaps along various highways. The normalized intensity thresholding leads to a better lane marking extraction with an F1-score of 78.9% in comparison to the original intensity thresholding with an F1-score of 72.3%. On the other hand, the deep learning model trained with automatically generated labels achieves a higher F1-score of 85.9% than the one trained on manually established labels with an F1-score of 75.1%. In concrete pavement area, the normalized intensity thresholding and both deep learning strategies obtain better lane marking extraction (i.e., lane markings along longer segments of the highway have been extracted) than the original intensity thresholding approach. For the lane width results, more estimates are observed, especially in areas with poor edge lane marking, using the two deep learning models when compared with the intensity thresholding strategies due to the higher recall rates for the former. The outcome of the proposed strategies is used to develop a framework for reporting lane marking gap regions, which can be subsequently visualized in RGB imagery to identify their cause.

Do the robot

2017 ◽  
Vol 54 (2) ◽  
pp. 193-214 ◽  
Author(s):  
Michael Colaresi ◽  
Zuhaib Mahmood
Keyword(s):  
Machine Learning ◽  
Model Development ◽  
Theory Development ◽  
Training Data ◽  
Generation Process ◽  
Data Generation ◽  
Research Strategies ◽  
Conflict Processes ◽  
Iterative Model ◽  
Out Of Sample

Increasingly, scholars interested in understanding conflict processes have turned to evaluating out-of-sample forecasts to judge and compare the usefulness of their models. Research in this vein has made significant progress in identifying and avoiding the problem of overfitting sample data. Yet there has been less research providing strategies and tools to practically improve the out-of-sample performance of existing models and connect forecasting improvement to the goal of theory development in conflict studies. In this article, we fill this void by building on lessons from machine learning research. We highlight a set of iterative tasks, which David Blei terms ‘Box’s loop’, that can be summarized as build, compute, critique, and think. While the initial steps of Box’s loop will be familiar to researchers, the underutilized process of model criticism allows researchers to iteratively learn more useful representations of the data generation process from the discrepancies between the trained model and held-out data. To benefit from iterative model criticism, we advise researchers not only to split their available data into separate training and test sets, but also sample from their training data to allow for iterative model development, as is common in machine learning applications. Since practical tools for model criticism in particular are underdeveloped, we also provide software for new visualizations that build upon already existing tools. We use models of civil war onset to provide an illustration of how our machine learning-inspired research design can simultaneously improve out-of-sample forecasting performance and identify useful theoretical contributions. We believe these research strategies can complement existing designs to accelerate innovations across conflict processes.

scholarly journals Compact single-shot metalens depth sensors inspired by eyes of jumping spiders

2019 ◽  
Vol 116 (46) ◽  
pp. 22959-22965 ◽  
Author(s):  
Qi Guo ◽  
Zhujun Shi ◽  
Yao-Wei Huang ◽  
Emma Alexander ◽  
Cheng-Wei Qiu ◽  
...  
Keyword(s):  
Depth Perception ◽  
Incident Light ◽  
Floating Point ◽  
Single Shot ◽  
Efficient Computation ◽  
Depth Sensor ◽  
Depth Sensing ◽  
Multiple Images ◽  
Jumping Spiders ◽  
Depth Sensors

Jumping spiders (Salticidae) rely on accurate depth perception for predation and navigation. They accomplish depth perception, despite their tiny brains, by using specialized optics. Each principal eye includes a multitiered retina that simultaneously receives multiple images with different amounts of defocus, and from these images, distance is decoded with relatively little computation. We introduce a compact depth sensor that is inspired by the jumping spider. It combines metalens optics, which modifies the phase of incident light at a subwavelength scale, with efficient computations to measure depth from image defocus. Instead of using a multitiered retina to transduce multiple simultaneous images, the sensor uses a metalens to split the light that passes through an aperture and concurrently form 2 differently defocused images at distinct regions of a single planar photosensor. We demonstrate a system that deploys a 3-mm-diameter metalens to measure depth over a 10-cm distance range, using fewer than 700 floating point operations per output pixel. Compared with previous passive depth sensors, our metalens depth sensor is compact, single-shot, and requires a small amount of computation. This integration of nanophotonics and efficient computation brings artificial depth sensing closer to being feasible on millimeter-scale, microwatts platforms such as microrobots and microsensor networks.

Sign in / Sign up

Export Citation Format

Share Document