Validation of a Human Pose Tracking Algorithm for Measuring Upper Limb Joints: Comparison With Photography-based Goniometry

Background: Range of motion (ROM) measurements are essential for diagnosing and evaluating upper extremity conditions. Clinical goniometry is the most commonly used methods but it is time-consuming and skill-demanding. Recent advances in human tracking algorithm suggest potential for automatic angle measuring from RGB images. It provides an attractive alternative for at-distance measuring. However, the reliability of this method has not been fully established. The purpose of this study is to evaluate if the results of algorithm are as reliable as human raters in upper limb movements.Methods: Thirty healthy young adults (20 males, 10 females) participated in this study. Participants were asked to performed a 6-motion task including movement of shoulder, elbow and wrist. Images of movements were capture by commercial digital camera. Each movement was measured by a pose tracking algorithm and compared with the surgeon-measurement results. The mean differences between the two measurements were compared. Pearson correlation coefficients were used to determine the relationship. Reliability was investigated by the intra-class correlation coefficients.Results: Comparing this algorithm-based method with manual measurement, the mean differences were less than 3 degrees in 5 motions (shoulder abduction: 0.51; shoulder elevation: 2.87; elbow flexion:0.38; elbow extension:0.65; wrist extension: 0.78) except wrist flexion. All the intra-class correlation coefficients were larger than 0.60. The Pearson coefficients also showed high correlations between the two measurements (p<0.001). Conclusions: Our results indicated that pose estimation is a reliable method to measure the shoulder and elbow angles, supporting RGB images for measuring joint ROM. Our results proved the possibility that patients can assess their ROM by photos taken by a digital camera.Trial registration: This study was registered in the Clinical Trials Center of The First Affiliated Hospital, Sun Yat-sen University (2021-387).

GeoAI: a review of Artificial Intelligence approaches for the interpretation of complex Geomatics data

Researchers have explored the benefits and applications of modern artificial intelligence (AI) algorithms in different scenario. For the processing of geomatics data, AI offers overwhelming opportunities. Fundamental questions include how AI can be specifically applied to or must be specifically created for geomatics data. This change is also having a significant impact on geospatial data. The integration of AI approaches in geomatics has developed into the concept of Geospatial Artificial Intelligence (GeoAI), which is a new paradigm for geographic knowledge discovery and beyond. However, little systematic work currently exists on how researchers have applied AI for geospatial domains. Hence, this contribution outlines AI-based techniques for analysing and interpreting complex geomatics data. Our analysis has covered several gaps, for instance defining relationships between AI-based approaches and geomatics data. First, technologies and tools used for data acquisition are outlined, with a particular focus on RGB images, thermal images, 3D point clouds, trajectories, and hyperspectral/multispectral images. Then, how AI approaches have been exploited for the interpretation of geomatic data is explained. Finally, a broad set of examples of applications are given, together with the specific method applied. Limitations point towards unexplored areas for future investigations, serving as useful guidelines for future research directions.

A Robust Method for Detecting Wind-Fallen Stems from Aerial RGB Images Using a Line Segment Detection Algorithm

Increased frequencies and windspeeds of storms may cause disproportionately high increases in windthrow damage. Storm-felled trees provide a surplus of breeding material for bark beetles, often resulting in calamities in the subsequent years. Thus, the timely removal of fallen trees is regarded as a good management practice that requires strategic planning of salvage harvesting. Precise information on the number of stems and their location and orientation are needed for the efficient planning of strip roads and/or cable yarding lines. An accurate assessment of these data using conventional field-based methods is very difficult and time-consuming; remote sensing techniques may be a cost-efficient alternative. In this research, a methodology for the automatic detection of fallen stems from aerial RGB images is presented. The presented methodology was based on a line segment detection algorithm and proved to be robust regarding image quality. It was shown that the method can detect frequency, position, spatial distribution and orientation of fallen stems with high accuracy, while stem lengths were systematically underestimated. The methodology can be used for the optimized planning of salvage harvesting in the future and may thus help to reduce consequential bark beetle calamities after storm events.

Improved Mask R-CNN for Rural Building Roof Type Recognition from UAV High-Resolution Images: A Case Study in Hunan Province, China

Accurate roof information of buildings can be obtained from UAV high-resolution images. The large-scale accurate recognition of roof types (such as gabled, flat, hipped, complex and mono-pitched roofs) of rural buildings is crucial for rural planning and construction. At present, most UAV high-resolution optical images only have red, green and blue (RGB) band information, which aggravates the problems of inter-class similarity and intra-class variability of image features. Furthermore, the different roof types of rural buildings are complex, spatially scattered, and easily covered by vegetation, which in turn leads to the low accuracy of roof type identification by existing methods. In response to the above problems, this paper proposes a method for identifying roof types of complex rural buildings based on visible high-resolution remote sensing images from UAVs. First, the fusion of deep learning networks with different visual features is investigated to analyze the effect of the different feature combinations of the visible difference vegetation index (VDVI) and Sobel edge detection features and UAV visible images on model recognition of rural building roof types. Secondly, an improved Mask R-CNN model is proposed to learn more complex features of different types of images of building roofs by using the ResNet152 feature extraction network with migration learning. After we obtained roof type recognition results in two test areas, we evaluated the accuracy of the results using the confusion matrix and obtained the following conclusions: (1) the model with RGB images incorporating Sobel edge detection features has the highest accuracy and enables the model to recognize more and more accurately the roof types of different morphological rural buildings, and the model recognition accuracy (Kappa coefficient (KC)) compared to that of RGB images is on average improved by 0.115; (2) compared with the original Mask R-CNN, U-Net, DeeplabV3 and PSPNet deep learning models, the improved Mask R-CNN model has the highest accuracy in recognizing the roof types of rural buildings, with F1-score, KC and OA averaging 0.777, 0.821 and 0.905, respectively. The method can obtain clear and accurate profiles and types of rural building roofs, and can be extended for green roof suitability evaluation, rooftop solar potential assessment, and other building roof surveys, management and planning.

Comparison of Multi-Methods for Identifying Maize Phenology Using PhenoCams

Accurately identifying the phenology of summer maize is crucial for both cultivar breeding and fertilizer controlling in precision agriculture. In this study, daily RGB images covering the entire growth of summer maize were collected using phenocams at sites in Shangqiu (2018, 2019 and 2020) and Nanpi (2020) in China. Four phenological dates, including six leaves, booting, heading and maturity of summer maize, were pre-defined and extracted from the phenocam-based images. The spectral indices, textural indices and integrated spectral and textural indices were calculated using the improved adaptive feature-weighting method. The double logistic function, harmonic analysis of time series, Savitzky–Golay and spline interpolation were applied to filter these indices and pre-defined phenology was identified and compared with the ground observations. The results show that the DLF achieved the highest accuracy, with the coefficient of determination (R2) and the root-mean-square error (RMSE) being 0.86 and 9.32 days, respectively. The new index performed better than the single usage of spectral and textural indices, of which the R2 and RMSE were 0.92 and 9.38 days, respectively. The phenological extraction using the new index and double logistic function based on the PhenoCam data was effective and convenient, obtaining high accuracy. Therefore, it is recommended the adoption of the new index by integrating the spectral and textural indices for extracting maize phenology using PhenoCam data.

Crossmodality Person Reidentification Based on Global and Local Alignment

RGB-infrared (RGB-IR) person reidentification is a challenge problem in computer vision due to the large crossmodality difference between RGB and IR images. Most traditional methods only carry out feature alignment, which ignores the uniqueness of modality differences and is difficult to eliminate the huge differences between RGB and IR. In this paper, a novel AGF network is proposed for RGB-IR re-ID task, which is based on the idea of global and local alignment. The AGF network distinguishes pedestrians in different modalities globally by combining pixel alignment and feature alignment and highlights more structure information of person locally by weighting channels with SE-ResNet-50, which has achieved ideal results. It consists of three modules, including alignGAN module ( A ), crossmodality paired-images generation module ( G ), and feature alignment module ( F ). First, at pixel level, the RGB images are converted into IR images through the pixel alignment strategy to directly reduce the crossmodality difference between RGB and IR images. Second, at feature level, crossmodality paired images are generated by exchanging the modality-specific features of RGB and IR images to perform global set-level and fine-grained instance-level alignment. Finally, the SE-ResNet-50 network is used to replace the commonly used ResNet-50 network. By automatically learning the importance of different channel features, it strengthens the ability of the network to extract more fine-grained structural information of person crossmodalities. Extensive experimental results conducted on SYSU-MM01 dataset demonstrate that the proposed method favorably outperforms state-of-the-art methods. In addition, we evaluate the performance of the proposed method on a stronger baseline, and the evaluation results show that a RGB-IR re-ID method will show better performance on a stronger baseline.

Vision-based Pakistani Sign Language Recognition Using Bag-of-Words and Support Vector Machines

In order to perform their daily activities, a person is required to communicating with others. This can be a major obstacle for the deaf population of the world, who communicate using sign languages (SL). Pakistani Sign Language (PSL) is used by more than 250,000 deaf Pakistanis. Developing a SL recognition system would greatly facilitate these people. This study aimed to collect data of static and dynamic PSL alphabets and to develop a vision-based system for their recognition using Bag-of-Words (BoW) and Support Vector Machine (SVM) techniques. A total of 5,120 images for 36 static PSL alphabet signs and 353 videos with 45,224 frames for 3 dynamic PSL alphabet signs were collected from 10 native signers of PSL. The developed system used the collected data as input, resized the data to various scales and converted the RGB images into grayscale. The resized grayscale images were segmented using Thresholding technique and features were extracted using Speeded Up Robust Feature (SURF). The obtained SURF descriptors were clustered using K-means clustering. A BoW was obtained by computing the Euclidean distance between the SURF descriptors and the clustered data. The codebooks were divided into training and testing using 5-fold cross validation. The highest overall classification accuracy for static PSL signs was 97.80% at 750×750 image dimensions and 500 Bags. For dynamic PSL signs a 96.53% accuracy was obtained at 480×270 video resolution and 200 Bags.

Digital image processing methods for estimating leaf area of cucumber plants

Increasingly <span>emerging technologies in agriculture such as computer vision, artificial intelligence technology, not only make it possible to increase production. To minimize the negative impact on climate and the environment but also to conserve resources. A key task of these technologies is to monitor the growth of plants online with a high accuracy rate and in non-destructive manners. It is known that leaf area (LA) is one of the most important growth indexes in plant growth monitoring system. Unfortunately, to estimate the LA in natural outdoor scenes (the presence of occlusion or overlap area) with a high accuracy rate is not easy and it still remains a big challenge in eco-physiological studies. In this paper, two accurate and non-destructive approaches for estimating the LA were proposed with top-view and side-view images, respectively. The proposed approaches successfully extract the skeleton of cucumber plants in red, green, and blue (RGB) images and estimate the LA of cucumber plants with high precision. The results were validated by comparing with manual measurements. The experimental results of our proposed algorithms achieve 97.64% accuracy in leaf segmentation, and the relative error in LA estimation varies from 3.76% to 13.00%, which could meet the requirements of plant growth monitoring </span>systems.

Classification of Crop Residue Cover in High-Resolution RGB Images Using Machine Learning

HighlightsA machine learning framework estimated residue cover in RGB images taken at three resolutions from 88 locations.The best results primarily used texture features, the RFE-SVM feature selection method, and the SVM classifier.Accounting for shadows and plants plus modifying and optimizing the texture features may improve performance.An automated system developed using machine learning is a viable strategy to estimate residue cover from RGB images obtained with handheld or UAV platforms.Abstract. Maintaining plant residue on the soil surface contributes to sustainable cultivation of arable land. Applying machine learning methods to RGB images of residue could overcome the subjectivity of manual methods. The objectives of this study were to use supervised machine learning while identifying the best feature selection method, the best classifier, and the most effective image feature types for classifying residue levels in RGB imagery. Imagery was collected from 88 locations in 40 row-crop fields in five Missouri counties between early May and late June in 2018 and 2019 using a tripod-mounted camera (0.014 cm pixel-1 ground sampling distance, GSD) and an unmanned aerial vehicle (UAV, 0.05 and 0.14 GSD). At each field location, 50 contiguous 0.3 × 0.2 m region of interest (ROI) images were extracted from the imagery, resulting in a dataset of 4,400 ROI images at each GSD. Residue percentages for ground truth were estimated using a bullseye grid method (n = 100 points) based on the 0.014 GSD images. Representative color, texture, and shape features were extracted and evaluated using four feature selection methods and two classifiers. Recursive feature elimination using support vector machine (RFE-SVM) was the best feature selection method, and the SVM classifier performed best for classifying the amount of residue as a three-class problem. The best features for this application were associated with texture, with local binary pattern (LBP) features being the most prevalent for all three GSDs. Shape features were irrelevant. The three residue classes were correctly identified with 88%, 84%, and 81% 10-fold cross-validation scores for the 2018 training data and 81%, 69%, and 65% accuracy for the 2019 testing data in decreasing resolution order. Converting image-wise data (0.014 GSD) to location residue estimates using a Bayesian model showed good agreement with the location-based ground truth (r2 = 0.90). This initial assessment documents the use of RGB images to match other methods of estimating residue, with potential to replace or be used as a quality control for line-transect assessments. Keywords: Feature selection, Soil erosion, Support vector machine, Texture features, Unmanned aerial vehicle.