A two-step computer vision-based framework for bolt loosening detection and its implementation on a smartphone application

Bolt loosening detection is a labor-intensive and time-consuming process for field engineers. This paper develops a two-step computer vision-based framework to quickly identify bolt loosening angle from field images captured by unmanned aerial vehicle (UAV). In step one, a total of 1200 image samples of bolted structures were used to train faster region based convolutional neural network (Faster R-CNN) for bolt detection from UAV captured images. In step two, computer vision-based technologies, including Gaussian filter, perspective transform, and Hough transform (HT), were performed to quantify bolt loosening angle. The developed framework was then integrated into web server and an iOS application (app) was designed to enable fast data communication between field workplace (UAV captured images) and web server (bolt loosening angle quantification), so that field engineers can quickly view the inspection results on their phone screens. The proposed framework and designed smartphone app greatly help field engineers to improve the accuracy and efficiency for onsite inspection and maintenance of bolted structures.

Reliability Updating of Offshore Structures Subjected to Marine Growth

Marine growth is a known problem for oceanic infrastructure and has been shown to negatively impact the reliability of bottom-fixed or floating offshore structures submitted to fatigue or extreme loading. Among other effects, it has been shown to change drag forces by increasing member diameters and modifying the roughness. Bio-colonization being highly random, the objective of this paper is to show how one-site inspection data increases reliability by decreasing uncertainties. This can be introduced in a reliability-based inspection framework for optimizing inspection and maintenance (here, cleaning). The modeling and computation are illustrated through the reliability analysis of a monopile in the European Atlantic area subjected to marine growth and according to the plastic collapse limit state. Based on surveys of structures in the North Sea, long-term stochastic modeling (space and time) of the marine growth thickness is first suggested. A Dynamic Bayesian Network is then developed for reliability updating from the inspection data. Finally, several realistic (10–20 measurements) inspection strategies are compared in terms of reliability improvement and the accuracy of reliability assessment.

Design and Experimental Evaluation of an Aerial Solution for Visual Inspection of Tunnel-like Infrastructures

Current railway tunnel inspections rely on expert operators performing a visual examination of the entire infrastructure and manually annotating encountered defects. Automatizing the inspection and maintenance task of such critical and aging infrastructures has the potential to decrease the associated costs and risks. Contributing to this aim, the present work describes an aerial robotic solution designed to perform autonomous inspections of tunnel-like infrastructures. The proposed robotic system is equipped with visual and thermal sensors and uses an inspection-driven path planning algorithm to generate a path that maximizes the quality of the gathered data in terms of photogrammetry goals while optimizing the surface coverage and the total trajectory length. The performance of the planning algorithm is demonstrated in simulation against state-of-the-art methods and a wall-following inspection trajectory. Results of a real inspection test conducted in a railway tunnel are also presented, validating the whole system operation.

Autonomous UAV System for Cleaning Insulators in Power Line Inspection and Maintenance

The inspection and maintenance tasks of electrical installations are very demanding. Nowadays, insulator cleaning is carried out manually by operators using scaffolds, ropes, or even helicopters. However, these operations involve potential risks for humans and the electrical structure. The use of Unmanned Aerial Vehicles (UAV) to reduce the risk of these tasks is rising. This paper presents an UAV to autonomously clean insulators on power lines. First, an insulator detection and tracking algorithm has been implemented to control the UAV in operation. Second, a cleaning tool has been designed consisting of a pump, a tank, and an arm to direct the flow of cleaning liquid. Third, a vision system has been developed that is capable of detecting soiled areas using a semantic segmentation neuronal network, calculating the trajectory for cleaning in the image plane, and generating arm trajectories to efficiently clean the insulator. Fourth, an autonomous system has been developed to land on a charging pad to charge the batteries and potentially fill the tank with cleaning liquid. Finally, the autonomous system has been validated in a controlled outdoor environment.

Intelligent Digital Twins and Augmented Reality in Inspection and Maintenance

Objective/Scope Simplify and semi-automate the creation of Intelligent Digital Twins available on an augmented hybrid solution (on/offline) based on the latest mobile solutions and wearable technologies (Smart Glasses) communicating over Bluetooth. The main objective was to implement an interactive environment of digital twins through Augmented Reality (AR) to improve decision-making, enhance performance and optimize efficiencies in Inspection and Maintenance. Methods, Procedures, Process A breakthrough algorithm enabled to semi-automate the creation of 3D models by providing a platform of Intelligent Digital Twins to connect to various disparate systems and use Artificial Intelligence-driven augmentation techniques to contextualize and enrich the model with Tag numbers from the asset register. An application has been developed for Smart Glasses by transforming the Digital Twins into sectioned files to enable a performant AR experience and an on/offline syncing functionality with the mobile solution over Bluetooth. Furthermore, the 3D model was embedded within the mobile application for visual support during Inspection and Maintenance. Results, Observations, Conclusions As a result, the following products have been developed to improve efficiency and productivity by leveraging innovative technologies: Application for Intelligent Digital Twins: Single Source of Truth for managing and maintaining Intelligent Digital Twins using breakthrough technology to semi-automate the creation of 3D models from Point Cloud data. A Mark-Up Tool and a Contextualization Editor were developed on the promise of Digital Twins to introduce a communication, collaboration, and planning tool. Furthermore, the 3D model has been embedded within the mobile application environment for visual support on the field tablets. Application for Smart Glasses (AR Headset): An application has been developed for the latest AR Headset to enable an Augmented Reality experience and a Mark-Up tool with Bluetooth communication to the mobile application as well as to support a hybrid on/offline syncing functionality as well as data capturing on the fly. With its basic functionality of hands-free video streaming and communication functionality, the AR Headset enabled a Remote Pre-Commissioning, Certification, and Audit with vendors and independent certifiers during the pandemic. Application Programming Interfaces (API): API interfaces were established to various source systems like the Computerized Maintenance Management System, Inspection Data Management System, Drawing System, and the Process Information System to integrate and visualize the big data within the platform of Intelligent Digital Twins. Novel/Additive Information The use of automation and artificial intelligence (AI) driven technologies for creating Intelligent Digital Twins is a key enabler for global scalability and accelerates creating the company's digital backbone during the digital transformation. Enhancing this technology with an Augmented Reality experience, including mark-up annotations, provides a solid basis for data insights, data-driven decision making, and performance optimization.

DAMAGE INSPECTION ON PIER OF SG. PERAK RESERVOIR BRIDGE USING ACCELEROMETER AND GNSS

Sg. Perak Reservoir Bridge where it is commonly known to be heavily utilized by heavy vehicles on daily basis. Therefore, the practices of inspection and maintenance of this bridge are essential tasks in prolonging its lifespan especially as it is utilized heavily with heavy vehicles. The act of inspecting damage that occurred at a pier is done through the use of technical equipment as an additional method invalidating visual inspection obtained through vibration reading, and validating in inspection reading analysis as additional information to confirm for any structural damages. The use of using Global Navigation Satellite System (GNSS) can be employed for continuous monitoring and the use of accelerometers, which are components of the ambient vibration method, served to be an integral part of the information obtained on the changes in the dynamic structural features detected. The vibration measurement can display natural frequencies that depended on the weight, material, pressure, and tension as well as the geometry of the object. This data obtained can therefore be used to furnish additional information on the capacity and condition of the structure. The results indicated that maximum vibration on two piers inspection are recorded at 53.7 mm/s2 and 49.6 mm/s2 using an Accelerometer indeed heavy vehicles traffic flow is a factor in influencing bridge vibration by traffic transport diversity between west and east lanes of both sides of the bridge.

Reliability of quantitative risk analysis through an industrial case study

PurposeThe purpose of this paper is to analyze the reliability of the quantitative risk model used for planning inspection and maintenance activities. The objective is to critically discuss the factors that contribute to the probability and consequence of failure calculations.Design/methodology/approachThe case study conducted using one of the most widely deployed risk models in the oil and gas industry where a full assessment was performed on an offshore gas producing platform.FindingsThe generic failure frequencies used as the basis for calculating the probability of failure are set at a value representative of the refining and petrochemical industry's failure data. This failure database does not cover offshore. The critical discussion indicated the lack of basis of the coefficient of variances, prior probabilities and conditional probabilities. Moreover, the risk model does not address the distribution of thickness measurements, corrosion rates and inspection effectiveness, whereas only overall deterministic values are used; this requires judgment to determine these values. Probabilities of ignition, probabilities of delayed ignition and other probabilities in Level 1 event tree are found selected based on expert judgment for each of the reference fluids and release types (i.e. continuous or instantaneous). These probabilities are constant and independent of the release rate or mass and lack of constructed model. Defining the release type is critical in the consequence of the failure methodology, whereas the calculated consequences differ greatly depending on the type of release, i.e. continuous or instantaneous. The assessment results show that both criteria of defining the type of release, i.e. continuous or instantaneous, do not affect the calculations of flammable consequences when the auto-ignition likely is zero at the storage temperature. While, the difference in the resulted toxic consequence was more than 31 times between the two criteria of defining the type of release.Research limitations/implicationsThere is a need to revamp this quantitative risk model to minimize the subjectivity in the risk calculation and to address the unique design features of offshore platforms.Originality/valueThis case study critically discuss the risk model being widely applied in the O&G industry and demonstrates to the end-users the subjectivity in the risk results. Hence, be vigilant when establishing the risk tolerance/target for the purpose of inspection and maintenance planning.

Recognition of Power Equipment Based on Multitask Sparse Representation

Image analysis of power equipment has important practical significance for power-line inspection and maintenance. This paper proposes an image recognition method for power equipment based on multitask sparse representation. In the feature extraction stage, based on the two-dimensional (2D) random projection algorithm, multiple projection matrices are constructed to obtain the multilevel features of the image. In the classification process, considering that the image acquisition process will inevitably be affected by factors such as light conditions and noise interference, the proposed method uses the multitask compressive sensing algorithm (MtCS) to jointly represent multiple feature vectors to improve the accuracy and robustness of reconstruction. In the experiment, the images of three types of typical power equipment of insulators, transformers, and circuit breakers are classified. The correct recognition rate of the proposed method reaches 94.32%. In addition, the proposed method can maintain strong robustness under the conditions of noise interference and partial occlusion, which further verifies its effectiveness.