Ensuring the validity of the results of calibrations and tests by statistical methods

According to the requirements of ISO/IEC 17025:2017, the validity of test and calibration results is ensured, inter alia, by intralaboratory check of the results obtained. In this case, it is preferable to use statistical methods. The ISO 5725 standards define a number of such methods, but the choice of specific methods is left to the laboratory, taking into account the requirements for the adequacy of the effort, resources and time for the purposes of the work performed and the risks of obtaining inappropriate results. In this case, the laboratory itself must in a certain way determine which objects of calibrations (tests) should be predominantly used in checks and what frequency of checks should be foreseen. In connection with the increase in the accuracy and complexity of measuring systems, the need to apply the methods of the theory of random processes becomes more and more obvious. It is shown that the use of the Poincaré plot makes it possible to comprehensively, effectively and visually evaluate changes in the measuring process from the point of view of the dynamics of the obtained measurement results. The results of the check, in particular, the intermediate precision, make it possible to obtain a more realistic evaluation of measurement uncertainty in accordance with ISO 21748. The paper analyses some practical approaches (of varying degrees of complexity) to intralaboratory checks of the validity of calibration (test) results.

Three-Dimensional Cephalometric Landmarking and Frankfort Horizontal Plane Construction: Reproducibility of Conventional and Novel Landmarks

In some dentofacial deformity patients, especially patients undergoing surgical orthodontic treatments, Computed Tomography (CT) scans are useful to assess complex asymmetry or to plan orthognathic surgery. This assessment would be made easier for orthodontists and surgeons with a three-dimensional (3D) cephalometric analysis, which would require the localization of landmarks and the construction of reference planes. The objectives of this study were to assess manual landmarking repeatability and reproducibility (R&R) of a set of 3D landmarks and to evaluate R&R of vertical cephalometric measurements using two Frankfort Horizontal (FH) planes as references for horizontal 3D imaging reorientation. Thirty-three landmarks, divided into “conventional”, “foraminal” and “dental”, were manually located twice by three experienced operators on 20 randomly-selected CT scans of orthognathic surgery patients. R&R confidence intervals (CI) of each landmark in the -x, -y and -z directions were computed according to the ISO 5725 standard. These landmarks were then used to construct 2 FH planes: a conventional FH plane (orbitale left, porion right and left) and a newly proposed FH plane (midinternal acoustic foramen, orbitale right and left). R&R of vertical cephalometric measurements were computed using these 2 FH planes as horizontal references for CT reorientation. Landmarks showing a 95% CI of repeatability and/or reproducibility > 2 mm were found exclusively in the “conventional” landmarks group. Vertical measurements showed excellent R&R (95% CI < 1 mm) with either FH plane as horizontal reference. However, the 2 FH planes were not found to be parallel (absolute angular difference of 2.41°, SD 1.27°). Overall, “dental” and “foraminal” landmarks were more reliable than the “conventional” landmarks. Despite the poor reliability of the landmarks orbitale and porion, the construction of the conventional FH plane provided a reliable horizontal reference for 3D craniofacial CT scan reorientation.

Introduction to Statistical Methods for Outlier Detection and Sample Homogeneity Assessment of Reference Materials and Proficiency Test Items

Due to the homogeneity of the product or sample, it will affect whether it meets the scope of application and purpose. For example, the reference materials(RM) produced by the reference material producer(RMP), and the proficiency test items selected by the proficiency testing provider(PTP), in order to ensure the reference materials or proficiency test items have consistent characteristics or comparability, they should be proved to have certain homogeneity. However, before performing homogeneity assessment, it is necessary to measure the characteristic parameters of the reference materials or proficiency test items to obtain a sufficient number of measured values for data analysis, but there may be outliers in the measured values that may affect data analysis and interpretation of the results. Therefore, this article will refer to ASTM E178-16a:2016[1], ISO 5725-2:1994[2], ISO 13528:2015[3], etc., to introduce several outlier detection and homogeneity assessment methods, supplemented by case studies. Finally, this article will remind the precautions for the use of the method, so that readers can choose the appropriate method for use in the actual analysis.

Reproducibility of three-dimensional landmarking and Frankfort horizontal plane construction: a comparison of conventional and novel landmarks in presurgical computed tomography scans

Objectives To assess manual landmarking repeatability and reproducibility (R&R) of a set of three-dimensional (3D) landmarks and to evaluate R&R of vertical cephalometric measurements using two Frankfort Horizontal (FH) planes as references for horizontal 3D imaging reorientation. Methods Thirty-three landmarks, divided into "conventional", "foraminal" and "dental", were manually located twice by 3 experienced operators on 20 computed tomography (CT) scans of orthognathic surgery patients. R&R of the landmark localization were computed according to the ISO 5725 standard. These landmarks were then used to construct 2 FH planes: a conventional FH plane (orbitale left, porion right and left) and a newly proposed FH plane (midinternal acoustic foramen, orbitale right and left). R&R of vertical cephalometric measurements were computed using these 2 FH planes as horizontal references for CT reorientation. Results Landmarks showing a 95% confidence interval (CI) of repeatability and/or reproducibility > 2mm were found exclusively in the "conventional" landmarks group. Vertical measurements showed excellent R&R (95% CI < 1mm) with either FH plane as horizontal reference. However, the 2 FH planes were not found to be parallel (absolute angular difference of 2.41°, SD 1.27°). The average time needed to landmark one CT scan was 14 ± 3 minutes. Conclusions The "dental" and "foraminal" landmarks tended to be more reliable than the "conventional" landmarks. Despite the poor overall reliability of the landmarks orbitale and porion, the construction of the conventional FH plane using 3 landmarks provided a reliable horizontal reference for 3D craniofacial CT scan reorientation.

The accuracy and reliability of comparative efficiency assessment of portable fire extinguishers used in the process of fire extinguishing of structural components of motor vehicles and automotive finishing

Introduction. The relevance of this work resides in the need to perform an unbiased comparative assessment of fire extinguishers. The focus must be placed on the efficiency of a fire-fighting agent, as it has a major impact on the process of fire extinguishing. The purpose of this work is to confirm the applicability of a dimensionless efficiency indicator of portable fire extinguishers, assess the measurement accuracy (correctness and precision) and their statistical values.Materials and methods. The co-authors used the methodology specified in GOST R 51057–2001 applicable to the fire extinguishing of class A model fire seats. The model fire seat had samples of materials attached to its upper edge at the angles whose values were obtained using a specialized test bed. A specialized test bed was used to perform the testing and identify the sample’s inclination angles that prevented the fire extinguishing agent, applied to their surface, from streaming down or crumbling. The accuracy (correctness and precision) was assessed pursuant to the methodologies specified in GOST R ISO 5725-1–2002, GOST R ISO 5725-6–2002. The statistical significance of measurements was assessed using Student’s t-test.Results and discussion. The co-authors offer the results of the experiments aimed at the identification of an inclination angle of the fire seat surface that prevents the fire extinguishing agent from streaming down or crumbling. The results of comparative fire testing of fire extinguishers that contain quick setting foam and powder are provided. The formula of a dimensionless efficiency indicator, applicable to portable fire extinguishers, is provided, and its applicability is proven.Conclusions. The measurement accuracy assessment, performed pursuant to the methodologies, specified in the regulations, has shown the effectiveness of mean arithmetic values of all indicators applied to calculate Peo efficiency indicator. The assessment of the statistical significance of experimentally obtained values of all indicators, performed with the help of Student’s t-test, has demonstrated that the discrepancy of their mean arithmetic values is reliable and it doesn’t have a random nature. The superiority of the quick setting foam extinguisher over the powder one was proven in an experiment. The value of its Peo efficiency indicator exceeds the one of a powder extinguisher 50-fold due to the comprehensive impact made by the extinguishing agent and the construction of a fire extinguisher.

Bivariate confidence probability plots as a method to test the accuracy and variability of microbiological measures

IntroductionIn an interlaboratory calibration analysis to validate a methodology that will be proposed as a European standard for domestic laundry disinfection, tests were carried out to detect if there are different behaviors in the measurements regarding accuracies and variabilities. Interlaboratory tests using different doses of disinfectant and microorganisms were carried out. ISO 5725-2 and ISO 13528 form the basis of validations of quantitative methods, providing validation specifications for interlaboratory studies. However, a need for a simple graphical method to detect interlaboratory differences in accuracy and variability was observed.ObjectivesThe general goal of this work is to present a new exploratory methodology, graphical and easy to interpret, that can determine the accuracy and variability (precision) of a variable, and compare it to the methodology applied in ISO 5725-2 and ISO 13528.MethodsWe used confidence probability plots of the multivariate Student’s t-distribution to observe the accuracy and variability of microbiological measures carried out by different laboratories during a ring trial exercise. A function in R was built for this purpose: Miriam.analysis.ellipse(Y, factor_a, eel.plot = “ t-Student”). The different observations of accuracy and variability are represented in the ellipses. If any of the points are outside the ellipse with 95% confidence, we can assume a deviation in accuracy and / or variability.ResultsTwo examples are provided with real microbiological data (logarithmic unit reductions (LR) for Pseudomonas aeruginosa, Escherichia coli, Staphilococcus aureus, Enterococcus hirae, Candida albicans and microbial counts in water (WW)). The proposed new method allowed us to detect possible deviations in the WWMEA variable and we believe it has future application for the rapid control of microbiological measures.

Selección de un método de aprendizaje automático para clasificar patrones biomarcadores de lesiones precancerosas de las cuerdas vocales

The National Survey on Drug, Alcohol and Tobacco, 2016-2017, notes that 15.6 million Mexicans are active smokers and, by 2030, expect the death of 8 million cancers of the larynx or lung. Therefore, the World Health Organization (WHO) recommends detecting precancerous lesions of the larynx. This is possible, as they are characterized by a biomarker pattern manifested by the alteration of the biomechanical interpretation of the vocal cords, regardless of the sex and age of the smoker. The goal of this article is to evaluate three machine learning methods: neural networks, Gaussian networks, and decision tree to determine the method that best solves the problem of detecting patterns of precancerous vocal cord injury biomarkers. It uses the WEKA tool and a knowledge bank, endorsed by NOM-012-SSA3-2012, with 250 patterns, and provided by the Luis Guillermo Ibarra National Institute of Rehabilitation, Ibarra. The performance of the methods was compared by ROC curves and confusion matrices, under the criteria established by ISO-5725. The decision tree the method that best responds to the detection of patterns of biomarkers of precancerous lesions of the vocal cords.

Selective Electromagnetic Measurements of 4G Signals: Results of an Italian National Intercomparison

In June 2016, with the aim of ensuring a global improvement in the performance of the Italian System of the Environmental Agencies (SNPA) and its homogeneity on the national territory, an intercomparison circuit (IC) was planned and conducted concerning the measurements of electromagnetic fields associated with Long Term Evolution (LTE) mobile communications, which were very recently introduced at that time. The intercomparison circuit, designed and built according to the criteria of ISO 17043, was organized as part of a consolidated collaboration between the Institute for Environmental Protection and Research (ISPRA) and the Piedmont Regional Agency for Environmental Protection (Arpa Piemonte). The results obtained, preceded by a brief description of the entire process of organization and analysis, are the subject of this work. The IC covered in particular: the narrow band measurement procedures used in the field; the choice of decoding, measurement and extrapolation of the synthesis result; the response of the instrumentation, limited to the models in the field. The site chosen by the organizers, primarily characterized through measurements and theoretical evaluation of the field, is the roof of the Lingotto Building in Turin. A total of 27 groups participated in the circuit: 25 SNPA departments (including the organizers Arpa Piemonte and ISPRA) and 2 private labs. All participants provided the results. The outcome of the comparison was decidedly positive: only 2 participants, for whom a joint assessment of possible causes will also be illustrated (according to ISO 5725:2), achieved significantly different results.

Rapid Automated System for the Detection of Objectionable Microorganisms in Nutraceutical and Dietary Supplements

Background: The U.S. Pharmacopeia (USP) established microbiological procedures to evaluate nonsterile products and nutritional and dietary supplements for objectionable microorganisms of concern in the industry. A rapid method has been developed to detect the presence of microorganisms based on real-time photometric monitoring of pH, carbon dioxide, or fluorescence indicators within a novel broth/agar combination vial and accompanying reader. Objective: To validate the performance of the Neogen Rapid Microbiology System (NRMS) method against the USP reference method for the ability to detect presence or the absence of Escherichia coli, Salmonella, Staphylococcus aureus, and Pseudomonas aeruginosa in nutraceutical and dietary supplements. Methods: Over 200 samples were evaluated for the presence of objectionable microorganisms. For each specified microorganism, a minimum of 23 matrices were tested uninoculated and with a low-level inoculum of &lt;1–27.4 CFU/g, as paired samples, by NRMS and USP methods. Results: In the four objectionable microorganism categories evaluated, 100% inclusivity, 97–100% exclusivity, 97–100% positive predictivity, 100% negative predictivity, and Kappa Indices of 0.97–1.0 were achieved following ISO 5725-6:1994 standards. Conclusions: The validation study demonstrates that NRMS is a rapid and effective method to detect objectionable microorganisms in a variety of nutraceutical and dietary supplements. Highlights: A novel method was validated against the relevant USP reference method for the detection of Escherichia coli, Salmonella, Staphylococcus aureus, and Pseudomonas aeruginosa in nutraceutical and dietary supplements. NRMS provided results statistically comparable to the USP reference method within 22 h following primary enrichment compared with 5–7 days by the reference method.