ROC Analysis: Assessment of Diagnostic Accuracy in the Laboratory

1991 ◽  
Vol 22 (10) ◽  
pp. 708-711 ◽  
Author(s):  
Mark H. Zweig
Keyword(s):  
Diagnostic Accuracy ◽  
Roc Analysis

The potential value of a pictorial atlas for aid in the visual diagnosis of 123I FP-CIT SPECT scans

2009 ◽  
Vol 48 (04) ◽  
pp. 173-178 ◽  
Author(s):  
H. Ham ◽  
A. Dobbeleir ◽  
P. Santens ◽  
Y. D'Asseler ◽  
I. Goethals
Keyword(s):  
Diagnostic Accuracy ◽  
Roc Analysis ◽  
Operating Characteristic ◽  
Interobserver Variability ◽  
Area Under The Curve ◽  
Visual Assessment ◽  
Added Value ◽  
Related Disorder ◽  
Visual Diagnosis ◽  
Spect Images

SummaryThe aim of our study was to evaluate the value of a pictorial atlas of 123I FP-CIT SPECT images for aid in the visual diagnosis. Patients, materials, methods: Sixty patients, of whom 20 were clinically diagnosed as ‘non-parkinsonian’ and 40 as having Parkinson's disease or any related disorder, were included in the study. An atlas consisting of 12 123I FP-CIT SPECT images was constructed first. Validity of the atlas was investigated by performing a receiver operating characteristic (ROC) analysis with the clinical diagnosis as the gold standard. The remaining 48 SPECT images were visually assessed twice by 5 observers, first with and secondly without consulting the atlas, or vice versa. The added value of the atlas was investigated by comparing the diagnostic accuracy and the interobserver variability for both methods. Results: ROC analysis performed on the atlas yielded an area under the curve of 1 for a threshold discriminating between clinically non-parkinsonian and parkinsonian patients that was situated between image 4 and 5 of the atlas. For the diagnostic accuracy, we found that the area under the ROC curve was systematically higher if observers had access to the atlas compared to when they had not (Wilcoxon's test, p<0.05). Also, the interobserver variability was significantly lower when observers used the atlas when compared to when they did not (p = 0.05). Conclusion: Diagnostic accuracy was significantly higher and interobserver variability significantly lower if observers had access to the atlas compared to when they had not. Hence, having a pictorial atlas available may facilitate the visual assessment of 123I FP-CIT SPECT scans.

scholarly journals T1 and ECV mapping texture analysis distinguishing hypertrophic cardiomyopathy from athletes heart better than median values

2021 ◽  
Vol 22 (Supplement_2) ◽  
Author(s):  
T Dresselaers ◽  
P Rafouli-Stergiou ◽  
R De Bosscher ◽  
S Tilborghs ◽  
C Dausin ◽  
...  
Keyword(s):  
Diagnostic Accuracy ◽  
Texture Analysis ◽  
Roc Analysis ◽  
Left Ventricular ◽  
Athlete’S Heart ◽  
Support Vector ◽  
Mapping Data ◽  
Intensive Training ◽  
Native T1 ◽  
Athlete's Heart

Abstract Funding Acknowledgements Type of funding sources: Public grant(s) – National budget only. Main funding source(s): Ph.D fellowship of the Research Foundation Flanders (FWO). The Master@Heart trial is funded by the FWO. Introduction Differentiating intensive training induced hypertrophy from hyperthropic cardiomyopathy (HCM) is important to identify those young athletes at risk of sudden cardiac death. Swoboda and colleagues demonstrated that T1 and ECV mapping can aid such a differentiation between athletic and pathological hypertrophy, particularly in subjects with indeterminate wall thickness (1). Recently texture analysis (TA) methods of CMR data have demonstrated improved diagnostic accuracy over conventional qualitative analysis in various heart diseases. Only few studies have applied TA to T1 and ECV mapping data (2-4). Here we aimed to demonstrate that a TA approach provides superior capacity to distinguish HCM from athlete’s heart over average native T1 and ECV values. Purpose It was our hypothesis that a texture analysis of T1 and ECV mapping images would identify features that could discriminate between a HCM and athlete’s heart with a higher classification accuracy (CA) than average T1 and ECV values. Methods This study included data from 97 subjects diagnosed with HCM (acc. to guidelines; 5) and 28 athletes that took part in the Master@Heart trial (an ongoing study assessing the beneficial effects of long-term endurance exercise for the prevention of coronary artery disease, 6).  Long and short axis T1 mapping data was acquired on a 1.5T Philips Ingenia system using MOLLI (seconds scheme). After offline motion correction and T1 and ECV map calculation (7), the left ventricular myocardium was manually delineated (3D Slicer; 8). Texture analysis of the masked images resulted in 194 features (Pyradiomics, standard settings; 9). The dataset was then split (75/25%) for training and testing purposes keeping images from the same subject within the same set. A fast correlation based filter rank was applied to the training data to derive relevant features. A further reduction to only two features was based on the CA of a support vector machine (SVM) learning method (linear kernel; cost 0.9 regression loss epsilon 0.1; leave-one-out). Finally, ROC analysis on the test data was used to determine the diagnostic accuracy for the following predictors: (1) median T1 and ECV (2) two most relevant features (training) (3) combination of (1) and (2) (ROC AUC statistics (10)). Results The two most relevant features were the histogram feature ECV energy and the gray level size zone matrix (GLSZM) feature native T1 zone entropy, a measure of heterogeneity in the texture pattern. A model to distinguish HCM from athletes based on these features outperformed the model using only median T1 and ECV values with both higher sensitivity and specificity (table 1) and a significantly  higher AUC in the ROC analysis (p &lt; 0.05, figure 1). Combining these two features with median values did not improve the CA further.  Conclusion Texture analysis of motion-corrected T1 and ECV mapping images out-performs classical analysis based on average values in distinguishing HCM from athlete"s heart.

scholarly journals Evaluating the diagnostic accuracy of gene xpert analysis in pediatric pulmonary tuberculosis: a prospective clinical study in North-Western India

2021 ◽  
Vol 8 (4) ◽  
pp. 716
Author(s):  
Venkatesh Karthik S. ◽  
Jigisha Patadiya
Keyword(s):  
Diagnostic Accuracy ◽  
Predictive Value ◽  
Roc Analysis ◽  
Gastric Lavage ◽  
Control Program ◽  
Tuberculin Test ◽  
Receiver Operating Curve ◽  
Prospective Clinical Study ◽  
Western India ◽  
X Rays

Background: The diagnosis of tuberculosis (TB) by microbiological tests is a major challenge particularly in children. The use of Xpert analysis, a rapid genetic testing modality is not widely reported in our locality. The aim of the study to evaluate the diagnostic accuracy of Gene xpert analysis in diagnosis of pediatric Pulmonary TB.Methods: A prospective hospital-based study was conducted among 140 participants with symptomatology pertaining to pulmonary TB as per Revised national tuberculosis control program (RNTCP, India) criteria. The Xpert testing (GXT) was performed as per standards and was compared with erythrocyte sedimentation rate (ESR), tuberculin test (TT) and chest X-rays (CXR). The obtained results were reported in terms of Sensitivity %, Specificity %, Positive Predictive Value % (PPV) and Negative Predictive Value % (NPV) for comparisons. The receiver operating curve (ROC) analysis was employed to evaluate the accuracy of diagnosis.Results: The GXT was positive (10.71 %) in suspected TB patients. TT has significantly (10 %) with a73.33% sensitivity, 93.60% specificity and a PPV of 57.89 % when compared with xpert. The ESR showed a sensitivity of 53.33% and a specificity of 56%. The CXR showed sensitivity of 93.33%. The ROC analysis showed that TT had a higher confidence interval (0.699-0.970) t5`han other methods. The Rifampicin resistance was found 7.5% (n=2) of 15 GXT positive cases.Conclusions: The xpert based diagnosis of gastric lavage samples after a tuberculin test (TT) had high sensitivity and specificity, followed by chest X ray while the ESR had lower clinical accuracy. The ‘gene xpert analysis’ is highly useful rapid tool for diagnosis of children with TB. 

Clinical Utility of Striational Antibodies in Paraneoplastic and Myasthenia Gravis Paraneoplastic Panels

Neurology ◽  
2021 ◽  
pp. 10.1212/WNL.0000000000012050
Author(s):  
Shahar Shelly ◽  
John R Mills ◽  
Divyansu Dubey ◽  
Andrew McKeon ◽  
Anastasia Zekeridou ◽  
...  
Keyword(s):  
Diagnostic Accuracy ◽  
Myasthenia Gravis ◽  
Clinical Utility ◽  
Roc Analysis ◽  
Thymic Cancer ◽  
Diagnostic Characteristics ◽  
Data Platform ◽  
Neurological Phenotype ◽  
Antibody Positivity

Objective:To critically assess the clinical utility of striational antibodies (StrAbs) within paraneoplastic and myasthenia gravis serological evaluations.Methods:All Mayo Clinic patients tested for StrAbs from January 1st 2012-December 31st 2018 utilizing Mayo’s Unified Data Platform (UDP) were reviewed for neurological diagnosis and cancer.Results:38,502 unique paraneoplastic and 1,899 MG patients were tested. In paraneoplastic evaluations, the StrAbs positivity rate was higher in cancer vs without cancer (5% [321/6775] vs 4% [1154/31727]; p<0.0001; OR 1.35; CI=1.19-1.53) but ROC analysis indicated no diagnostic accuracy in cancer (AUC=0.505). No neurological phenotype was significantly associated with StrAbs in the paraneoplastic group. Positivity was more common in all MG cancers compared to paraneoplastic cancers (p<0.0001). In MG evaluations, the StrAbs positivity rate was higher in those with cancer vs without (46% [217/474] vs 26% [372/1425]; p<0.0001; OR 2.39, CI 1.9-2.96) with ROC analysis indicating poor diagnostic accuracy for thymic cancer (AUC 0.634, recommended cutoff=1:60, sensitivity=56%, specificity=71%), with worse accuracy for extrathymic cancers (AUC 0.543). In paraneoplastic or MG evaluations, the value of antibody positivity did not improve cancer predictions. Paraneoplastic evaluated patients were more likely with positive StrAbs to obtain computed tomography (CT) (p=0.0001) with 3% (12/468) cancer found.Conclusion:Despite a statistically significant association with cancer, an expansive review of performance in clinical service demonstrates that StrAbs are neither specific nor sensitive in predicting malignancy or neurological phenotypes. CT imaging is over utilized with positive StrAbs results. Removal of StrAbs from paraneoplastic or MG evaluations will improve the diagnostic characteristics of the current MG test.Classification of Evidence:This study provides Class II evidence that the presence of StrAbs do not accurately identify patients with malignancy or neurological phenotypes.

scholarly journals Cut-off limits of the peak GH response to stimulation tests for the diagnosis of GH deficiency in children and adolescents: study in patients with organic GHD

2016 ◽  
Vol 175 (1) ◽  
pp. 41-47 ◽  
Author(s):  
Chiara Guzzetti ◽  
Anastasia Ibba ◽  
Sabrina Pilia ◽  
Nadia Beltrami ◽  
Natascia Di Iorgi ◽  
...  
Keyword(s):  
Diagnostic Accuracy ◽  
Children And Adolescents ◽  
Roc Analysis ◽  
Operating Characteristic ◽  
Gh Deficiency ◽  
Tolerance Test ◽  
Gh Secretion ◽  
Insulin Tolerance ◽  
Igf I ◽  
Stimulation Tests

ObjectiveThe diagnosis of GH deficiency (GHD) in children and adolescents is established when GH concentrations fail to reach an arbitrary cut-off level after at least two provocative tests. The objective of the study was to define the optimal GH cut-offs to provocative tests in children and adolescents.DesignRetrospective study in 372 subjects who underwent evaluation of GH secretion. GH and IGF-I were measured by chemiluminescence assay in all samples. Receiver operating characteristic (ROC) analysis was used to evaluate the optimal GH cut-offs and the diagnostic accuracy of provocative tests.MethodsSeventy four patients with organic GHD (GH peak <10μg/L after two provocative tests) and 298 control subjects (GH response >10μg/L to at least one test) were included in the study. The provocative tests used were arginine, insulin tolerance test (ITT) and clonidine. Diagnostic criteria based on cut-offs identified by ROC analysis (best pair of values for sensitivity and specificity) were evaluated for each test individually and for each test combined with IGF-I SDS.ResultsThe optimal GH cut-off for arginine resulted 6.5μg/L, 5.1μg/L for ITT and 6.8μg/L for clonidine. IGF-I SDS has low accuracy in diagnosing GHD (AUC=0.85). The combination of the results of provocative tests with IGF-I concentrations increased the specificity.ConclusionsThe results of the ROC analysis showed that the cut-off limits which discriminate between normal and GHD are lower than those commonly employed. IGF-I is characterized by low diagnostic accuracy.

scholarly journals 215 Assessment of the Diagnostic Sensitivity and Specificity of Pain Biomarkers in Cattle Using Receiver Operating Characteristic Curves

2020 ◽  
Vol 98 (Supplement_3) ◽  
pp. 7-8
Author(s):  
Miriam S Martin ◽  
Michael Kleinhenz ◽  
Karen Schwartzkopf-Genswein ◽  
Johann Coetzee
Keyword(s):  
Diagnostic Accuracy ◽  
Receiver Operating Characteristic ◽  
Sensitivity And Specificity ◽  
Predictive Value ◽  
Roc Analysis ◽  
Operating Characteristic ◽  
Diagnostic Sensitivity ◽  
Diagnostic Sensitivity And Specificity ◽  
Good Diagnostic Accuracy ◽  
Receiver Operating

Abstract Biomarkers are commonly used to assess pain and analgesic drug efficacy in livestock. However, the diagnostic sensitivity and specificity of these biomarkers for different pain conditions over time have not been described. Receiver operating characteristic (ROC) curves are graphical plots that illustrate the diagnostic ability of a test as its discrimination threshold is varied. The objective of this analysis was to use area under the curve (AUC) values derived from ROC analysis to assess the predictive value of pain biomarkers at specific timepoints. The biomarkers included in the analysis were blood cortisol, salivary cortisol, hair cortisol, infrared thermography (IRT), mechanical nociceptive threshold (MNT), substance P, and outcomes from a pressure/force measurement system and visual analog scale. A total sample size of 7,992 biomarker outcomes were collected from 6 pain studies involving pain associated with castration, dehorning, lameness, and surgery were included in the analysis. Each study consisted of three treatments; pain, no pain, and analgesia. All statistics were performed using statistical software (JMP Pro 14.0, SAS Institute, Inc., Cary, NC). Results comparing analgesia verses pain yielded good diagnostic accuracy (AUC &gt; 0.7; 95% CI: 0.40 to 0.99) for blood cortisol (timepoints 1.5, 2, and 6 hours); IRT (timepoints 6, 8, 12, and 72 hours); and MNT (timepoints 6, 25, and 49 hours). These results indicate that ROC analysis can be a useful indicator of the predictive value of pain biomarkers and certain timepoints seem to yield good diagnostic accuracy while many do not.

811 WHETHER 10 WATER SWALLOWS NECESSARY TO OBTAIN REPRODUCIBLE RESULTS OF HIGH-RESOLUTION OESOPHAGEAL MANOMETRY?

2021 ◽  
Vol 34 (Supplement_1) ◽  
Author(s):  
Sergey Morozov ◽  
Vasily Kropochev ◽  
Alexey Artemov
Keyword(s):  
High Resolution ◽  
Diagnostic Accuracy ◽  
False Positive ◽  
Roc Analysis ◽  
Characteristic Curve ◽  
False Negative ◽  
Ratio Test ◽  
Type I ◽  
Primary Position ◽  
Oesophageal Manometry

Abstract   Not less than ten wet swallows assessment in the primary test position is recommended by Chicago classification 4.0 for high-resolution oesophageal manometry (HREM); however, the required number of measurements are not sufficiently supported. Aim to evaluate the number of wet swallows necessary for correct interpretation of the results of lower esophageal sphincter integrated relaxation pressure (IRP) with low probability of type I and II errors. Methods Patients referred to perform HREM were enrolled. Solid-state 10Fr catheter and Solar (Laborie) software were used. Minimum 10 swallows by 5 mL water were obtained. These were analysed for cumulative means of IRP after 1…9 measurements. Conclusion made at each moment was compared with one based on 10 measurements. The results were characterized as true/false positive/negative for calculation of diagnostic accuracy. To exclude sample influence, Monte-Carlo simulation of sequential decision-making was performed with the use of sequential probability ratio test. Association of the diagnostic accuracy from recall was studied with the use of receiver operating characteristic curve (ROC) analysis. Results One hundred subjects were enrolled (25 with disorders of EGJ outflow). During the simulation, the probability of matching the decisions based on the 10 measurements and lower number of them was high. ROC analysis showed that actual probability to obtain false-positive results was twice as lower then ‘allowed’ rate of 5%. The probability to make false-negative results did not exceed 10% in any number of measurements. The probability that the conclusions made after 2 and after 10 measurements match was 0.9584 in those with disorders of EGJ outflow and 0.9652 in those without (figure 1). Conclusion The standard number of measurements required to support the presence of disorders of EGJ outflow during evaluation of 5 mL wet swallows in the primary position is excessive. Values of the IRP after 2 swallows allows to make similar decision to that after 10 swallows with &gt;95% probability. This allows to reduce the number of wet swallows to assess in the primary position and save time for assessments in alternative position or perform provocation tests.

Sign in / Sign up

Export Citation Format

Share Document