Ancestral sequence reconstructions evaluated by extant sequence cross-validation

Ancestral sequence reconstruction (ASR) has become widely used to analyze the properties of ancient biomolecules and to elucidate the mechanisms of molecular evolution. By recapitulating the structural, mechanistic, and functional changes of proteins during their evolution, ASR has been able to address many fundamental and challenging evolutionary questions where more traditional methods have failed. Despite the tangible successes of ASR, the accuracy of its reconstructions is currently unknown, because it is generally impossible to compare resurrected proteins to the true ancient ancestors that are now extinct. Which evolutionary models are the best for ASR? How accurate are the resulting inferences? Here we answer these questions by applying cross-validation (CV) to sets of aligned extant sequences. To assess the adequacy of a chosen evolutionary model for predicting extant sequence data, our column-wise CV method iteratively cross-validates each column in an alignment. Unlike other phylogenetic model selection criteria, this method does not require bias correction and does not make restrictive assumptions commonly violated by phylogenetic data. We find that column-wise CV generally provides a more conservative criterion than the AIC by preferring less complex models. To validate ASR methods, we also apply cross-validation to each sequence in an alignment by reconstructing the extant sequences using ASR methodology, a method we term extant sequence reconstruction (ESR). We can thus quantify the accuracy of ASR methodology by comparing ESR reconstructions to the corresponding true sequences. We find that a common measure of the quality of a reconstructed sequence, the average probability of the sequence, is indeed a good estimate of the fraction of the sequence that is correct when the evolutionary model is accurate or overparameterized. However, the average probability is a poor measure for comparing reconstructions, because more accurate phylogenetic models typically result in reconstructions with lower average probabilities. In contrast, the entropy of the reconstructed distribution is a reliable indicator of the quality of a reconstruction, as the entropy provides an accurate estimate of the log-probability of the true sequence. Both column-wise CV and ESR are useful methods to validate evolutionary models used for ASR and can be applied in practice to any phylogenetic analysis of real biological sequences.

Using deep learning to predict the outcome of live birth from more than 10,000 embryo data

Background Recently, the combination of deep learning and time-lapse imaging provides an objective, standard and scientific solution for embryo selection. However, the reported studies were based on blastocyst formation or clinical pregnancy as the end point. To the best of our knowledge, there is no predictive model that uses the outcome of live birth as the predictive end point. Can a deep learning model predict the probability of live birth from time-lapse system? Methods This study retrospectively analyzed the time-lapse data and live birth outcomes of embryos samples from January 2018 to November 2019. We used the SGD optimizer with an initial learning rate of 0.025 and cosine learning rate reduction strategy. The network is randomly initialized and trained for 200 epochs from scratch. The model is quantitively evaluated over a hold-out test and a 5-fold cross-validation by the average area under the curve (AUC) of the receiver operating characteristic (ROC) curve. Results The deep learning model was able to predict live birth outcomes from time-lapse images with an AUC of 0.968 in 5-fold stratified cross-validation. Conclusions This research reported a deep learning model that predicts the live birth outcome of a single blastocyst transfer. This efficient model for predicting the outcome of live births can automatically analyze the time-lapse images of the patient’s embryos without the need for manual embryo annotation and evaluation, and then give a live birth prediction score for each embryo, and sort the embryos by the predicted value.

Assimilation of GOSAT Methane in the Hemispheric CMAQ; Part II: Results Using Optimal Error Statistics

We applied the parametric variance Kalman filter (PvKF) data assimilation designed in Part I of this two-part paper to GOSAT methane observations with the hemispheric version of CMAQ to obtain the methane field (i.e., optimized analysis) with its error variance. Although the Kalman filter computes error covariances, the optimality depends on how these covariances reflect the true error statistics. To achieve more accurate representation, we optimize the global variance parameters, including correlation length scales and observation errors, based on a cross-validation cost function. The model and the initial error are then estimated according to the normalized variance matching diagnostic, also to maintain a stable analysis error variance over time. The assimilation results in April 2010 are validated against independent surface and aircraft observations. The statistics of the comparison of the model and analysis show a meaningful improvement against all four types of available observations. Having the advantage of continuous assimilation, we showed that the analysis also aims at pursuing the temporal variation of independent measurements, as opposed to the model. Finally, the performance of the PvKF assimilation in capturing the spatial structure of bias and uncertainty reduction across the Northern Hemisphere is examined, indicating the capability of analysis in addressing those biases originated, whether from inaccurate emissions or modelling error.

Data-Enhancement Strategies in Weather-Related Health Studies

Although the relationship between weather and health is widely studied, there are still gaps in this knowledge. The present paper proposes data transformation as a way to address these gaps and discusses four different strategies designed to study particular aspects of a weather–health relationship, including (i) temporally aggregating the series, (ii) decomposing the different time scales of the data by empirical model decomposition, (iii) disaggregating the exposure series by considering the whole daily temperature curve as a single function, and (iv) considering the whole year of data as a single, continuous function. These four strategies allow studying non-conventional aspects of the mortality-temperature relationship by retrieving non-dominant time scale from data and allow to study the impact of the time of occurrence of particular event. A real-world case study of temperature-related cardiovascular mortality in the city of Montreal, Canada illustrates that these strategies can shed new lights on the relationship and outlines their strengths and weaknesses. A cross-validation comparison shows that the flexibility of functional regression used in strategies (iii) and (iv) allows a good fit of temperature-related mortality. These strategies can help understanding more accurately climate-related health.

Generalization Challenges in Drug-Resistant Tuberculosis Detection from Chest X-rays

Classification of drug-resistant tuberculosis (DR-TB) and drug-sensitive tuberculosis (DS-TB) from chest radiographs remains an open problem. Our previous cross validation performance on publicly available chest X-ray (CXR) data combined with image augmentation, the addition of synthetically generated and publicly available images achieved a performance of 85% AUC with a deep convolutional neural network (CNN). However, when we evaluated the CNN model trained to classify DR-TB and DS-TB on unseen data, significant performance degradation was observed (65% AUC). Hence, in this paper, we investigate the generalizability of our models on images from a held out country’s dataset. We explore the extent of the problem and the possible reasons behind the lack of good generalization. A comparison of radiologist-annotated lesion locations in the lung and the trained model’s localization of areas of interest, using GradCAM, did not show much overlap. Using the same network architecture, a multi-country classifier was able to identify the country of origin of the X-ray with high accuracy (86%), suggesting that image acquisition differences and the distribution of non-pathological and non-anatomical aspects of the images are affecting the generalization and localization of the drug resistance classification model as well. When CXR images were severely corrupted, the performance on the validation set was still better than 60% AUC. The model overfitted to the data from countries in the cross validation set but did not generalize to the held out country. Finally, we applied a multi-task based approach that uses prior TB lesions location information to guide the classifier network to focus its attention on improving the generalization performance on the held out set from another country to 68% AUC.

BIOTAS: BIOTelemetry Analysis Software, for the semi-automated removal of false positives from radio telemetry data

Introduction Radio telemetry, one of the most widely used techniques for tracking wildlife and fisheries populations, has a false-positive problem. Bias from false-positive detections can affect many important derived metrics, such as home range estimation, site occupation, survival, and migration timing. False-positive removal processes have relied upon simple filters and personal opinion. To overcome these shortcomings, we have developed BIOTAS (BIOTelemetry Analysis Software) to assist with false-positive identification, removal, and data management for large-scale radio telemetry projects. Methods BIOTAS uses a naïve Bayes classifier to identify and remove false-positive detections from radio telemetry data. The semi-supervised classifier uses spurious detections from unknown tags and study tags as training data. We tested BIOTAS on four scenarios: wide-band receiver with a single Yagi antenna, wide-band receiver that switched between two Yagi antennas, wide-band receiver with a single dipole antenna, and single-band receiver that switched between five frequencies. BIOTAS has a built in a k-fold cross-validation and assesses model quality with sensitivity, specificity, positive and negative predictive value, false-positive rate, and precision-recall area under the curve. BIOTAS also assesses concordance with a traditional consecutive detection filter using Cohen’s $$\kappa$$ κ . Results Overall BIOTAS performed equally well in all scenarios and was able to discriminate between known false-positive detections and valid study tag detections with low false-positive rates (< 0.001) as determined through cross-validation, even as receivers switched between antennas and frequencies. BIOTAS classified between 94 and 99% of study tag detections as valid. Conclusion As part of a robust data management plan, BIOTAS is able to discriminate between detections from study tags and known false positives. BIOTAS works with multiple manufacturers and accounts for receivers that switch between antennas and frequencies. BIOTAS provides the framework for transparent, objective, and repeatable telemetry projects for wildlife conservation surveys, and increases the efficiency of data processing.

Automatic Classification of the Ripeness Stage of Mango Fruit Using Machine a Learning Approach

Most mango farms classify the maturity stage manually by trained workers using external indicators such as size, shape, and skin color, which can lead to human error or inconsistencies. We developed four common machine learning (ML) classifiers, the k-mean, naïve Bayes, support vector machine, and feed-forward artificial neural network (FANN), all of which were aimed at classifying the ripeness stage of mangoes at harvest. The ML classifiers were trained on biochemical data and then tested on physical and electrical data.The performance of the ML models was compared using fourfold cross validation. The FANN classifier performed the best, with a mean accuracy of 89.6% for unripe, ripe, and overripe classes, when compared to the other classifiers.

A prediction model for recurrence after translabyrinthine surgery for vestibular schwannoma: toward personalized postoperative surveillance

Purpose The aim of this study is to compute and validate a statistical predictive model for the risk of recurrence, defined as regrowth of tumor necessitating salvage treatment, after translabyrinthine removal of vestibular schwannomas to individualize postoperative surveillance. Methods The multivariable predictive model for risk of recurrence was based on retrospectively collected patient data between 1995 and 2017 at a tertiary referral center. To assess for internal validity of the prediction model tenfold cross-validation was performed. A ‘low’ calculated risk of recurrence in this study was set at < 1%, based on clinical criteria and expert opinion. Results A total of 596 patients with 33 recurrences (5.5%) were included for analysis. The final prediction model consisted of the predictors ‘age at time of surgery’, ‘preoperative tumor growth’ and ‘first postoperative MRI outcome’. The area under the receiver operating curve of the prediction model was 89%, with a C-index of 0.686 (95% CI 0.614–0.796) after cross-validation. The predicted probability for risk of recurrence was low (< 1%) in 373 patients (63%). The earliest recurrence in these low-risk patients was detected at 46 months after surgery. Conclusion This study presents a well-performing prediction model for the risk of recurrence after translabyrinthine surgery for vestibular schwannoma. The prediction model can be used to tailor the postoperative surveillance to the estimated risk of recurrence of individual patients. It seems that especially in patients with an estimated low risk of recurrence, the interval between the first and second postoperative MRI can be safely prolonged.

Model averaging with the hybrid model: An asymptotic study and demonstration

In this paper, we present a new model averaging technique that can be applied in medical research. The dataset is first partitioned by the values of its categorical explanatory variables. Then for each partition, a model average is determined by minimising some form of squared errors, which could be the leave-one-out cross-validation errors. From our asymptotic optimality study and the results of simulations, we demonstrate under several high-level assumptions and modelling conditions that this model averaging procedure may outperform jackknife model averaging, which is a well-established technique. We also present an example where a cross-validation procedure does not work (that is, a zero-valued cross-validation error is obtained) when determining the weights for model averaging.