Improving machine learning performance on small chemical reaction data with unsupervised contrastive pretraining

Machine learning (ML) methods have great potential to transform chemical discovery by accelerating the exploration of chemical space and drawing scientific insights from data. However, modern chemical reaction ML models, such as those based on graph neural networks (GNNs), must be trained on a large amount of labelled data in order to avoid overfitting the data and thus possessing low accuracy and transferability. In this work, we propose a strategy to leverage unlabelled data to learn accurate ML models for small labelled chemical reaction data. We focus on an old and prominent problem—classifying reactions into distinct families—and build a GNN model for this task. We first pretrain the model on unlabelled reaction data using unsupervised contrastive learning and then fine-tune it on a small number of labelled reactions. The contrastive pretraining learns by making the representations of two augmented versions of a reaction similar to each other but distinct from other reactions. We propose chemically consistent reaction augmentation methods that protect the reaction center and find they are the key for the model to extract relevant information from unlabelled data to aid the reaction classification task. The transfer learned model outperforms a supervised model trained from scratch by a large margin. Further, it consistently performs better than models based on traditional rule-driven reaction fingerprints, which have long been the default choice for small datasets. In addition to reaction classification, the effectiveness of the strategy is tested on regression datasets; the learned GNN-based reaction fingerprints can also be used to navigate the chemical reaction space, which we demonstrate by querying for similar reactions. The strategy can be readily applied to other predictive reaction problems to uncover the power of unlabelled data for learning better models with a limited supply of labels.

Advanced breakup-nucleon enhancement of deuteron-induced reaction cross sections

Following the EUROfusion PPPT-programme action for an advanced modeling approach of deuteron-induced reaction cross sections, as well as specific data evaluations in addition of the TENDL files, an assessment of the details and corresponding outcome for the latter option of TALYS for the breakup model has been carried out. The breakup enhancement obtained in the meantime within computer code TALYS, by using the evaluated nucleon-induced reaction data of TENDL-2019, is particularly concerned. Discussion of the corresponding results, for deuteron-induced reactions on $$^{58}$$ 58 Ni, $$^{96}$$ 96 Zr, and $$^{231}$$ 231 Pa target nuclei up to 200 MeV incident energy, includes limitations still existing with reference to the direct-reaction account.

Improving machine learning performance on small chemical reaction data with unsupervised contrastive pretraining

Machine learning (ML) methods have great potential to transform chemical discovery by accelerating the exploration of chemical space and drawing scientific insights from data. However, modern chemical reaction ML models,...

Thermal behavior of Sr-form of gordaite

Cation-exchanged Sr-form of gordaite was successfully obtained from Ca-form of gordaite by an ion-exchange reaction. Data of XRD, SEM-EDS and DTA-TG-MS were used to characterize the Sr-form. Thermal decomposition of Sr-gordaite was studied for the first time in regards of thermal events and mass loss during volatile releasing. It was found similarity with Sr-gordaite and Ca-gordaite in terms of processes, type, and amount of volatiles released, but also some differences were found concerning the temperature correspondence of the volatiles evolving and the type of thermal decomposition products. The influence of the exchangeable cations (Na, Ca, or Sr) on the dehydration of the interlayer in the gordaite type structure were also established.

Improving machine learning performance on small chemical reaction data with unsupervised contrastive pretraining

Machine learning (ML) methods have great potential to transform chemical discovery by accelerating the exploration of chemical space and drawing scientific insights from data. However, modern chemical reaction ML models, such as those based on graph neural networks (GNNs), must be trained on a large amount of labelled data in order to avoid overfitting the data and thus possessing low accuracy and transferability. In this work, we propose a strategy to leverage unlabelled data to learn accurate ML models for small labelled chemical reaction data. We focus on an old and prominent problem—classifying reactions into distinct families—and build a GNN model for this task. We first pretrain the model on unlabelled reaction data using unsupervised contrastive learning and then fine-tune it on a small number of labelled reactions. The contrastive pretraining learns by making the representations of two augmented versions of a reaction similar to each other but distinct from other reactions. We propose chemically consistent reaction augmentation methods that protect the reaction center and find they are the key for the model to extract relevant information from unlabelled data to aid the reaction classification task. The transfer learned model outperforms a supervised model trained from scratch by a large margin. Further, it consistently performs better than models based on traditional rule-driven reaction fingerprints, which have long been the default choice for small datasets. In addition to reaction classification, the learned GNN-based reaction fingerprints can also be used to navigate the chemical reaction space, which we demonstrate by querying for similar reactions. The strategy can be readily applied to other predictive reaction problems to uncover the power of unlabelled data for learning better models with a limited supply of labels.

Prediction of the chemical context for Buchwald-Hartwig coupling reactions

We present machine learning models for predicting the chemical context for Buchwald-Hartwig coupling reactions. Using reaction data from in-house electronic lab notebooks, we train two models: one based on single-label data and one based on multi-label data. Both models show excellent top-3 accuracy around 90%, which suggests strong predictivity. There seems to be an advantage of including multi-label data because the multi-label model shows higher accuracy and better sensitivity for the individual contexts than the single-label model. Although the models are performant, we also show that such models need to be re-trained periodically. There is a strong temporal characteristic to the usage of different contexts. Therefore, a model trained on historical data will decrease in usefulness with time as newer and better contexts emerge and replace older ones. We hypothesize that these significant transitions in the context-use will likely affect any model predicting chemical contexts trained on historical data. Consequently, training such models warrants careful planning of what data is used for training and how often the model needs to be re-trained.

Coupled reaction channels study of the ¹⁶O(d,⁶Li) reaction

The transfer 16O(d,6Li)12C reaction has been studied within the coupled reaction channels (CRC) approach, inluding both the direct and indirect α transfer processes. The obtained results show an important contribution of the indirect α transfer via the 2+ and 4+ states of 12C. The CRC results show that the best-fit α spectroscopic factors of 16O becomes smaller when the indirect transfer processes are taken into account. The α spectroscopic factors deduced from the present CRC analysis of the 16O(d,6Li)12C reaction data measured at Ed=54.25 and 80 MeV are quite close to each other.