BERT-Based Natural Language Processing of Drug Labeling Documents: A Case Study for Classifying Drug-Induced Liver Injury Risk

Background & Aims: The United States Food and Drug Administration (FDA) regulates a broad range of consumer products, which account for about 25% of the United States market. The FDA regulatory activities often involve producing and reading of a large number of documents, which is time consuming and labor intensive. To support regulatory science at FDA, we evaluated artificial intelligence (AI)-based natural language processing (NLP) of regulatory documents for text classification and compared deep learning-based models with a conventional keywords-based model.Methods: FDA drug labeling documents were used as a representative regulatory data source to classify drug-induced liver injury (DILI) risk by employing the state-of-the-art language model BERT. The resulting NLP-DILI classification model was statistically validated with both internal and external validation procedures and applied to the labeling data from the European Medicines Agency (EMA) for cross-agency application.Results: The NLP-DILI model developed using FDA labeling documents and evaluated by cross-validations in this study showed remarkable performance in DILI classification with a recall of 1 and a precision of 0.78. When cross-agency data were used to validate the model, the performance remained comparable, demonstrating that the model was portable across agencies. Results also suggested that the model was able to capture the semantic meanings of sentences in drug labeling.Conclusion: Deep learning-based NLP models performed well in DILI classification of drug labeling documents and learned the meanings of complex text in drug labeling. This proof-of-concept work demonstrated that using AI technologies to assist regulatory activities is a promising approach to modernize and advance regulatory science.

Labeling of Immunobiological Drugs, Produced by the Russian Research An-ti-Plague Institution "Microbe" of the Rospotrebnadzor (Review)

Introduction. This publication describes the design and implementation sequence of technological procedures for labeling immunobiological medicinal products produced by the FGHI RusRAPI "Microbe" of the Rospotrebnadzor. In light of meeting the requirements of the Federal Act "On the Circulation of Pharmaceutical Products", the materials of this study are undoubtedly relevant.Text. The paper presents a step-by-step sequence of introducing technological procedures for labeling and interaction with the system for monitoring the movement of pharmaceutical products (MMPP) into the production process of medicines. At the preparatory stage, the following main issues were addressed: verification of the identity of information about medicinal products in the State Register of Medicines and in the automatic identification system "UNISCAN/GS1 RUS"; determination of the method and possibility of applying the identification means onto the secondary packaging; amendments to the pharma-copoeial monographs of the enterprise for each type of drug. Stage 2 [development of requirements for the system of labeling, serialization, verification and aggregation (LSVAS)] included the following activities: development of a functional model of the labeling process in the FGHI RusRAPI "Microbe" and determination of the responsible for the implementation of this scheme units; determination of the method of secondary packaging (manual or automatic), as well as the required degree of aggregation and the required automation of the process, based on the analysis of the functional model and the technological process of labeling; analysis of the experience of introducing drug labeling systems; analysis of the existing IT-structure of the FGHI RusRAPI "Microbe"; monitoring of the market of hardware and software manufacturers; development of technical requirements for the created system of marking, serialization, verification and aggregation. Stage 3 (implementation of the labeling, serialization, verification and aggregation system at the production sites) included the following activities: equipment supply and commissioning; equipment qualification (IQ/OQ); training of the personnel; amendments to regulatory documents. In the materials devoted to the implementation of the final stage, the issues of validation of technological procedures for drug labeling and interaction with the system of labeling, serialization, verification and aggregation are considered.Conclusion. The works performed made it possible to produce medicines in accordance with the requirements of the Federal Act "On the Circulation of Pharmaceutical Products" and the Decree of the Government of the Russian Federation dated December 14, 2018 № 1556 "On Approval of the Regulation on the System for Monitoring the Movement of Drugs for Medical Use". The material presented may be of interest to manufacturers who produce medicines in small amounts.

DICE: A Drug Indication Classification and Encyclopedia for AI-Based Indication Extraction

Drug labeling contains an ‘INDICATIONS AND USAGE’ that provides vital information to support clinical decision making and regulatory management. Effective extraction of drug indication information from free-text based resources could facilitate drug repositioning projects and help collect real-world evidence in support of secondary use of approved medicines. To enable AI-powered language models for the extraction of drug indication information, we used manual reading and curation to develop a Drug Indication Classification and Encyclopedia (DICE) based on FDA approved human prescription drug labeling. A DICE scheme with 7,231 sentences categorized into five classes (indications, contradictions, side effects, usage instructions, and clinical observations) was developed. To further elucidate the utility of the DICE, we developed nine different AI-based classifiers for the prediction of indications based on the developed DICE to comprehensively assess their performance. We found that the transformer-based language models yielded an average MCC of 0.887, outperforming the word embedding-based Bidirectional long short-term memory (BiLSTM) models (0.862) with a 2.82% improvement on the test set. The best classifiers were also used to extract drug indication information in DrugBank and achieved a high enrichment rate (>0.930) for this task. We found that domain-specific training could provide more explainable models without performance sacrifices and better generalization for external validation datasets. Altogether, the proposed DICE could be a standard resource for the development and evaluation of task-specific AI-powered, natural language processing (NLP) models.

Information Extraction From FDA Drug Labeling to Enhance Product-Specific Guidance Assessment Using Natural Language Processing

Towards the objectives of the UnitedStates Food and Drug Administration (FDA) generic drug science and research program, it is of vital importance in developing product-specific guidances (PSGs) with recommendations that can facilitate and guide generic product development. To generate a PSG, the assessor needs to retrieve supportive information about the drug product of interest, including from the drug labeling, which contain comprehensive information about drug products and instructions to physicians on how to use the products for treatment. Currently, although there are many drug labeling data resources, none of them including those developed by the FDA (e.g., Drugs@FDA) can cover all the FDA-approved drug products. Furthermore, these resources, housed in various locations, are often in forms that are not compatible or interoperable with each other. Therefore, there is a great demand for retrieving useful information from a large number of textual documents from different data resources to support an effective PSG development. To meet the needs, we developed a Natural Language Processing (NLP) pipeline by integrating multiple disparate publicly available data resources to extract drug product information with minimal human intervention. We provided a case study for identifying food effect information to illustrate how a machine learning model is employed to achieve accurate paragraph labeling. We showed that the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model is able to outperform the traditional machine learning techniques, setting a new state-of-the-art for labelling food effect paragraphs from drug labeling and approved drug products datasets.

Characterization of Pharmacogenetic Information in Food and Drug Administration Drug Labeling and the Table of Pharmacogenetic Associations

Background: The US Food and Drug Administration (FDA) recommends using only FDA-reviewed pharmacogenetic information to make prescribing decisions based on genetic test results. Such information is available in drug labeling and in the Table of Pharmacogenetic Associations (“Associations table”). Objective: To compile a list of drug-gene pairs from drug labeling and the Associations table and categorize the pharmacogenetic information and clinical outcome associated with each drug-gene pair. Methods: This was a cross-sectional analysis of pharmacogenetic information in the Associations table and individual drug labeling in March 2020. We used the Table of Pharmacogenomic Biomarkers in Drug Labeling to identify drug labels to review. We categorized the pharmacogenetic information for each drug-gene pair according to whether the purpose was to describe (1) polymorphisms affecting drug disposition (metabolism or transport), (2) polymorphisms affecting a direct drug target, (3) variants associated with adverse drug reaction (ADR) susceptibility, (4) variants associated with therapeutic failure, (5) a biomarker-defined indication, or (6) a biomarker-defined ADR. We also categorized the clinical outcome—efficacy, safety, or unknown—associated with each drug-gene pair. We reported counts and proportions of drug-gene pairs in each pharmacogenetic information and clinical outcome category. Results: We identified 308 drug-gene pairs, of which 36% were associated with a biomarker-defined drug indication, 33% with polymorphic drug metabolism, and 28% with ADR susceptibility. Most drug-gene pairs (n = 267, 87%) were associated with an efficacy or safety-related outcome. Conclusion and Relevance: FDA-reviewed pharmacogenetic information is available for more than 300 drug-gene pairs and can help guide prescribing decisions.