Tri-training for Dependency Parsing Domain Adaptation

2022 ◽  
Vol 21 (3) ◽  
pp. 1-17
Author(s):  
Shu Jiang ◽  
Zuchao Li ◽  
Hai Zhao ◽  
Bao-Liang Lu ◽  
Rui Wang
Keyword(s):  
Transfer Learning ◽  
High Performance ◽  
Domain Adaptation ◽  
Language Model ◽  
Training Methods ◽  
Dependency Parsing ◽  
Cross Domain ◽  
Cross Lingual ◽  
Domain Transfer ◽  
Domain Transfer Learning

In recent years, the research on dependency parsing focuses on improving the accuracy of the domain-specific (in-domain) test datasets and has made remarkable progress. However, there are innumerable scenarios in the real world that are not covered by the dataset, namely, the out-of-domain dataset. As a result, parsers that perform well on the in-domain data usually suffer from significant performance degradation on the out-of-domain data. Therefore, to adapt the existing in-domain parsers with high performance to a new domain scenario, cross-domain transfer learning methods are essential to solve the domain problem in parsing. This paper examines two scenarios for cross-domain transfer learning: semi-supervised and unsupervised cross-domain transfer learning. Specifically, we adopt a pre-trained language model BERT for training on the source domain (in-domain) data at the subword level and introduce self-training methods varied from tri-training for these two scenarios. The evaluation results on the NLPCC-2019 shared task and universal dependency parsing task indicate the effectiveness of the adopted approaches on cross-domain transfer learning and show the potential of self-learning to cross-lingual transfer learning.

scholarly journals Ontology-Driven Cross-Domain Transfer Learning

2020 ◽  
Author(s):  
Mattia Fumagalli ◽  
Gábor Bella ◽  
Samuele Conti ◽  
Fausto Giunchiglia
Keyword(s):  
Learning Community ◽  
Transfer Learning ◽  
Domain Adaptation ◽  
Classification Performance ◽  
Knowledge Domains ◽  
Learning Framework ◽  
Cross Domain ◽  
First Results ◽  
Domain Transfer ◽  
Domain Transfer Learning

The aim of transfer learning is to reuse learnt knowledge across different contexts. In the particular case of cross-domain transfer (also known as domain adaptation), reuse happens across different but related knowledge domains. While there have been promising first results in combining learning with symbolic knowledge to improve cross-domain transfer results, the singular ability of ontologies for providing classificatory knowledge has not been fully exploited so far by the machine learning community. We show that ontologies, if properly designed, are able to support transfer learning by improving generalization and discrimination across classes. We propose an architecture based on direct attribute prediction for combining ontologies with a transfer learning framework, as well as an ontology-based solution for cross-domain generalization based on the integration of top-level and domain ontologies. We validate the solution on an experiment over an image classification task, demonstrating the system’s improved classification performance.

scholarly journals Automatic Microsurgical Skill Assessment Based on Cross-Domain Transfer Learning

2020 ◽  
Vol 5 (3) ◽  
pp. 4148-4155
Author(s):  
Dandan Zhang ◽  
Zicong Wu ◽  
Junhong Chen ◽  
Anzhu Gao ◽  
Xu Chen ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Skill Assessment ◽  
Cross Domain ◽  
Domain Transfer ◽  
Domain Transfer Learning

scholarly journals Damage detection using in-domain and cross-domain transfer learning

2021 ◽  
Author(s):  
Zaharah A. Bukhsh ◽  
Nils Jansen ◽  
Aaqib Saeed
Keyword(s):  
Damage Detection ◽  
Learning Strategies ◽  
Transfer Learning ◽  
Large Scale ◽  
Visual Inspection ◽  
Superior Performance ◽  
Cross Domain ◽  
Domain Transfer ◽  
Domain Transfer Learning ◽  
The Impact

AbstractWe investigate the capabilities of transfer learning in the area of structural health monitoring. In particular, we are interested in damage detection for concrete structures. Typical image datasets for such problems are relatively small, calling for the transfer of learned representation from a related large-scale dataset. Past efforts of damage detection using images have mainly considered cross-domain transfer learning approaches using pre-trained ImageNet models that are subsequently fine-tuned for the target task. However, there are rising concerns about the generalizability of ImageNet representations for specific target domains, such as for visual inspection and medical imaging. We, therefore, evaluate a combination of in-domain and cross-domain transfer learning strategies for damage detection in bridges. We perform comprehensive comparisons to study the impact of cross-domain and in-domain transfer, with various initialization strategies, using six publicly available visual inspection datasets. The pre-trained models are also evaluated for their ability to cope with the extremely low-data regime. We show that the combination of cross-domain and in-domain transfer persistently shows superior performance specially with tiny datasets. Likewise, we also provide visual explanations of predictive models to enable algorithmic transparency and provide insights to experts about the intrinsic decision logic of typically black-box deep models.

scholarly journals Deep Transfer Learning for Biology Cross-Domain Image Classification

2021 ◽  
Vol 2021 ◽  
pp. 1-19
Author(s):  
Chunfeng Guo ◽  
Bin Wei ◽  
Kun Yu
Keyword(s):  
Image Classification ◽  
Transfer Learning ◽  
Poor Performance ◽  
Target Domain ◽  
Deep Convolutional Neural Networks ◽  
Specific Domain ◽  
Source Domain ◽  
Cross Domain ◽  
Domain Transfer ◽  
Domain Transfer Learning

Automatic biology image classification is essential for biodiversity conservation and ecological study. Recently, due to the record-shattering performance, deep convolutional neural networks (DCNNs) have been used more often in biology image classification. However, training DCNNs requires a large amount of labeled data, which may be difficult to collect for some organisms. This study was carried out to exploit cross-domain transfer learning for DCNNs with limited data. According to the literature, previous studies mainly focus on transferring from ImageNet to a specific domain or transferring between two closely related domains. While this study explores deep transfer learning between species from different domains and analyzes the situation when there is a huge difference between the source domain and the target domain. Inspired by the analysis of previous studies, the effect of biology cross-domain image classification in transfer learning is proposed. In this work, the multiple transfer learning scheme is designed to exploit deep transfer learning on several biology image datasets from different domains. There may be a huge difference between the source domain and the target domain, causing poor performance on transfer learning. To address this problem, multistage transfer learning is proposed by introducing an intermediate domain. The experimental results show the effectiveness of cross-domain transfer learning and the importance of data amount and validate the potential of multistage transfer learning.

scholarly journals Cross-Domain Transfer Learning for PCG Diagnosis Algorithm

Biosensors ◽  
2021 ◽  
Vol 11 (4) ◽  
pp. 127
Author(s):  
Kuo-Kun Tseng ◽  
Chao Wang ◽  
Yu-Feng Huang ◽  
Guan-Rong Chen ◽  
Kai-Leung Yung ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Segmentation Method ◽  
Signal Segmentation ◽  
Sustainable Performance ◽  
Cross Domain ◽  
Segmentation Methods ◽  
Diagnosis Algorithm ◽  
Diagnostic Framework ◽  
Domain Transfer ◽  
Domain Transfer Learning

Cardiechema is a way to reflect cardiovascular disease where the doctor uses a stethoscope to help determine the heart condition with a sound map. In this paper, phonocardiogram (PCG) is used as a diagnostic signal, and a deep learning diagnostic framework is proposed. By improving the architecture and modules, a new transfer learning and boosting architecture is mainly employed. In addition, a segmentation method is designed to improve on the existing signal segmentation methods, such as R wave to R wave interval segmentation and fixed segmentation. For the evaluation, the final diagnostic architecture achieved a sustainable performance with a public PCG database.

Sign in / Sign up

Export Citation Format

Share Document