scholarly journals Optimal Transport with Dimensionality Reduction for Domain Adaptation

Symmetry ◽  
2020 ◽  
Vol 12 (12) ◽  
pp. 1994
Author(s):  
Ping Li ◽  
Zhiwei Ni ◽  
Xuhui Zhu ◽  
Juan Song ◽  
Wenying Wu
Keyword(s):  
Dimensionality Reduction ◽  
Optimal Transport ◽  
Domain Adaptation ◽  
Wasserstein Distance ◽  
Local Information ◽  
Target Domain ◽  
Source Domain ◽  
Second Stage ◽  
Cross Domain ◽  
Feature Based

Domain adaptation manages to learn a robust classifier for target domain, using the source domain, but they often follow different distributions. To bridge distribution shift between the two domains, most of previous works aim to align their feature distributions through feature transformation, of which optimal transport for domain adaptation has attract researchers’ interest, as it can exploit the local information of the two domains in the process of mapping the source instances to the target ones by minimizing Wasserstein distance between their feature distributions. However, it may weaken the feature discriminability of source domain, thus degrade domain adaptation performance. To address this problem, this paper proposes a two-stage feature-based adaptation approach, referred to as optimal transport with dimensionality reduction (OTDR). In the first stage, we apply the dimensionality reduction with intradomain variant maximization but source intraclass compactness minimization, to separate data samples as much as possible and enhance the feature discriminability of the source domain. In the second stage, we leverage optimal transport-based technique to preserve the local information of the two domains. Notably, the desirable properties in the first stage can mitigate the degradation of feature discriminability of the source domain in the second stage. Extensive experiments on several cross-domain image datasets validate that OTDR is superior to its competitors in classification accuracy.

scholarly journals Joint Partial Optimal Transport for Open Set Domain Adaptation

2020 ◽  
Author(s):  
Renjun Xu ◽  
Pelen Liu ◽  
Yin Zhang ◽  
Fang Cai ◽  
Jindong Wang ◽  
...  
Keyword(s):  
Optimal Transport ◽  
Transport Model ◽  
Domain Adaptation ◽  
General Setting ◽  
Feature Space ◽  
Set Domain ◽  
Target Domain ◽  
Source Domain ◽  
Open Set ◽  
Proposed Model

Domain adaptation (DA) has achieved a resounding success to learn a good classifier by leveraging labeled data from a source domain to adapt to an unlabeled target domain. However, in a general setting when the target domain contains classes that are never observed in the source domain, namely in Open Set Domain Adaptation (OSDA), existing DA methods failed to work because of the interference of the extra unknown classes. This is a much more challenging problem, since it can easily result in negative transfer due to the mismatch between the unknown and known classes. Existing researches are susceptible to misclassification when target domain unknown samples in the feature space distributed near the decision boundary learned from the labeled source domain. To overcome this, we propose Joint Partial Optimal Transport (JPOT), fully utilizing information of not only the labeled source domain but also the discriminative representation of unknown class in the target domain. The proposed joint discriminative prototypical compactness loss can not only achieve intra-class compactness and inter-class separability, but also estimate the mean and variance of the unknown class through backpropagation, which remains intractable for previous methods due to the blindness about the structure of the unknown classes. To our best knowledge, this is the first optimal transport model for OSDA. Extensive experiments demonstrate that our proposed model can significantly boost the performance of open set domain adaptation on standard DA datasets.

scholarly journals Distributional Correspondence Indexing for Cross-Lingual and Cross-Domain Sentiment Classification (Extended Abstract)

2018 ◽  
Author(s):  
Alejandro Moreo Fernández ◽  
Andrea Esuli ◽  
Fabrizio Sebastiani
Keyword(s):  
Domain Adaptation ◽  
State Of The Art ◽  
Sentiment Classification ◽  
Training Data ◽  
Target Domain ◽  
Source Domain ◽  
Machine Learning Methods ◽  
Cross Domain ◽  
Current State ◽  
Cross Lingual

Domain Adaptation (DA) techniques aim at enabling machine learning methods learn effective classifiers for a “target” domain when the only available training data belongs to a different “source” domain. In this extended abstract, we briefly describe our new DA method called Distributional Correspondence Indexing (DCI) for sentiment classification. DCI derives term representations in a vector space common to both domains where each dimension reflects its distributional correspondence to a pivot, i.e., to a highly predictive term that behaves similarly across domains. The experiments we have conducted show that DCI obtains better performance than current state-of-the-art techniques for cross-lingual and cross-domain sentiment classification.

scholarly journals Transfer Learning for Wireless Fingerprinting Localization Based on Optimal Transport

Sensors ◽  
2020 ◽  
Vol 20 (23) ◽  
pp. 6994
Author(s):  
Siqi Bai ◽  
Yongjie Luo ◽  
Qun Wan
Keyword(s):  
Transfer Learning ◽  
Optimal Transport ◽  
Wasserstein Distance ◽  
Target Domain ◽  
The Public ◽  
Empirical Distributions ◽  
Feature Based ◽  
And Performance ◽  
The Cost ◽  
Radio Map

Wireless fingerprinting localization (FL) systems identify locations by building radio fingerprint maps, aiming to provide satisfactory location solutions for the complex environment. However, the radio map is easy to change, and the cost of building a new one is high. One research focus is to transfer knowledge from the old radio maps to a new one. Feature-based transfer learning methods help by mapping the source fingerprint and the target fingerprint to a common hidden domain, then minimize the maximum mean difference (MMD) distance between the empirical distributions in the latent domain. In this paper, the optimal transport (OT)-based transfer learning is adopted to directly map the fingerprint from the source domain to the target domain by minimizing the Wasserstein distance so that the data distribution of the two domains can be better matched and the positioning performance in the target domain is improved. Two channel-models are used to simulate the transfer scenarios, and the public measured data test further verifies that the transfer learning based on OT has better accuracy and performance when the radio map changes in FL, indicating the importance of the method in this field.

scholarly journals Few shot domain adaptation for in situ macromolecule structural classification in cryoelectron tomograms

2020 ◽  
Author(s):  
Liangyong Yu ◽  
Ran Li ◽  
Xiangrui Zeng ◽  
Hongyi Wang ◽  
Jie Jin ◽  
...  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Spatial Organization ◽  
Domain Adaptation ◽  
Single Cells ◽  
Supplementary Information ◽  
Target Domain ◽  
Source Domain ◽  
Cellular Processes ◽  
Cross Domain

Abstract Motivation Cryoelectron tomography (cryo-ET) visualizes structure and spatial organization of macromolecules and their interactions with other subcellular components inside single cells in the close-to-native state at submolecular resolution. Such information is critical for the accurate understanding of cellular processes. However, subtomogram classification remains one of the major challenges for the systematic recognition and recovery of the macromolecule structures in cryo-ET because of imaging limits and data quantity. Recently, deep learning has significantly improved the throughput and accuracy of large-scale subtomogram classification. However, often it is difficult to get enough high-quality annotated subtomogram data for supervised training due to the enormous expense of labeling. To tackle this problem, it is beneficial to utilize another already annotated dataset to assist the training process. However, due to the discrepancy of image intensity distribution between source domain and target domain, the model trained on subtomograms in source domain may perform poorly in predicting subtomogram classes in the target domain. Results In this article, we adapt a few shot domain adaptation method for deep learning-based cross-domain subtomogram classification. The essential idea of our method consists of two parts: (i) take full advantage of the distribution of plentiful unlabeled target domain data, and (ii) exploit the correlation between the whole source domain dataset and few labeled target domain data. Experiments conducted on simulated and real datasets show that our method achieves significant improvement on cross domain subtomogram classification compared with baseline methods. Availability and implementation Software is available online https://github.com/xulabs/aitom. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Differentially Private Optimal Transport: Application to Domain Adaptation

2019 ◽  
Author(s):  
Nam LeTien ◽  
Amaury Habrard ◽  
Marc Sebban
Keyword(s):  
Optimal Transport ◽  
Transport Model ◽  
Domain Adaptation ◽  
Wasserstein Distance ◽  
Source Distribution ◽  
Target Domain ◽  
The Past ◽  
Series Of Experiments ◽  
Adaptation Scenarios ◽  
Transportation Plan

Optimal transport has received much attention during the past few years to deal with domain adaptation tasks. The goal is to transfer knowledge from a source domain to a target domain by finding a transportation of minimal cost moving the source distribution to the target one. In this paper, we address the challenging task of privacy preserving domain adaptation by optimal transport. Using the Johnson-Lindenstrauss transform together with some noise, we present the first differentially private optimal transport model and show how it can be directly applied on both unsupervised and semi-supervised domain adaptation scenarios. Our theoretically grounded method allows the optimization of the transportation plan and the Wasserstein distance between the two distributions while protecting the data of both domains.We perform an extensive series of experiments on various benchmarks (VisDA, Office-Home and Office-Caltech datasets) that demonstrates the efficiency of our method compared to non-private strategies.

Cross-Domain Metric and Multiple Kernel Learning Based on Information Theory

2018 ◽  
Vol 30 (3) ◽  
pp. 820-855 ◽  
Author(s):  
Wei Wang ◽  
Hao Wang ◽  
Chen Zhang ◽  
Yang Gao
Keyword(s):  
Domain Adaptation ◽  
Convex Combination ◽  
Learning Algorithms ◽  
Metric Learning ◽  
Multiple Kernel Learning ◽  
Kernel Learning ◽  
Target Domain ◽  
Source Domain ◽  
Cross Domain ◽  
Multiple Kernel

Learning an appropriate distance metric plays a substantial role in the success of many learning machines. Conventional metric learning algorithms have limited utility when the training and test samples are drawn from related but different domains (i.e., source domain and target domain). In this letter, we propose two novel metric learning algorithms for domain adaptation in an information-theoretic setting, allowing for discriminating power transfer and standard learning machine propagation across two domains. In the first one, a cross-domain Mahalanobis distance is learned by combining three goals: reducing the distribution difference between different domains, preserving the geometry of target domain data, and aligning the geometry of source domain data with label information. Furthermore, we devote our efforts to solving complex domain adaptation problems and go beyond linear cross-domain metric learning by extending the first method to a multiple kernel learning framework. A convex combination of multiple kernels and a linear transformation are adaptively learned in a single optimization, which greatly benefits the exploration of prior knowledge and the description of data characteristics. Comprehensive experiments in three real-world applications (face recognition, text classification, and object categorization) verify that the proposed methods outperform state-of-the-art metric learning and domain adaptation methods.

scholarly journals AlignFlow: Cycle Consistent Learning from Multiple Domains via Normalizing Flows

2020 ◽  
Vol 34 (04) ◽  
pp. 4028-4035 ◽  
Author(s):  
Aditya Grover ◽  
Christopher Chute ◽  
Rui Shu ◽  
Zhangjie Cao ◽  
Stefano Ermon
Keyword(s):  
Domain Adaptation ◽  
Likelihood Estimation ◽  
Learning Objectives ◽  
Target Domain ◽  
Modeling Framework ◽  
Source Domain ◽  
Cross Domain ◽  
Generative Modeling ◽  
Adversarial Training ◽  
Multiple Domains

Given datasets from multiple domains, a key challenge is to efficiently exploit these data sources for modeling a target domain. Variants of this problem have been studied in many contexts, such as cross-domain translation and domain adaptation. We propose AlignFlow, a generative modeling framework that models each domain via a normalizing flow. The use of normalizing flows allows for a) flexibility in specifying learning objectives via adversarial training, maximum likelihood estimation, or a hybrid of the two methods; and b) learning and exact inference of a shared representation in the latent space of the generative model. We derive a uniform set of conditions under which AlignFlow is marginally-consistent for the different learning objectives. Furthermore, we show that AlignFlow guarantees exact cycle consistency in mapping datapoints from a source domain to target and back to the source domain. Empirically, AlignFlow outperforms relevant baselines on image-to-image translation and unsupervised domain adaptation and can be used to simultaneously interpolate across the various domains using the learned representation.

A comparative study of pivot selection strategies for unsupervised cross-domain sentiment classification

2018 ◽  
Vol 33 ◽  
Author(s):  
Xia Cui ◽  
Noor Al-Bazzaz ◽  
Danushka Bollegala ◽  
Frans Coenen
Keyword(s):  
Mutual Information ◽  
Comparative Study ◽  
Domain Adaptation ◽  
Selection Strategy ◽  
Target Domain ◽  
Source Domain ◽  
Selection Strategies ◽  
Cross Domain ◽  
Pivot Selection ◽  
Pointwise Mutual Information

AbstractSelecting pivot features that connect a source domain to a target domain is an important first step in unsupervised domain adaptation (UDA). Although different strategies such as the frequency of a feature in a domain, mutual (or pointwise mutual) information have been proposed in prior work in domain adaptation (DA) for selecting pivots, a comparative study into (a) how the pivots selected using existing strategies differ, and (b) how the pivot selection strategy affects the performance of a target DA task remain unknown. In this paper, we perform a comparative study covering different strategies that use both labelled (available for the source domain only) as well as unlabelled (available for both the source and target domains) data for selecting pivots for UDA. Our experiments show that in most cases pivot selection strategies that use labelled data outperform their unlabelled counterparts, emphasising the importance of the source domain labelled data for UDA. Moreover, pointwise mutual information and frequency-based pivot selection strategies obtain the best performances in two state-of-the-art UDA methods.

scholarly journals Discriminative Sparse Filtering for Multi-Source Image Classification

Sensors ◽  
2020 ◽  
Vol 20 (20) ◽  
pp. 5868 ◽  
Author(s):  
Chao Han ◽  
Deyun Zhou ◽  
Zhen Yang ◽  
Yu Xie ◽  
Kai Zhang
Keyword(s):  
Domain Adaptation ◽  
State Of The Art ◽  
Source Image ◽  
Unified Framework ◽  
Target Domain ◽  
Maximum Mean Discrepancy ◽  
Source Domain ◽  
Cross Domain ◽  
Sparse Filtering ◽  
Novel Approach

Distribution mismatch caused by various resolutions, backgrounds, etc. can be easily found in multi-sensor systems. Domain adaptation attempts to reduce such domain discrepancy by means of different measurements, e.g., maximum mean discrepancy (MMD). Despite their success, such methods often fail to guarantee the separability of learned representation. To tackle this issue, we put forward a novel approach to jointly learn both domain-shared and discriminative representations. Specifically, we model the feature discrimination explicitly for two domains. Alternating discriminant optimization is proposed to obtain discriminative features with an l2 constraint in labeled source domain and sparse filtering is introduced to capture the intrinsic structures exists in the unlabeled target domain. Finally, they are integrated in a unified framework along with MMD to align domains. Extensive experiments compared with state-of-the-art methods verify the effectiveness of our method on cross-domain tasks.

scholarly journals Adversarial Hard Attention Adaptation

Information ◽  
2020 ◽  
Vol 11 (4) ◽  
pp. 224
Author(s):  
Hui Tao ◽  
Jun He ◽  
Quanjie Cao ◽  
Lei Zhang
Keyword(s):  
Visual Attention ◽  
Domain Knowledge ◽  
Domain Adaptation ◽  
Experimental Results ◽  
Target Domain ◽  
Source Domain ◽  
Visual Attention Model ◽  
Attention Model ◽  
Cross Domain ◽  
Adversarial Training

Domain adaptation is critical to transfer the invaluable source domain knowledge to the target domain. In this paper, for a particular visual attention model, saying hard attention, we consider to adapt the learned hard attention to the unlabeled target domain. To tackle this kind of hard attention adaptation, a novel adversarial reward strategy is proposed to train the policy of the target domain agent. In this adversarial training framework, the target domain agent competes with the discriminator which takes the attention features generated from the both domain agents as input and tries its best to distinguish them, and thus the target domain policy is learned to align the local attention feature to its source domain counterpart. We evaluated our model on the benchmarks of the cross-domain tasks, such as the centered digits datasets and the enlarged non-centered digits datasets. The experimental results show that our model outperforms the ADDA and other existing methods.

Sign in / Sign up

Export Citation Format

Share Document