scholarly journals Synthesising Facial Macro- and Micro-Expressions Using Reference Guided Style Transfer

2021 ◽  
Vol 7 (8) ◽  
pp. 142
Author(s):  
Chuin Hong Yap ◽  
Ryan Cunningham ◽  
Adrian K. Davison ◽  
Moi Hoon Yap
Keyword(s):  
Evaluation Method ◽  
Performance Metrics ◽  
Synthetic Data ◽  
Future Research ◽  
Reference Image ◽  
Source Image ◽  
Generative Adversarial Network ◽  
Style Transfer ◽  
Facial Movements ◽  
Action Units

Long video datasets of facial macro- and micro-expressions remains in strong demand with the current dominance of data-hungry deep learning methods. There are limited methods of generating long videos which contain micro-expressions. Moreover, there is a lack of performance metrics to quantify the generated data. To address the research gaps, we introduce a new approach to generate synthetic long videos and recommend assessment methods to inspect dataset quality. For synthetic long video generation, we use the state-of-the-art generative adversarial network style transfer method—StarGANv2. Using StarGANv2 pre-trained on the CelebA dataset, we transfer the style of a reference image from SAMM long videos (a facial micro- and macro-expression long video dataset) onto a source image of the FFHQ dataset to generate a synthetic dataset (SAMM-SYNTH). We evaluate SAMM-SYNTH by conducting an analysis based on the facial action units detected by OpenFace. For quantitative measurement, our findings show high correlation on two Action Units (AUs), i.e., AU12 and AU6, of the original and synthetic data with a Pearson’s correlation of 0.74 and 0.72, respectively. This is further supported by evaluation method proposed by OpenFace on those AUs, which also have high scores of 0.85 and 0.59. Additionally, optical flow is used to visually compare the original facial movements and the transferred facial movements. With this article, we publish our dataset to enable future research and to increase the data pool of micro-expressions research, especially in the spotting task.

scholarly journals Style Transfer Using Generative Adversarial Networks for Multi-Site MRI Harmonization

2021 ◽  
Author(s):  
Mengting Liu ◽  
Piyush Maiti ◽  
Sophia Thomopoulos ◽  
Alyssa Zhu ◽  
Yaqiong Chai ◽  
...  
Keyword(s):  
Large Scale ◽  
Transfer Problem ◽  
Large Data ◽  
Generative Adversarial Networks ◽  
Reference Image ◽  
Demographic Information ◽  
Mr Images ◽  
Generative Adversarial Network ◽  
Style Transfer ◽  
Cross Site

AbstractLarge data initiatives and high-powered brain imaging analyses require the pooling of MR images acquired across multiple scanners, often using different protocols. Prospective cross-site harmonization often involves the use of a phantom or traveling subjects. However, as more datasets are becoming publicly available, there is a growing need for retrospective harmonization, pooling data from sites not originally coordinated together. Several retrospective harmonization techniques have shown promise in removing cross-site image variation. However, most unsupervised methods cannot distinguish between image-acquisition based variability and cross-site population variability, so they require that datasets contain subjects or patient groups with similar clinical or demographic information. To overcome this limitation, we consider cross-site MRI image harmonization as a style transfer problem rather than a domain transfer problem. Using a fully unsupervised deep-learning framework based on a generative adversarial network (GAN), we show that MR images can be harmonized by inserting the style information encoded from a reference image directly, without knowing their site/scanner labels a priori. We trained our model using data from five large-scale multi-site datasets with varied demographics. Results demonstrated that our styleencoding model can harmonize MR images, and match intensity profiles, successfully, without relying on traveling subjects. This model also avoids the need to control for clinical, diagnostic, or demographic information. Moreover, we further demonstrated that if we included diverse enough images into the training set, our method successfully harmonized MR images collected from unseen scanners and protocols, suggesting a promising novel tool for ongoing collaborative studies.

scholarly journals Shall I Work with Them? A Knowledge Graph-Based Approach for Predicting Future Research Collaborations

Entropy ◽  
2021 ◽  
Vol 23 (6) ◽  
pp. 664
Author(s):  
Nikos Kanakaris ◽  
Nikolaos Giarelis ◽  
Ilias Siachos ◽  
Nikos Karacapilidis
Keyword(s):  
Language Processing ◽  
Scientific Knowledge ◽  
Link Prediction ◽  
Performance Metrics ◽  
Future Research ◽  
Knowledge Graph ◽  
Prediction Problem ◽  
Textual Information ◽  
Research Collaborations ◽  
Processing Techniques

We consider the prediction of future research collaborations as a link prediction problem applied on a scientific knowledge graph. To the best of our knowledge, this is the first work on the prediction of future research collaborations that combines structural and textual information of a scientific knowledge graph through a purposeful integration of graph algorithms and natural language processing techniques. Our work: (i) investigates whether the integration of unstructured textual data into a single knowledge graph affects the performance of a link prediction model, (ii) studies the effect of previously proposed graph kernels based approaches on the performance of an ML model, as far as the link prediction problem is concerned, and (iii) proposes a three-phase pipeline that enables the exploitation of structural and textual information, as well as of pre-trained word embeddings. We benchmark the proposed approach against classical link prediction algorithms using accuracy, recall, and precision as our performance metrics. Finally, we empirically test our approach through various feature combinations with respect to the link prediction problem. Our experimentations with the new COVID-19 Open Research Dataset demonstrate a significant improvement of the abovementioned performance metrics in the prediction of future research collaborations.

scholarly journals Generative adversarial network for glioblastoma ensures morphologic variations and improves diagnostic model for isocitrate dehydrogenase mutant type

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Ji Eun Park ◽  
Dain Eun ◽  
Ho Sung Kim ◽  
Da Hyun Lee ◽  
Ryoung Woo Jang ◽  
...  
Keyword(s):  
Isocitrate Dehydrogenase ◽  
Synthetic Data ◽  
Diagnostic Model ◽  
Generative Adversarial Network ◽  
Classification Rate ◽  
Mutant Type ◽  
Clinical Model ◽  
The Real ◽  
Adversarial Network ◽  
Synthetic Images

AbstractGenerative adversarial network (GAN) creates synthetic images to increase data quantity, but whether GAN ensures meaningful morphologic variations is still unknown. We investigated whether GAN-based synthetic images provide sufficient morphologic variations to improve molecular-based prediction, as a rare disease of isocitrate dehydrogenase (IDH)-mutant glioblastomas. GAN was initially trained on 500 normal brains and 110 IDH-mutant high-grade astocytomas, and paired contrast-enhanced T1-weighted and FLAIR MRI data were generated. Diagnostic models were developed from real IDH-wild type (n = 80) with real IDH-mutant glioblastomas (n = 38), or with synthetic IDH-mutant glioblastomas, or augmented by adding both real and synthetic IDH-mutant glioblastomas. Turing tests showed synthetic data showed reality (classification rate of 55%). Both the real and synthetic data showed that a more frontal or insular location (odds ratio [OR] 1.34 vs. 1.52; P = 0.04) and distinct non-enhancing tumor margins (OR 2.68 vs. 3.88; P < 0.001), which become significant predictors of IDH-mutation. In an independent validation set, diagnostic accuracy was higher for the augmented model (90.9% [40/44] and 93.2% [41/44] for each reader, respectively) than for the real model (84.1% [37/44] and 86.4% [38/44] for each reader, respectively). The GAN-based synthetic images yield morphologically variable, realistic-seeming IDH-mutant glioblastomas. GAN will be useful to create a realistic training set in terms of morphologic variations and quality, thereby improving diagnostic performance in a clinical model.

scholarly journals An Improved Method of Reservoir Facies Modeling Based on Generative Adversarial Networks

Energies ◽  
2021 ◽  
Vol 14 (13) ◽  
pp. 3873
Author(s):  
Qingbin Liu ◽  
Wenling Liu ◽  
Jianpeng Yao ◽  
Yuyang Liu ◽  
Mao Pan
Keyword(s):  
Synthetic Data ◽  
Oil Field ◽  
Generative Adversarial Networks ◽  
Parameter Distribution ◽  
Good Prospect ◽  
Facies Modeling ◽  
Generative Adversarial Network ◽  
Hard Data ◽  
Reservoir Facies ◽  
Facies Models

As the reservoir and its attribute distribution are obviously controlled by sedimentary facies, the facies modeling is one of the important bases for delineating the area of high-quality reservoir and characterizing the attribute parameter distribution. There are a large number of continental sedimentary reservoirs with strong heterogeneity in China, the geometry and distribution of various sedimentary microfacies are relatively complex. The traditional geostatistics methods which have shortage in characterization of the complex and non-stationary geological patterns, have limitation in facies modeling of continental sedimentary reservoirs. The generative adversarial network (GANs) is a recent state-of-the-art deep learning method, which has capabilities of pattern learning and generation, and is widely used in the domain of image generation. Because of the similarity in content and structure between facies models and specific images (such as fluvial facies and the images of modern rivers), and the various images generated by GANs are often more complex than reservoir facies models, GANs has potential to be used in reservoir facies modeling. Therefore, this paper proposes a reservoir facies modeling method based on GANs: (1) for unconditional modeling, select training images (TIs) based on priori geological knowledge, and use GANs to learn priori geological patterns in TIs, then generate the reservoir facies model by GANs; (2) for conditional modeling, a training method of “unconditional-conditional simulation cooperation” (UCSC) is used to realize the constraint of hard data while learning the priori geological patterns. Testing the method using both synthetic data and actual data from oil field, the results meet perfectly the priori geological patterns and honor the well point hard data, and show that this method can overcome the limitation that traditional geostatistics are difficult to deal with the complex non-stationary patterns and improve the conditional constraint effect of GANs based methods. Given its good performance in facies modeling, the method has a good prospect in practical application.

scholarly journals Opportunistic Large Array Propagation Models: A Comprehensive Survey

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4206
Author(s):  
Farhan Nawaz ◽  
Hemant Kumar ◽  
Syed Ali Hassan ◽  
Haejoon Jung
Keyword(s):  
Energy Efficient ◽  
Large Scale ◽  
Performance Metrics ◽  
Experimental Studies ◽  
Cooperative Transmission ◽  
Analytical Models ◽  
Future Research ◽  
Large Array ◽  
Propagation Models ◽  
Comprehensive Survey

Enabled by the fifth-generation (5G) and beyond 5G communications, large-scale deployments of Internet-of-Things (IoT) networks are expected in various application fields to handle massive machine-type communication (mMTC) services. Device-to-device (D2D) communications can be an effective solution in massive IoT networks to overcome the inherent hardware limitations of small devices. In such D2D scenarios, given that a receiver can benefit from the signal-to-noise-ratio (SNR) advantage through diversity and array gains, cooperative transmission (CT) can be employed, so that multiple IoT nodes can create a virtual antenna array. In particular, Opportunistic Large Array (OLA), which is one type of CT technique, is known to provide fast, energy-efficient, and reliable broadcasting and unicasting without prior coordination, which can be exploited in future mMTC applications. However, OLA-based protocol design and operation are subject to network models to characterize the propagation behavior and evaluate the performance. Further, it has been shown through some experimental studies that the most widely-used model in prior studies on OLA is not accurate for networks with networks with low node density. Therefore, stochastic models using quasi-stationary Markov chain are introduced, which are more complex but more exact to estimate the key performance metrics of the OLA transmissions in practice. Considering the fact that such propagation models should be selected carefully depending on system parameters such as network topology and channel environments, we provide a comprehensive survey on the analytical models and framework of the OLA propagation in the literature, which is not available in the existing survey papers on OLA protocols. In addition, we introduce energy-efficient OLA techniques, which are of paramount importance in energy-limited IoT networks. Furthermore, we discuss future research directions to combine OLA with emerging technologies.

Multi-exposure X-ray image fusion quality evaluation based on CSF and gradient amplitude similarity

2021 ◽  
pp. 1-13
Author(s):  
Yanjie Qi ◽  
Zehui Yang ◽  
Lin Kang
Keyword(s):  
Image Fusion ◽  
Quality Evaluation ◽  
Evaluation Method ◽  
Dynamic Range ◽  
Reference Image ◽  
Subjective Perception ◽  
Steel Component ◽  
Imaging Device ◽  
X Ray ◽  
Gradient Amplitude

Due to the limitation of dynamic range of the imaging device, the fixed-voltage X-ray images often produce overexposed or underexposed regions. Some structure information of the composite steel component is lost. This problem can be solved by fusing the multi-exposure X-ray images taken by using different voltages in order to produce images with more detailed structures or information. Due to the lack of research on multi-exposure X-ray image fusion technology, there is no evaluation method specially for multi-exposure X-ray image fusion. For the multi-exposure X-ray fusion images obtained by different fusion algorithms may have problems such as the detail loss and structure disorder. To address these problems, this study proposes a new multi-exposure X-ray image fusion quality evaluation method based on contrast sensitivity function (CSF) and gradient amplitude similarity. First, with the idea of information fusion, multiple reference images are fused into a new reference image. Next, the gradient amplitude similarity between the new reference image and the test image is calculated. Then, the whole evaluation value can be obtained by weighting CSF. In the experiments of MEF Database, the SROCC of the proposed algorithm is about 0.8914, and the PLCC is about 0.9287, which shows that the proposed algorithm is more consistent with subjective perception in MEF Database. Thus, this study demonstrates a new objective evaluation method, which generates the results that are consistent with the subjective feelings of human eyes.

scholarly journals Development and Formative Evaluation of a Low-Fidelity Equine Castration Model for Veterinary Education

2021 ◽  
Vol 8 ◽  
Author(s):  
M. Katie Sheats ◽  
Megan J. Burke ◽  
James B. Robertson ◽  
Katherine E. Fiebrandt ◽  
Callie A. Fogle
Keyword(s):  
Student Performance ◽  
Student Attitudes ◽  
Formative Evaluation ◽  
Performance Metrics ◽  
Early Stage ◽  
Future Research ◽  
Student Survey ◽  
Significant Difference ◽  
Veterinary Faculty ◽  
Fidelity Model

Entrustable Professional Activities (EPAs) are units of activity that early-stage professionals perform in the workplace that necessitate simultaneous integration of multiple competencies. EPA #6 requires students to perform a common surgical procedure on a stable patient, including pre-operative and post-operative management. Castration is one of the most common surgeries performed by equine primary care practitioners and is considered an “entry-level competency” for veterinary graduates entering equine private practice, however, to our knowledge there are no equine castration models available for veterinary student education. Therefore, we developed an inexpensive, low-fidelity model of equine field castration and evaluated it using a mixed-methods approach. Two different groups of students, with or without model experience, completed surveys before and after live horse castration. Students who used the model also completed model specific surveys. Videos of the students completing the model were evaluated by at least two different equine veterinary faculty using a 15-point rubric, and inter-rater reliability of the rubric was determined. After completing the model, students reflected on strengths and weaknesses of their performance. From our student survey results, we determined that student attitudes toward the model were mostly positive. Interestingly, there were several student attitudes toward the model that became significantly more favorable after live horse castration. Prior to live horse castration, there was no significant difference in confidence in model vs. no-model groups. Following live horse castration, students who used the model had higher confidence in procedure preparation and hand-ties than students who did not use the model, but they had lower scores for confidence during patient recovery. When reflecting on model castration, students most commonly cited preparation and surgical description as strengths, and ligature placement and hand-ties as weaknesses. Experts provided several suggestions to improve the model, including incorporation of emasculators and the need for better model stabilization. Our findings suggest that both students and veterinary educators feel that this low-fidelity model has educational value. Rubric performance metrics were favorable, but additional steps are needed to improve grading consistency among educators. Future research will determine whether student performance on the model is predictive of competence score during live-horse castration.

scholarly journals Financial Thought Experiment: A GAN-based Approach to Vast Robust Portfolio Selection

2020 ◽  
Author(s):  
Chi Seng Pun ◽  
Lei Wang ◽  
Hoi Ying Wong
Keyword(s):  
Stock Market ◽  
Thought Experiment ◽  
Empirical Studies ◽  
Synthetic Data ◽  
Risk Minimization ◽  
Portfolio Risk ◽  
Generative Adversarial Network ◽  
Day Trading ◽  
Adversarial Network ◽  
Robust Portfolio

Modern day trading practice resembles a thought experiment, where investors imagine various possibilities of future stock market and invest accordingly. Generative adversarial network (GAN) is highly relevant to this trading practice in two ways. First, GAN generates synthetic data by a neural network that is technically indistinguishable from the reality, which guarantees the reasonableness of the experiment. Second, GAN generates multitudes of fake data, which implements half of the experiment. In this paper, we present a new architecture of GAN and adapt it to portfolio risk minimization problem by adding a regression network to GAN (implementing the second half of the experiment). The new architecture is termed GANr. Battling against two distinctive networks: discriminator and regressor, GANr's generator aims to simulate a stock market that is close to the reality while allow for all possible scenarios. The resulting portfolio resembles a robust portfolio with data-driven ambiguity. Our empirical studies show that GANr portfolio is more resilient to bleak financial scenarios than CLSGAN and LASSO portfolios.

scholarly journals RoboCoDraw: Robotic Avatar Drawing with GAN-Based Style Transfer and Time-Efficient Path Optimization

2020 ◽  
Vol 34 (06) ◽  
pp. 10402-10409
Author(s):  
Tianying Wang ◽  
Wei Qi Toh ◽  
Hao Zhang ◽  
Xiuchao Sui ◽  
Shaohua Li ◽  
...  
Keyword(s):  
Face Image ◽  
Path Optimization ◽  
Robotic Arm ◽  
Human Face ◽  
Generative Adversarial Network ◽  
Style Transfer ◽  
Adversarial Network ◽  
Face Images ◽  
Drawing System ◽  
Collaborative Robot

Robotic drawing has become increasingly popular as an entertainment and interactive tool. In this paper we present RoboCoDraw, a real-time collaborative robot-based drawing system that draws stylized human face sketches interactively in front of human users, by using the Generative Adversarial Network (GAN)-based style transfer and a Random-Key Genetic Algorithm (RKGA)-based path optimization. The proposed RoboCoDraw system takes a real human face image as input, converts it to a stylized avatar, then draws it with a robotic arm. A core component in this system is the AvatarGAN proposed by us, which generates a cartoon avatar face image from a real human face. AvatarGAN is trained with unpaired face and avatar images only and can generate avatar images of much better likeness with human face images in comparison with the vanilla CycleGAN. After the avatar image is generated, it is fed to a line extraction algorithm and converted to sketches. An RKGA-based path optimization algorithm is applied to find a time-efficient robotic drawing path to be executed by the robotic arm. We demonstrate the capability of RoboCoDraw on various face images using a lightweight, safe collaborative robot UR5.

Digital supply chain performance metrics: a literature review

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Faisal Rasool ◽  
Marco Greco ◽  
Michele Grimaldi
Keyword(s):  
Supply Chain ◽  
Literature Review ◽  
Systematic Literature Review ◽  
Performance Metrics ◽  
Research Process ◽  
Future Research ◽  
Balance Scorecard ◽  
Content Type ◽  
Previous Decade ◽  
Qualitative Performance

Purpose In the previous decade, a substantial amount of research has been undertaken to measure the digitalized supply chain (DSC) performance. This paper aims to present a systematic literature review on DSC performance measurement metrics to apprehend current practices, recognize gaps and advocate future research itineraries. Design/methodology/approach To guarantee a replicable, rigorous and transparent research process, the authors used a systematic literature review methodology to synthesize the research. A combination of 25 keywords was used to obtain 248 scientific studies in the first step. The balance scorecard (BSC) model was used to categorize 299 gathered performance metrics into 4 BSC perspectives. Findings The review highlighted the need for qualitative performance measuring metrics for DSC. During the review, only one study was identified that primarily focused on developing performance metrics for DSC. Additionally, the review identified that metrics related to internal and financial perspectives received the most attention while the “growth and learning” perspective received the least attention. The review also identified that external partners, such as distributors and suppliers, were virtually ignored in previous literature. Originality/value Although numerous literature reviews have been conducted in the past on the performance measuring metrics for supply chain management, no literature review aiming to synthesize the measuring metrics for DSC has yet been undertaken.

Sign in / Sign up

Export Citation Format

Share Document