scholarly journals Laughing Heads: Can Transformers Detect What Makes a Sentence Funny?

2021 ◽  
Author(s):  
Maxime Peyrard ◽  
Beatriz Borges ◽  
Kristina Gligorić ◽  
Robert West
Keyword(s):  
Natural Language Processing ◽  
Language Processing ◽  
Test Sentence ◽  
High Accuracy ◽  
The Other ◽  
Grand Challenge ◽  
Training Time ◽  
Minimal Pairs ◽  
Different Sources ◽  
Benchmarking Performance

The automatic detection of humor poses a grand challenge for natural language processing. Transformer-based systems have recently achieved remarkable results on this task, but they usually (1) were evaluated in setups where serious vs humorous texts came from entirely different sources, and (2) focused on benchmarking performance without providing insights into how the models work. We make progress in both respects by training and analyzing transformer-based humor recognition models on a recently introduced dataset consisting of minimal pairs of aligned sentences, one serious, the other humorous. We find that, although our aligned dataset is much harder than previous datasets, transformer-based models recognize the humorous sentence in an aligned pair with high accuracy (78\%). In a careful error analysis, we characterize easy vs hard instances. Finally, by analyzing attention weights, we obtain important insights into the mechanisms by which transformers recognize humor. Most remarkably, we find clear evidence that one single attention head learns to recognize the words that make a test sentence humorous, even without access to this information at training time.

Senti-BAS: A BERT-based model with sentiment computing for happiness research (Preprint)

2021 ◽  
Author(s):  
Zeyuan Zeng ◽  
Yijia Zhang ◽  
Liang Yang ◽  
Hongfei Lin
Keyword(s):  
Deep Learning ◽  
Natural Language Processing ◽  
Language Processing ◽  
High Accuracy ◽  
Language Models ◽  
Fine Grained ◽  
Label Information ◽  
Common Criterion ◽  
Text Content ◽  
Sentiment Computing

BACKGROUND Happiness becomes a rising topic that we all care about recently. It can be described in various forms. For the text content, it is an interesting subject that we can do research on happiness by utilizing natural language processing (NLP) methods. OBJECTIVE As an abstract and complicated emotion, there is no common criterion to measure and describe happiness. Therefore, researchers are creating different models to study and measure happiness. METHODS In this paper, we present a deep-learning based model called Senti-BAS (BERT embedded Bi-LSTM with self-Attention mechanism along with the Sentiment computing). RESULTS Given a sentence that describes how a person felt happiness recently, the model can classify the happiness scenario in the sentence with two topics: was it controlled by the author (label ‘agency’), and was it involving other people (label ‘social’). Besides language models, we employ the label information through sentiment computing based on lexicon. CONCLUSIONS The model performs with a high accuracy on both ‘agency’ and ‘social’ labels, and we also make comparisons with several popular embedding models like Elmo, GPT. Depending on our work, we can study the happiness at a more fine-grained level.

Finding Gene Names

2006 ◽  
Author(s):  
Soumya Raychaudhuri
Keyword(s):  
Natural Language Processing ◽  
Language Processing ◽  
Critical Issue ◽  
High Accuracy ◽  
Scientific Text ◽  
Recognition Algorithms ◽  
Wide Range ◽  
Genomics Research ◽  
Processing Techniques ◽  
Mining Algorithms

Successful use of text mining algorithms to facilitate genomics research hinges on the ability to recognize the names of genes in scientific text. In this chapter we address the critical issue of gene name recognition. Once gene names can be recognized in the scientific text, we can begin to understand what the text says about those genes. This is a much more challenging issue than one might appreciate at first glance. Gene names can be inconsistent and confusing; automated gene name recognition efforts have therfore turned out to be quite challenging to implement with high accuracy. Gene name recognition algorithms have a wide range of useful applications. Until this chapter we have been avoiding this issue and have been using only gene-article indices. In practice these indices are manually assembled. Gene name recognition algorithms offer the possibility of automating and expediting the laborious task of building reference indices. Article indices can be built that associate articles to genes based on whether or not the article mentions the gene by name. In addition, gene name recognition is the first step in doing more detailed sentence-by-sentence text analysis. For example, in Chapter 10 we will talk about identifying relationships between genes from text. Frequently, this requires identifying sentences refering to two gene names, and understanding what sort of relationship the sentence is describing between these genes. Sophisticated natural language processing techniques to parse sentences and understand gene function cannot be done in a meaningful way without recognizing where the gene names are in the first place. The major concepts of this chapter are presented in the frame box. We begin by describing the commonly used strategies that can be used alone or in concert to identify gene names. At the end of the chapter we introduce one successful name finding algorithm that combines many of the different strategies. There are several commonly used approaches that can be exploited to recognize gene names in text (Chang, Shutze, et al. 2004). Often times these approaches can be combined into even more effective multifaceted algorithms.

Research on Sustainable Mining Engineering

2013 ◽  
Vol 340 ◽  
pp. 126-130 ◽  
Author(s):  
Xiao Guang Yue ◽  
Guang Zhang ◽  
Qing Guo Ren ◽  
Wen Cheng Liao ◽  
Jing Xi Chen ◽  
...  
Keyword(s):  
Information Processing ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Social Economy ◽  
Scientific Research ◽  
The Other ◽  
Mining Engineering ◽  
Chinese Information Processing ◽  
Development Tendency

The concepts of Chinese information processing and natural language processing (NLP) and their development tendency are summarized. There are different comprehension of Chinese information processing and natural language processing in China and the other countries. But the work appears to emerge in the study of key point of languages processing. Mining engineering is very important for our country. Though the final task of languages processing is difficult, Chinese information processing has contributed substantially to our scientific research and social economy and it will play an important part for mining engineering in our future.

scholarly journals What counts as an exemplar model, anyway? A commentary on Ambridge (2020)

2020 ◽  
Author(s):  
Kyle Mahowald ◽  
George Kachergis ◽  
Michael C. Frank
Keyword(s):  
Neural Networks ◽  
Natural Language Processing ◽  
Language Processing ◽  
The Other ◽  
Training Data ◽  
Exemplar Model ◽  
Exemplar Models ◽  
Modern Natural ◽  
Architectural Constraints ◽  
Future Work

Ambridge (2019) calls for exemplar-based accounts of language acquisition. Do modern neural networks such as transformers or word2vec – which have been extremely successful in modern natural language processing (NLP) applications – count? Although these models often have ample parametric complexity to store exemplars from their training data, they also go far beyond simple storage by processing and compressing the input via their architectural constraints. The resulting representations have been shown to encode emergent abstractions. If these models are exemplar-based then Ambridge’s theory only weakly constrains future work. On the other hand, if these systems are not exemplar models, why is it that true exemplar models are not contenders in modern NLP?

scholarly journals A Topological Method for Comparing Document Semantics

2020 ◽  
Author(s):  
Yuqi Kong ◽  
Fanchao Meng ◽  
Ben Carterette
Keyword(s):  
Information Retrieval ◽  
Natural Language Processing ◽  
Language Processing ◽  
State Of The Art ◽  
Vector Space Model ◽  
The Other ◽  
Space Model ◽  
Topological Persistence ◽  
Art Methods ◽  
Novel Algorithm

Comparing document semantics is one of the toughest tasks in both Natural Language Processing and Information Retrieval. To date, on one hand, the tools for this task are still rare. On the other hand, most relevant methods are devised from the statistic or the vector space model perspectives but nearly none from a topological perspective. In this paper, we hope to make a different sound. A novel algorithm based on topological persistence for comparing semantics similarity between two documents is proposed. Our experiments are conducted on a document dataset with human judges’ results. A collection of state-of-the-art methods are selected for comparison. The experimental results show that our algorithm can produce highly human-consistent results, and also beats most state-of-the-art methods though ties with NLTK.

scholarly journals Multi-Sense Embeddings per Word

2020 ◽  
Author(s):  
Masashi Sugiyama
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Research Area ◽  
Word Embedding ◽  
The Other ◽  
Word Embeddings ◽  
Word Similarity ◽  
Better Than ◽  
Non Parametric

Recently, word embeddings have been used in many natural language processing problems successfully and how to train a robust and accurate word embedding system efficiently is a popular research area. Since many, if not all, words have more than one sense, it is necessary to learn vectors for all senses of word separately. Therefore, in this project, we have explored two multi-sense word embedding models, including Multi-Sense Skip-gram (MSSG) model and Non-parametric Multi-sense Skip Gram model (NP-MSSG). Furthermore, we propose an extension of the Multi-Sense Skip-gram model called Incremental Multi-Sense Skip-gram (IMSSG) model which could learn the vectors of all senses per word incrementally. We evaluate all the systems on word similarity task and show that IMSSG is better than the other models.

scholarly journals Natural Language Processing in Serious Games: A state of the art.

2015 ◽  
Vol 2 (3) ◽  
Author(s):  
Davide Picca ◽  
Dominique Jaccard ◽  
Gérald Eberlé
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Serious Games ◽  
State Of The Art ◽  
Serious Game ◽  
The Other ◽  
Other Hand ◽  
The One ◽  
High Level

In the last decades, Natural Language Processing (NLP) has obtained a high level of success. Interactions between NLP and Serious Games have started and some of them already include NLP techniques. The objectives of this paper are twofold: on the one hand, providing a simple framework to enable analysis of potential uses of NLP in Serious Games and, on the other hand, applying the NLP framework to existing Serious Games and giving an overview of the use of NLP in pedagogical Serious Games. In this paper we present 11 serious games exploiting NLP techniques. We present them systematically, according to the following structure:  first, we highlight possible uses of NLP techniques in Serious Games, second, we describe the type of NLP implemented in the each specific Serious Game and, third, we provide a link to possible purposes of use for the different actors interacting in the Serious Game.

Predicting Catastrophic Events Using Machine Learning Models for Natural Language Processing

2022 ◽  
pp. 223-243
Author(s):  
Muskaan Chopra ◽  
Sunil K. Singh ◽  
Kriti Aggarwal ◽  
Anshul Gupta
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Communication Technologies ◽  
High Accuracy ◽  
Data Prediction ◽  
The Government ◽  
Media Applications ◽  
Disaster Prediction ◽  
Machine Learning Models

In recent years, there has been widespread improvement in communication technologies. Social media applications like Twitter have made it much easier for people to send and receive information. A direct application of this can be seen in the cases of disaster prediction and crisis. With people being able to share their observations, they can help spread the message of caution. However, the identification of warnings and analyzing the seriousness of text is not an easy task. Natural language processing (NLP) is one way that can be used to analyze various tweets for the same. Over the years, various NLP models have been developed that are capable of providing high accuracy when it comes to data prediction. In the chapter, the authors will analyze various NLP models like logistic regression, naive bayes, XGBoost, LSTM, and word embedding technologies like GloVe and transformer encoder like BERT for the purpose of predicting disaster warnings from the scrapped tweets. The authors focus on finding the best disaster prediction model that can help in warning people and the government.

A text-based multi-span network for reading comprehension

2021 ◽  
pp. 1-13
Author(s):  
Deguang Chen ◽  
Ziping Ma ◽  
Lin Wei ◽  
Yanbin Zhu ◽  
Jinlin Ma ◽  
...  
Keyword(s):  
Reading Comprehension ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
State Of The Art ◽  
Market Value ◽  
The State ◽  
Experimental Results ◽  
Training Time ◽  
The Public

Text-based reading comprehension models have great research significance and market value and are one of the main directions of natural language processing. Reading comprehension models of single-span answers have recently attracted more attention and achieved significant results. In contrast, multi-span answer models for reading comprehension have been less investigated and their performances need improvement. To address this issue, in this paper, we propose a text-based multi-span network for reading comprehension, ALBERT_SBoundary, and build a multi-span answer corpus, MultiSpan_NMU. We also conduct extensive experiments on the public multi-span corpus, MultiSpan_DROP, and our multi-span answer corpus, MultiSpan_NMU, and compare the proposed method with the state-of-the-art. The experimental results show that our proposed method achieves F1 scores of 84.10 and 92.88 on MultiSpan_DROP and MultiSpan_NMU datasets, respectively, while it also has fewer parameters and a shorter training time.

Researching in Shallow Parsing Based on Function Invariance of Maximum Likelihood Estimation

2013 ◽  
Vol 274 ◽  
pp. 359-362
Author(s):  
Shuang Zhang ◽  
Shi Xiong Zhang
Keyword(s):  
Natural Language Processing ◽  
Maximum Likelihood ◽  
Maximum Likelihood Estimation ◽  
Natural Language ◽  
Language Processing ◽  
Likelihood Estimation ◽  
The Other ◽  
Applied Technology ◽  
New Strategy ◽  
Shallow Parsing

Abstract. Shallow parsing is a new strategy of language processing in the domain of natural language processing recently years. It is not focus on the obtaining of the full parsing tree but requiring of the recognition of some simple composition of some structure. It separated parsing into two subtasks: one is the recognition and analysis of chunks the other is the analysis of relationships among chunks. In this essay, some applied technology of shallow parsing is introduced and a new method of it is experimented.

Sign in / Sign up

Export Citation Format

Share Document