Natural Language Processing for Social Media. Diana Inkpen

without the prior permission of the publisher.

      Natural Language Processing for Social Media, Second Edition

      Atefeh Farzindar and Diana Inkpen

       www.morganclaypool.com

      ISBN: 9781681736129 paperback

      ISBN: 9781681736136 ebook

      ISBN: 9781681736143 hardcover

      DOI 10.2200/S00809ED2V01Y201710HLT038

      A Publication in the Morgan & Claypool Publishers series

       SYNTHESIS LECTURES ON HUMAN LANGUAGE TECHNOLOGIES

      Lecture #38

      Series Editor: Graeme Hirst, University of Toronto Series ISSN Print 1947-4040 Electronic 1947-4059

       Natural Language Processing for Social Media

       Second Edition

      Atefeh Farzindar

      University of Southern California

      Diana Inkpen

      University of Ottawa

       SYNTHESIS LECTURES ON HUMAN LANGUAGE TECHNOLOGIES #38

       ABSTRACT TO THE SECOND EDITION

      In recent years, online social networking has revolutionized interpersonal communication. The newer research on language analysis in social media has been increasingly focusing on the latter’s impact on our daily lives, both on a personal and a professional level. Natural language processing (NLP) is one of the most promising avenues for social media data processing. It is a scientific challenge to develop powerful methods and algorithms which extract relevant information from a large volume of data coming from multiple sources and languages in various formats or in free form. We discuss the challenges in analyzing social media texts in contrast with traditional documents.

      Research methods in information extraction, automatic categorization and clustering, automatic summarization and indexing, and statistical machine translation need to be adapted to a new kind of data. This book reviews the current research on NLP tools and methods for processing the non-traditional information from social media data that is available in large amounts (big data), and shows how innovative NLP approaches can integrate appropriate linguistic information in various fields such as social media monitoring, healthcare, business intelligence, industry, marketing, and security and defence.

      We review the existing evaluation metrics for NLP and social media applications, and the new efforts in evaluation campaigns or shared tasks on new datasets collected from social media. Such tasks are organized by the Association for Computational Linguistics (such as SemEval tasks) or by the National Institute of Standards and Technology via the Text REtrieval Conference (TREC) and the Text Analysis Conference (TAC). In the concluding chapter, we discuss the importance of this dynamic discipline and its great potential for NLP in the coming decade, in the context of changes in mobile technology, cloud computing, virtual reality, and social networking.

      In this second edition, we have added information about recent progress in the tasks and applications presented in the first edition. We discuss new methods and their results. The number of research projects and publications that use social media data is constantly increasing due to continuously growing amounts of social media data and the need to automatically process them. We have added 85 new references to the more than 300 references from the first edition. Besides updating each section, we have added a new application (digital marketing) to the section on media monitoring and we have augmented the section on healthcare applications with an extended discussion of recent research on detecting signs of mental illness from social media.

       KEYWORDS

      social media, social networking, natural language processing, social computing, big data, semantic analysis

      This effort is dedicated to my husband, Massoud, and to my daughters, Tina and Amanda, who are just about the best children a mom could hope for: happy, loving, and fun to be with.

       – Atefeh Farzindar

      To my wonderful husband, Nicu, with whom I can climb any mountain, and to our sweet baby daughter Nicoleta.

       – Diana Inkpen

       Contents

       Preface to the Second Edition

       Acknowledgments

       1 Introduction to Social Media Analysis

       1.1 Introduction

       1.2 Social Media Applications

       1.2.1 Cross-language Document Analysis in Social Media Data

       1.2.2 Real-world Applications

       1.3 Challenges in Social Media Data

       1.4 Semantic Analysis of Social Media

       1.5 Summary

       2 Linguistic Pre-processing of Social Media Texts

       2.1 Introduction

       2.2 Generic Adaptation Techniques for NLP Tools

       2.2.1 Text Normalization

       2.2.2 Re-training NLP Tools for Social Media Texts

       2.3 Tokenizers

       2.4 Part-of-speech Taggers

       2.5 Chunkers and Parsers

       2.6 Named Entity Recognizers

       2.7 Existing NLP Toolkits for English and Their Adaptation

       2.8 Multi-linguality and Adaptation to Social Media Texts

       2.8.1 Language Identification

       2.8.2 Dialect Identification