Natural Language Processing

- Overview 

Natural language processing (NLP) is the ability of "intelligent" computer systems to understand human language (written and spoken). This is often called natural language. 

NLP is a subfield of artificial intelligence (AI). It helps machines process and understand human language so that they can automate repetitive tasks. Examples include machine translation, summarization, ticket classification, and spell checking.

NLP has been around for more than fifty years, with the technology originating in linguistics, or the study of human language. It has various practical applications in many industries and fields, including intelligent search engines, advanced medical research, and business processing intelligence. 

Take sentiment analysis, for example, which uses NLP to detect emotions in text. This classification task is one of the most popular tasks in NLP and is often used by companies to automatically detect brand sentiment on social media. Analyzing these interactions can help brands identify urgent customer issues that require an immediate response, or monitor overall customer satisfaction.

NLP is not only concerned with processing, as recent developments in the field such as the introduction of Large Language Models (LLMs) and GPT3, are also aimed at language generation as well.

Although research in NLP covers a wide variety of tasks, most of it can be summarized into three themes: syntax, semantics, and relations.

- Why is NLP Important?

One of the main reasons why NLP is so important to businesses is that it can be used to analyze large amounts of textual data, such as social media comments, customer support tickets, online reviews, news reports, and more.

All this business data contains a wealth of valuable insights that NLP can quickly help businesses uncover. It does this by helping machines understand human language faster, more accurately, and more consistently than human agents. 

NLP tools process data on the fly 24/7 and apply the same criteria to all data, so you can be sure that the results you receive are accurate and not riddled with inconsistencies.

Once NLP tools can understand the meaning of a piece of text and even measure things like sentiment, businesses can start to prioritize and organize material in a way that suits their needs.  

Recent advances in NLP have given rise to some useful tools that have become integrated into our daily lives, such as: spam and phishing classification keeping inboxes sane; automated chatbots offloading customer support staff and empowering customers instant feedback; machine translation bridges the gap between cultures.

NLP draws on many other scientific fields, from formal linguistics to statistics. The goal of NLP is to provide new computing capabilities around human language: for example, conducting conversations, summarizing articles, etc.

[International Commerce Center, Hong Kong - Civil Engineering Discoveries]

- Natural Language Generation (NLG) and Natural Language Understanding (NLU)

Natural language understanding (NLU) is the ability of a computer to understand the meaning of written or spoken language. NLU uses syntactic and semantic analysis to determine the intent of the language. NLU is a subset of natural language processing (NLP). 

Natural language generation (NLG) is the process of creating natural language text or speech based on a given data set. NLG is a field of AI that focuses on generating natural language output. 

In general terms, NLG and NLU are subsections of a more general NLP domain that encompasses all software which interprets or produces human language, in either spoken or written form:

• NLU takes up the understanding of the data based on grammar, the context in which it was said, and decide on intent and entities.
• NLP converts a text into structured data.
• NLG generates text based on structured data.

- Computational Linguistics

Computational linguistics is the scientific study of language from a computational perspective. Computational linguists are interested in providing computational models of various kinds of linguistic phenomena. These models may be "knowledge-based" ("hand-crafted") or "data-driven" ("statistical" or "empirical"). 

Work in computational linguistics is in some cases motivated from a scientific perspective in that one is trying to provide a computational explanation for a particular linguistic or psycholinguistic phenomenon; and in other cases the motivation may be more purely technological in that one wants to provide a working component of a speech or natural language system. 

Indeed, the work of computational linguists is incorporated into many working systems today, including speech recognition systems, text-to-speech synthesizers, automated voice response systems, web search engines, text editors, language instruction materials, to name just a few. 

Computational linguists develop computer systems that deal with human language. They need a good understanding of both programming and linguistics. This is a challenging and technical field, but skilled computational linguists are in demand and highly paid. Following are the areas a computational linguist should concentrate on: programming skills, math and statistics, linguistics, natural language processing. 

[More to come ...]