Have you ever looked at the emails in your spam folder and noticed similarities in the subject lines? Or wondered how Google is able to autocomplete your search requests and even provide you with a list of related searches?
Welcome to the world of Artificial Intelligence (AI)! Such a scary yet powerful tool that is capable of making our lives so much more convenient. Today we zoom in on an important branch of this burgeoning technology - Natural Language Processing (NLP).
But before we dive into what NLP is, let’s have a look at what it is capable of.
What is NLP?
Natural Language Processing (NLP) is defined as the ability of a computer program to understand the human language.
The human language, also known as natural language, is undoubtedly complex. There are over 6500 languages in the world with infinite expressions. Each language has its own unique set of grammar and syntax rules. And let’s not forget the additional variations caused by misspellings, abbreviated words, accents and mixed languages (like Singlish).
NLP makes it possible for computers to understand our language. It gives them the ability to read text, write text, hear speech, interpret it and even measure sentiment.
There are 2 components of NLP:
- Natural language understanding: The process of understanding natural language inputs.
- Natural language generation: The process of producing natural language content of comparable quality to that of a human being.
How does NLP work?
There are 4 types of analyses that occur in NLP. Each step helps to filter out the exact meaning of the text that the machine is trying to decipher.
Lexical Analysis (also known as tokenisation)
In this step, text is broken down into sentences, key phrases or individual words (tokens). Characters such as punctuation marks and other symbols are discarded in the process. This allows the machine to isolate the paragraphs, sentences and words for analysis.
Syntactic Analysis (also known as Parsing)
During this stage, the machine goes through the sentences word by word to distinguish the parts of speech (POS) category that it belongs to. The nouns, verbs, adjectives, etc. are differentiated to grasp how a word functions grammatically and in meaning in a sentence.
Semantic Analysis
This stage involves the machine figuring out the exact meaning of each individual word and combination of words. It tries to understand the relationship between the words and explore how the words work together to create meaning.
Pragmatic Analysis
Pragmatic Analysis goes deeper in order to get a better interpretation of the text. This time it considers the context of the text by studying the previous and succeeding sentences. It also applies real-world knowledge as part of its evaluation process. This is a crucial step because the meaning of text, both written and spoken, are heavily influenced by the context that it is delivered in.
Uses of NLP
1) Sentiment Analysis
Also known as opinion mining, NLP allows data scientist to assess the moods and subjective opinions in a piece of text. This is especially useful for web and social media monitoring because it helps you analyse the emotional state of the writer engaging with your content and attitude towards your brand. These insights will prompt you to make adjustments in order to improve your customers’ experience.
Want to learn more about opinion mining? Sign up for our Data Learning Centre here.
2) Transcription
Transcription is the act of turning speech into written words. Automated transcriptions have been around since the 1960s. But it was only with the advent of NLP that the transcript results have attained a satisfactory level of accuracy. One good use of this would be turning recorded meetings into transcribed copies for the participants. It saves time on minutes writing and allows for more active participation in the meeting.
3) Translation
Gone are the days when foreign languages were obstacles to us interpreting foreign text. With machine translation, we are now able to transform entire web pages from a foreign language to a language we know. Advancements in NLP have made the translation results more comprehensible albeit not 100% accurate still.
4) Voice Search
Alexa, Siri, Google Assistant, Bixby and all the other variations of digital assistants are slowly permeating our homes and places of work. It’s now easier, faster and more convenient for us to conduct searches when our hands are occupied. NLP has given computers the ability to understand what we say, communicate with us and execute our requests with precision. It is an absolutely crucial capability these machines need in order to be able to work in our space, and add value to our lives.
5) Chatbots
A chatbot is defined as a computer program which conducts conversations with human users, via auditory or textual means, especially over the internet. The introduction of NLP into chatbots has allowed developers to convincingly simulate how humans would respond as a conversation partner. With chatbots that use NLP, businesses are able to improve their response time without having to sacrifice their precious sleep.
In sum, NLP is what controls a machine’s language capabilities. Though the use of it is becoming more widespread, it is still not perfect. For example, NLP does not pick up sarcasm well and is unable to discern how the meaning of a sentence changes when different words are emphasised upon. Great headways have been made over the years in this field and we can only expect it to become more robust over time.
More insights