Ch 7 Leveraging Linguistic and Computer Science

1. Introduction :
Linguistics is a field of study that focuses on the strategic application of
linguistic principles and practices to meet specific objectives or goals.
Understanding human language is difficult, even for people who have
grown up with it. Human language is incredibly complex, full of strange
expressions that seem to contradict each other. NLP is a branch of Artificial
Intelligence (AI) that allows computers to understand, create, and manipulate
human speech. NLP has the capability to query the data with natural-language
text or voice. It is also known as “language in”. NLP is at the core of the
technology behind virtual assistants like ODA, Siri, Cortana, Alexa, etc. NLP can
be applied to both written text and speech. Some examples of tools that are
powered by NLP are: Web search Email spam filtering Auto-translate text or
speech Document summarization Sentiment analysis.
2. Introduction to Natural Language Processing - In NLP, machines segment
sentences and extract meaning from “tokens” of human language. Human
language is unstructured. Although it is loosely held together by rules of
grammar, our language expresses information in many confusing ways.
unstructured information is messy and difficult to understand.
To deal with the “messiness” of unstructured information, computers begin
with one sentence at a time. This is called sentence segmentation. Computers then
break the information into small chunks of information, called tokens, that can be
individually classified. Once the tokens in text have been sorted into a structure
based on what they mean, NLP can work with them
Classification Problem - Here is an old-fashioned riddle: Why does your nose
run and your feet smell?
Human language is full of terms that are vague or have double meanings. This is
called a classification problem.
Chatbots - Chatbots are software applications or computer programs designed to
simulate conversation with human users, typically through text-based or voice-
based interactions. They use artificial intelligence (AI), natural language
processing (NLP), and machine learning techniques to understand user queries
and provide appropriate responses. Chatbots can be integrated into various
platforms such as websites, messaging apps, and voice assistants etc
Structure of a chatbot -
A chatbot has a “frontend” and a “backend”.
The frontend of a chatbot serves as the messaging channel through which
users interact, providing a user-friendly interface. However, one limitation of the
frontend is that it may lack contextual understanding, meaning it might struggle
to grasp the full meaning or context of user messages beyond the immediate input.
The backend of a chatbot is where the hard work takes place. The backend
operates application logic and has enough memory to remember earlier parts of a
conversation as dialog continue.
Chatbots understand a question by breaking it into parts and relating those parts
to things in its memory. A chatbot’s goal is to identify entities and intents
Intent - An intent is a purpose: the reason why a user is contacting the chatbot.
Think of it as something like a verb: a kind of action. Users may have various
intents when interacting with a chatbot, such as filing a complaint, asking for
directions etc
Entity - An entity is a noun: a person, place, or thing. Once you have a list of the
intents you want your chatbot to fulfill, you are ready to continue. If a user asks,
“What are the hours for the Bangalore office?”, then providing business hours is
the intent and Bangalore is the entity. A chatbot needs a full list of entities to be
helpful.
Natural Language Processing – Converting Speech to Text & analysing its intent
Natural language processing (NLP) involves a series of five phases that enable
machines to analyse, categorize, and understand both spoken and written
language.
1. Lexical analysis This step involves understanding and examining the structure
of words in a language. It breaks down the text into paragraphs, phrases, and
words
2. Syntactical Analysis Syntactic Analysis is used to check grammar, word
layouts, and word relationships.
3. Semantic Analysis Semantic analysis aims to understand the various meanings
conveyed by a sentence in a clear and contextually appropriate manner. It extracts
relevant insights from the text to comprehend its intended message.
4. Discourse Integration This involves understanding the context of a statement
or word based on preceding sentences or words
5. Pragmatic Analysis It denotes the study of meanings in a particular language.
Process of extracting insights from a text. It involves verbal repetition.
Applications of NLP :
1) Sentiment Analysis: Natural language processing (NLP) may evaluate
consumer comments, social media posts, product reviews, and other text data to
determine the sentiment (positive, negative, or neutral) associated with a specific
brand, product or service. This data aids firms in understanding client impressions
and sentiments.
2) Voice Assistants: - Whether it's Siri, Alexa, or Google Assistant, practically
everyone uses one to make calls, create reminders, plan meetings, set alarms,
browse the internet, and so on.
3) Email Filtering:- Natural Language Processing comes into play. It classifies
incoming emails as "important" or "spam" and assigns them accordingly.
4) Document Analysis: Document analysis is another use of natural language
processing. Companies, institutions, and schools, among other places, are
constantly inundated with data that must be properly organized, stored, and
searched. All of this may be accomplished with NLP