Introduction
to Linguistics
Instructor: Prof. Dr. Siusana Kweldju
Computational Linguistics
Computational linguistics is an interdisciplinary field of scientific and engineering
discipline concerned with the computational modelling of natural language, as well
as the study of appropriate computational approaches to linguistic questions. It is
concerned with understanding written and spoken language from a computational
perspective. And since language is our most natural and most versatile means of
communication, linguistically competent computers would greatly facilitate our
interaction with machines and software of all sorts, and put at our fingertips, in
ways that truly meet our needs, the vast textual and other resources of the internet.
In short, computational linguistics seeks to develop the computational machinery
needed for an agent to exhibit various forms of linguistic behavior. Agent can be
both human beings and artificial agents such as computer programs. Machinery is
computer programs as well as the linguistic knowledge that they contain.
In general, computational linguistics draws upon linguistics, computer
science, artificial intelligence, mathematics, logic, philosophy, cognitive
science, cognitive psychology, psycholinguistics, anthropology and neuroscience,
among others.
The theoretical goals of computational linguistics:
(a) formulation of grammatical and semantic frameworks to enable computationally
tractable implementations of syntactic and semantic analysis
(b) discovering processing techniques and learning principles that exploit both the
structural and statistic distributional properties of language
(c) developing cognitively and neuroscientifically plausible computational models of
how language processing and learning might occur in the brain.
The practical goals of the field are broad and varied; for examples:
(a) efficient text retrieval on some desired topic/search engines
(b) effective machine translation (MT)
(c) question answering (QA), ranging from simple factual questions to ones requiring
inference and descriptive or discursive answers (perhaps with justifications)
(d) text summarization
(e) analysis of texts or spoken language for topic, sentiment, or other psychological
attributes
(f) dialogue agents for accomplishing particular tasks (purchases, technical trouble
shooting, trip planning, schedule maintenance, medical advising, etc.)
(g) creation of computational systems with human-like competency in dialogue, in
acquiring language, and in gaining knowledge from text.
(h) Speech recognition systems
(i) Text-to-speech synthesizers
(j) Text-editors
(k) Language instruction materials
Computational Linguistics and knowledge representation are subdisciplines of
artificial intelligence (AI). AI, however, includes machine learning, or statistical
pattern recognition; historically referred to a variety of learning and inference
algorithms, many of them non-statistical, that were inspired as much by research in
psychology and cognitive science as by probability and information theory. That
was because the goals of early AI research centred around the development of
thinking machines. In the past the computer had to perform in a manner that
corresponded to human cognition. While there was a great deal of debate as to how
close and at what level that correspondence had to be. Therefore, a computational
linguist is an expert in machine learning, deep learning, AI, cognitive computing
and neuroscience.
Today a linguist needs a master’s or doctoral degree in a computer science-related
field or a bachelor's degree with work experience developing natural language
software. Tech software companies, such as Microsoft, typically hire
computational linguists to work on natural language processing (NLP), helping
programmers to create voice user interfaces that enable humans to communicate
with computing devices as if they were another person. Most of the time the terms
computational linguistics and NLP are used interchangeably.
Applications of CL typically include the following:
Machine translation. This is the process of using AI to translate one human
language to another.
Application clustering. This is the process of turning multiple computer servers
into a cluster.
Sentiment analysis. This approach to NLP identifies the emotional tone behind
a body of text.
Chatbots. These software or computer programs simulate human conversation
or chatter through text or voice interactions.
Knowledge extraction. This is the creation of knowledge from structured and
unstructured text.
Natural language interfaces. These are computer-human interfaces where
words, phrases or clauses act as user interface controls.
Content filtering. This process blocks various language-based web content from
reaching end users.
History of computational linguistics
Although the concept of computational linguistics is often associated with AI, CL
predates AI's development, according to the Association for Computational
Linguistics. One of the first instances of CL came from an attempt to translate text
from Russian to English. The thought was that computers could make systematic
calculations faster and more accurately than a person, so it would not take long to
process a language. However, the complexities found in languages were
underestimated, taking much more time and effort to develop a working program.
Two programs were developed in the early 1970s that had more complicated
syntax and semantic mapping rules. SHRDLU was a primary language parser
developed in 1971 by computer scientist Terry Winograd at MIT. SHRDLU
combined human linguistic models with reasoning methods. This was a major
accomplishment for natural language processing research.
Also in 1971, NASA developed Lunar and demonstrated it at a space convention.
The Lunar system answered convention attendees' questions about the composition
of the rocks returned from the Apollo moon missions.
Translating languages was a difficult task before this, as the system had to
understand grammar and the syntax in which words were used. Since then,
strategies to implement CL began moving away from procedural approaches to
ones that were more linguistic, understandable and modular. In the late 1980s,
computing processing power increased, which led to a shift to statistical methods
when considering CL. This is also around the time when corpus-based statistical
approaches were developed.
Modern CL relies on many of the same tools and processes as NLP. These systems
may use a variety of tools, including AI, ML, deep learning and cognitive
computing. As an example, GPT-3, or the third-generation Generative Pre-trained
Transformer, is a neural network machine learning model that produces text based
on user input. It was released by OpenAI in 2020 and was trained using internet
data to generate any type of text. The program requires a small amount of input
text to generate large relevant volumes of text. GPT-3 is a model with over 175
billion machine learning parameters. Compared to the largest trained language
model before this, Microsoft's Turing-NLG model only had 17 billion parameters.
