MODULE 4: PART 2
• Word Sense Disambiguation
• Importance in NLP
• Challenges
• Types of Word Sense Disambiguation Approaches
Word Sense Disambiguation
• Word Sense Disambiguation (WSD) is a fundamental task in Natural
Language Processing (NLP) that involves determining the correct meaning
(or sense) of a word based on the context in which it is used.
• This task is challenging because many words in a have multiple senses or
meanings.
Why WSD Is Important in NLP
• Machine Translation: WSD is critical for translating ambiguous words correctly in
different languages.
Example: The word "bank" must be translated differently depending on
whether it refers to a financial institution or the side of a river.
• Information Retrieval: Search engines need WSD to deliver relevant results
when a user searches for ambiguous words.
Example: A search for "apple" could refer to the fruit or the technology
company.
• Question Answering and Chatbots: Understanding the meaning of a word
is crucial for accurate responses.
• Speech Recognition and Summarization: WSD enhances the system’s
ability to produce coherent summaries or transcriptions by interpreting
words correctly in context.
Types of Word Sense Disambiguation Approaches
• WSD can be approached using several methods:
• 1. knowledge-based./ Dictionary Based – Lesk Algo
• 2. supervised.- Naïve Bayes Approach
• 3. unsupervised.-
• 4. semi-supervised.-
1. Knowledge-based./ Dictionary Based/Approach
Based
• These methods use external lexical resources such as WordNet, BabelNet, or
Wikipedia to assign senses to words. These resources contain predefined
senses and semantic relationships between words.
• One approach for disambiguation is :Lesk Algorithm
• Lesk Algorithm: One of the most famous knowledge-based algorithms, it
disambiguates a word by comparing the overlap of words in the context of
the ambiguous word with the definitions of each sense from a lexical
database like WordNet.
Here’s a step-by-step example of the Lesk Algorithm applied to
disambiguate the word "bark" in different contexts.
• Example Context:
• Sentence 1: "The dog began to bark loudly at the stranger."
• Sentence 2: "The tree's bark was rough and peeling."
• Step 1: Retrieve Senses
• Let's consider two possible senses of the word "bark" from a lexical
resource like WordNet:
[Link] 1: Bark (noun)
1. Definition: "The sound made by a dog."
[Link] 2: Bark (noun)
1. Definition: "The protective outer covering of the trunk, branches, and twigs of a
tree."
Step 2: Identify Context Words
• For both sentences, we identify the context words:
• Sentence 1 Context: "The dog began to bark loudly at the stranger."
• Context words: {the, dog, began, to, loudly, at, the, stranger}
• Sentence 2 Context: "The tree's bark was rough and peeling."
• Context words: {the, tree's, was, rough, and, peeling}
Step 3: Compute Overlap
• Now we compare the context words with the definitions of each sense:
• For Sentence 1:
[Link] 1 (dog's sound):
1. Definition: "The sound made by a dog."
2. Overlap with context words: {dog} (1 word)
[Link] 2 (tree's covering):
1. Definition: "The protective outer covering of the trunk, branches, and twigs of a tree."
2. Overlap with context words: {} (0 words)
• For Sentence 2:
[Link] 1 (dog's sound):
1. Definition: "The sound made by a dog."
2. Overlap with context words: {} (0 words)
[Link] 2 (tree's covering):
1. Definition: "The protective outer covering of the trunk, branches, and twigs
of a tree."
2. Overlap with context words: {tree} (1 word)
Step 4: Select the Sense
• For Sentence 1:
• Sense 1 (dog's sound) has 1 overlapping word.
• Sense 2 (tree's covering) has 0 overlapping words.
• Result: The algorithm selects Sense 1 for "bark" in Sentence 1.
• For Sentence 2:
• Sense 1 (dog's sound) has 0 overlapping words.
• Sense 2 (tree's covering) has 1 overlapping word.
• Result: The algorithm selects Sense 2 for "bark" in Sentence 2.
Summary of the Example
• In Sentence 1, the Lesk Algorithm disambiguates "bark" as the sound
made by a dog.
• In Sentence 2, it correctly identifies "bark" as the protective outer
covering of a tree.
• END
2. Supervised Naive Bayes Working
• Naive Bayes is a simple yet effective probabilistic classifier often used in
Natural Language Processing (NLP) for tasks involving disambiguation.
• In the context of disambiguation, Naive Bayes can help resolve
ambiguities such as word sense disambiguation, part-of-speech tagging,
and text classification.
Word Sense Disambiguation (WSD):
• Problem: A word may have multiple meanings, and the goal is to
determine the correct sense based on the context.
• Naive Bayes Solution: Treat the task as a classification problem, where the
input word is classified into one of its possible senses. Naive Bayes can
compute the probability of each sense given the surrounding words
(context).
• Formula:
P(s∣C)=P(C∣s)⋅P(s)P(C)
• Where:
• s = sense of the word
• C = context (neighbouring words)
• P(s∣C) = probability of sense given the context
• P(C∣s) = likelihood of the context given the sense
• P(s) = prior probability of the sense
• Let's walk through an example of how Naive Bayes can be used for word
sense disambiguation (WSD), a common task in NLP.
• Problem
• Suppose we want to disambiguate the word "bank" based on its context.
"Bank" could either mean:
• Bank (financial institution)
• Bank (riverbank)
We will use the following context sentence:
"I went to the bank to deposit money."
Step 1: Collect data and define features.
Let’s assume we have a dataset where occurrences of "bank" have been labeled with either "financial institution"
or "riverbank." From this, we extract context words and their frequency for each sense.
Sense: Financial
Context Word Sense: Riverbank
Institution
money 50 5
river 2 45
deposit 40 0
water 3 42
withdraw 30 1
boat 1 30
•
Let's assume from the dataset:
The sense "Financial Institution" appears in 60% of the cases.
The sense "Riverbank" appears in 40% of the cases.
Thus:
P(Financial_Institution)=0.6
P(Riverbank)=0.4
Step 4: Calculate the likelihood of the context words.
Given the context: "I went to the bank to deposit money."
We focus on the words "deposit" and "money" to compute the likelihood.
Sense: Financial
Context Word Sense: Riverbank
Institution
money 50 5
river 2 45
deposit 40 0
water 3 42
withdraw 30 1
boat 1 30
Sense: Financial
Context Word Sense: Riverbank
Institution
money 50 5
river 2 45
deposit 40 0
water 3 42
withdraw 30 1
boat 1 30
"I went to the bank to deposit money."
3. Un-superwised : Yarowsky WSD Approach
• The Yarowsky Algorithm is a well-known unsupervised learning method
for word sense disambiguation (WSD), proposed by David Yarowsky in
1995. The algorithm is based on two key linguistic principles:
• One Sense Per Collocation: A word tends to have the same
meaning when used in similar contexts.
• One Sense Per Discourse: A word is likely to have the same
meaning throughout a given discourse or text.
Yarowsky’s Algorithm
• Yarowsky’s algorithm is a semi-supervised learning technique used for
training supervised machine learning models with limited labeled data.
• This method follows bootstrapping approach which starts from seed
data.
• The algorithm leverages a small set of labeled instances, called seed
instances, and a much larger set of unlabeled instances to iteratively
improve the model’s performance. It relies on the assumption that
similar instances tend to have similar labels.
How Yarowsky’s Algorithm Works
To understand how Yarowsky’s algorithm works, let’s walk through the steps involved:
1. Select Seed Instances: Start by selecting a small set of labeled instances that
represent the different classes or categories you want to classify.
• These seed instances should be carefully chosen to cover the range of variations
present in the data.
2. Train Initial Model: Use the seed instances to train an initial model using any
supervised learning algorithm.
• This initial model will serve as the starting point for the iterative process.
3. Label Unlabeled Instances: Apply the initial model to the remaining unlabeled
instances and assign labels to them based on the model’s predictions.
4. Refine Model: Combine the labeled seed instances and the newly labeled instances
from the previous step to create an augmented training set.
Retrain the model using this augmented dataset
5. Iterate: Repeat steps 3 and 4 for a fixed number of iterations or until the model’s
performance reaches a satisfactory level. With each iteration, the model’s
predictions become more accurate as it learns from its own predictions.
6. Evaluate Model: Finally, evaluate the performance of the model on a separate
validation or test set to assess its effectiveness.
• One Sense Per Collocation
• "bank", which can mean either:
1.A financial institution (e.g., "I went to the bank to deposit money").
2.A riverbank (e.g., "We had a picnic by the river bank").
• Collocations: The hypothesis states that specific collocations (neighboring words or
phrases) are highly associated with a particular sense of the word.
• For the financial sense of "bank", collocations might include:
• "deposit money," "loan," "account," "branch," "ATM," "interest rates."
• For the riverbank sense, collocations might include:
• "river," "water," "shore," "sand," "fishing," "flow."
Unsupervised approach-Hyperlex
Disambiguation
