Distributional semantics
Download as PPTX, PDF0 likes517 views
Distributional semantics is a research area that uses statistical analysis of linguistic contexts to develop theories and methods for determining the semantic similarities between words and linguistic items based on their distributional properties in large text corpora. It is based on the distributional hypothesis that words with similar distributions have similar meanings. Distributional semantic models represent words as vectors in a high-dimensional semantic space based on their co-occurrence with other words, allowing semantic similarity to be measured using vector similarity methods. Common distributional semantic models include term frequency-inverse document frequency (tf-idf), latent semantic analysis (LSA), latent Dirichlet allocation (LDA), and word embeddings.
Distributional semantics
- 2. Clustering maps of biomedical articles
- 3. Distributional Semantics Distributional semantics is a research area that develops and studies theories and methods for quantifying and categorizing semantic similarities between linguistic items based on their distributional properties in large samples of language data. Distributional hypothesis: linguistic items with similar distributions have similar meaning a word is characterized by the company it keeps" was popularized by Firth
- 4. Distributional Semantics ❖ Depends on Statistical Semantics statistical semantics applies the methods of statistics to the problem of determining the meaning of words or phrases, ideally through unsupervised learning, to a degree of precision at least sufficient for the purpose of information retrieval.
- 5. Theory Distributional semantics favor the use of linear algebra as computational tool and representational framework. The basic approach is to collect distributional information in high-dimensional vectors, and to define distributional/semantic similarity in terms of vector similarity.
- 6. Basics Theory of NLP Corpus: A corpus is a large body of natural language text used for accumulating statistics on natural language text. Lexicons: A collection of information about the words of a language about the lexical categories to which they belong. A lexicon is usually structured as a collection of lexical entries, like ("pig" N V ADJ). "pig" is familiar as a N, but also occurs as a verb ("Jane pigged herself on pizza") and an adjective, in the phrase "pig iron".
- 7. Distributional semantic models (DSMs) ❖ Idea of using corpus-based statistics to extract information about semantic properties of words and other linguistic units is extremely common in computational linguistics. ❖ 1) Word-Doc Distribution ,2) Topic-Doc Distribution 3) Word distribution is a semantic Space. Usecase: Information retrieval , document clustering, document quick understanding
- 8. Distributional Semantics Models ❖ Term frequency–Inverse document frequency(tf-idf) ❖ Latent Semantic Analysis(LSA) ❖ Latent Dirichlet Allocation (LDA) ❖ WordEmbedding (word2vec)
- 9. Distributional Semantics Models ❖ Term frequency–Inverse document frequency(tf-idf) Term Frequency: The number of times a term occurs in a document is called its term frequency. Inverse document frequency: An inverse document frequency factor is incorporated which diminishes the weight of terms that occur very frequently in the document set and increases the weight of terms that occur rarely.
- 10. Distributional Semantics Models the idf of a rare term is high, whereas the idf of a frequent term is likely to be low.
- 12. Distributional Semantics Models ❖ Latent Semantic Analysis(LSA) - Dimensionality Reduction - Finding latent relationship between words and documents - Words and documents are sorted by their relationship
- 13. Distributional Semantics Models ❖ Latent Semantic Analysis(LSA) Dimensionality Reduction Reduce the target-word-by-context matrix to a lower dimensionality matrix (a matrix with less – linearly independent – columns/dimensions). Two main reasons: 1)Smoothing: capture “latent dimensions” that generalize over sparser surface dimensions (Singular Value Decomposition or SVD) 2)Efficiency/space: sometimes the matrix is so large that you don’t even want to construct it explicitly (Random Indexing)
- 14. Distributional Semantics Models ❖ Latent Semantic Analysis(LSA) Where the animation video !!
- 16. Distributional Semantics Models Ranking System Design: Query => {terms} Docs={term-document,term-concepts} Query on Docs => Finding relevant docs.
- 17. Thank you