Show simple item record

dc.contributor.advisor Baraniuk, Richard G.
dc.creatorSonkar, Shashank
dc.date.accessioned 2020-09-03T14:00:35Z
dc.date.available 2020-09-03T14:00:35Z
dc.date.created 2020-12
dc.date.issued 2020-09-02
dc.date.submitted December 2020
dc.identifier.citation Sonkar, Shashank. "AWE: Attention Word Embedding." (2020) Master’s Thesis, Rice University. https://hdl.handle.net/1911/109309.
dc.identifier.urihttps://hdl.handle.net/1911/109309
dc.description.abstract Word embedding models learn semantically rich vector representations of words and are widely used to initialize natural processing language (NLP) models. The popular continuous bag-of-words (CBOW) model of word2vec learns a vector embedding by masking a given word in a sentence and then using the other words as a context to predict it. A limitation of CBOW is that it equally weights the context words when making a prediction, which is inefficient, since some words have higher predictive value than others. We tackle this inefficiency by introducing the Attention Word Embedding (AWE) model, which integrates the attention mechanism into the CBOW model. We also propose AWE-S, which incorporates subword information. We demonstrate that AWE and AWE-S outperform the state-of-the-art word embedding models both on a variety of word similarity datasets and when used for initialization of NLP models.
dc.format.mimetype application/pdf
dc.language.iso eng
dc.subjectNatural Language Processing
Machine Learning
Word Embeddings
dc.title AWE: Attention Word Embedding
dc.type Thesis
dc.date.updated 2020-09-03T14:00:35Z
dc.type.material Text
thesis.degree.department Electrical and Computer Engineering
thesis.degree.discipline Engineering
thesis.degree.grantor Rice University
thesis.degree.level Masters
thesis.degree.name Master of Science


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record