Fig. 40. §3 AI and Computing
Word Vectors
Word vectors turn each word into a list of numbers, a point in a space with many directions, so that words used in similar ways land near each other.
1Facts
- Shelf
- AI and Computing
- Kind
- Built from a recipe
- Your own
- Your own words
- Sound
- Yes
- Licenses
- PDDL
2How to play
Tap it to work out king − man + woman. Type your own words in the Splash tab.
3Its controls
Table I. Every control on this Splash.
| Control | Kind | Range | Start |
|---|---|---|---|
| A − B + C | Tap | ||
| Add | Button |
4About it
Word vectors turn each word into a list of numbers, a point in a space with many directions, so that words used in similar ways land near each other. These have 50 numbers per word, learned from a huge amount of text. Directions in the space can carry meaning: the step from man to woman is much like the step from king to queen.
Tap it: an arrow runs out to KING, the step from MAN to WOMAN is added on from there, and it lands right next to QUEEN, which lights up. Type your own A − B + C in the Splash tab, with any of 24,000 common words, and it finds the nearest of the 10,000 most common words, with the runner-up in gray. A text file containing A − B + C works too. These relationships are approximate and can reflect biases in the training text.
5Details
- Splats
- Opens at 200,000 at Mid Detail, 60,000 at Low, and up to 600,000 at Max
- Your own files
- .txt
6Is it right?
The asset is a quantized subset of Stanford’s 50-dimensional GloVe vectors. The analogy calculation really uses all 50 dimensions and finds queen for king − man + woman. The nearest-word score approximates cosine similarity, and the 3D picture preserves the arrows rather than all distances.
Correct.
These are named, real 50-dimensional word vectors.[1]
Rebuilding the asset from the first 40,000 rows of Stanford’s official GloVe 6B 50D release gives identical bytes. Four original source vectors are also checked directly.
Correct.
A − B + C is computed over all 50 dimensions.[1]
Independent dot-product rankings match queen/throne, rome/milan, large/one, and swimming/diving for four analogies. The input vocabulary has 24,000 entries and the answer search uses 10,000.
Close.
The nearest-word score approximates cosine similarity.[1]
Only the query is normalized after loading. Quantized candidate lengths range from 0.99153 to 1.00849, so their dot products can differ from cosine by less than 0.009.
Correct.
The 3D arrows preserve the analogy parallelogram.[1]
The linear projection gives sum = A − B + C in the displayed coordinates for all four checked queries.
Simplified.
Word relationships are approximate rather than universal truths.[1]
The retained vocabulary and training corpus determine the answers; queen is the nearest example, not an exact equality or a rule for all words.
- [1]GloVe: Global Vectors for Word Representation, Stanford NLP; Pennington, Socher, and Manning
- Simplified
- The source is filtered to 24,000 lowercase words and quantized to signed bytes. Answers search only the first 10,000 retained words and exclude the three inputs. The 3D projection rescales axes; it is not a map of every high-dimensional distance.
7Sources and credits
| Work | By | License |
|---|---|---|
| GloVe: Global Vectors for Word Representation (Wikipedia 2014 + Gigaword 5, 50 dimensions) | Jeffrey Pennington, Richard Socher and Christopher D. Manning | ODC Public Domain Dedication and License 1.0 |
