Embedding Algebra

Words are Coordinates: Modern AI models don't actually "understand" English. Instead, they read millions of books and articles, and map every word they see to a massive coordinate grid (the model we are using here uses 3,072 dimensions!). Words that frequently appear in similar contexts get clustered together in this vast space. This specific set of coordinates for a given word is called an Embedding.

What are the Dimensions? You can think of each of the 3,072 dimensions as representing a different abstract concept or trait—like gender, royalty, color, age, or sentiment. The AI figures out these traits on its own during training. Because human brains can't picture 3,072 dimensions, this lab uses a mathematical technique called UMAP to squash those thousands of dimensions down into 3D space so we can visualize the semantic clusters.

Vector Geometry: Because embeddings are just coordinates, we can perform standard geometry on them! The distance and direction between words encode their semantic relationships. For example, if you start at the coordinate for King, subtract the vector that represents Man (removing the concept of masculinity), and add the vector that represents Woman (adding femininity), you physically travel across the 3072-dimensional space and land almost exactly on the coordinate for Queen.

Cosine Similarity: When you click "Find Semantic Matches", we calculate the final target coordinate. Then, we ask the AI to brainstorm a list of candidate words. Finally, we measure the angle between the target vector and the candidate vectors—a metric known as Cosine Similarity. Words that point in the exact same mathematical direction have a similarity of 1.0 and are deemed a perfect semantic match!