# Big Dimensions, and What You Can Do About It

DevFeed: [Big Dimensions, and What You Can Do About It](<https://devfeed.tech/articles/big-dimensions-and-what-you-can-do-about-it-40396.md>)

Original publisher: [Read original article](<https://www.jeremykun.com/2016/02/08/big-dimensions-and-what-you-can-do-about-it/>)

Published: 2016-02-08T10:00:14Z

Content type: article

Language: en

Sources: [Jeremy Kun](<https://devfeed.tech/sources/jeremy-kun.md>)

Topics: [data](<https://devfeed.tech/topics/data.md>), [math](<https://devfeed.tech/topics/math.md>), [clustering](<https://devfeed.tech/topics/clustering.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Algorithms, Complexity](<https://devfeed.tech/topics/algorithms-complexity.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [clustering](<https://devfeed.tech/tags/clustering.md>), [data](<https://devfeed.tech/tags/data.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [math](<https://devfeed.tech/tags/math.md>), [techniques](<https://devfeed.tech/tags/techniques.md>)

## AI overview

The article uses movie-title data to illustrate the curse of dimensionality: as vector dimensions grow, geometric intuition and clustering algorithms can perform poorly. It introduces dimension reduction as a way to condense high-dimensional data while retaining important geometric properties, and discusses embeddings and their mathematical guarantees.

## Source excerpt

Data is abundant, data is big, and big is a problem. Let me start with an example. Let's say you have a list of movie titles and you want to learn their genre: romance, action, drama, etc. And maybe in this scenario IMDB doesn't exist so you can't scrape the answer. Well, the title alone is almost never enough information. One nice way to get more data is to do the following: