Solved – Explain “Curse of dimensionality” to a child

dimensionality reductionhigh-dimensionalmachine learning

I heard many times about curse of dimensionality, but somehow I'm still unable to grasp the idea, it's all foggy.

Can anyone explain this in the most intuitive way, as you would explain it to a child, so that I (and the others confused as I am) could understand this ones for good?

EDIT:

Now, let's say that the child somehow heard about clustering (for example, they know how to cluster their toys 🙂 ). How would the increase of dimensionality make the job of clustering their toys harder?

For example, they used to consider only the shape of the toy and the color of the toy (one-color toys), but now need to consider the size and the weight of toys also.
Why is it more difficult for the child to find similar toys?

EDIT 2

For the sake of discussion I need to clarify that by – "Why is it more difficult for the child to find similar toys" – I also mean why is the notion of distance lost in high-dimensional spaces?

Best Answer

Probably the kid will like to eat cookies, so let us assume that you have a whole truck with cookies having a different colour, a different shape, a different taste, a different price ...

If the kid has to choose but only take into account one characteristic e.g. the taste, then it has four possibilities: sweet, salt, sour, bitter, so the kid only has to try four cookies to find what (s)he likes most.

If the kid likes combinations of taste and colour, and there are 4 (I am rather optimistic here :-) ) different colours, then he already has to choose among 4x4 different types;

If he wants, in addition, to take into account the shape of the cookies and there are 5 different shapes then he will have to try 4x4x5=80 cookies

We could go on, but after eating all these cookies he might already have belly-ache ... before he can make his best choice :-) Apart from the belly-ache, it can get really difficult to remember the differences in the taste of each cookie.

As you can see (@Almo) most (all?) things become more complicated as the number of dimensions increases, this holds for adults, for computers and also for kids.

Related Question