One optical illusion that I find particularly compelling is the relatively new (newly discovered that is, the illusion was there all along of course) illusion of relative motion, first described by Pinna and Brelstaff. I'm sure you've already figured out how the illusion works, but here it goes: Fixate on the central dot in the image below and move your head slowly towards and away from the image. The circles will appear to rotate in opposite directions. (The example below is actually a variation by Gurnsey and colleagues. The original illusion uses geometrical shapes rather than Gabor patches, but the principle is the same. You can download a script to generate these type of images here, it's fun to play around with.)

How does this illusion work? To the best of my knowledge this is not entirely clear, although Pinna and Brelstaff as well Gurnsey and colleagues provide a discussion which goes some way towards an explanation.

One way to think about how this illusion might work is to consider what happens if the circles really move as they appear to do in the illusion. Let's consider the patch at the bottom of the inner circle (see (a) in the figure below, for clarity the patch is larger that the patches used in the illusion). If the circle rotates counter clockwise (as it appears to do if you move your head towards the image) the patch will move to the right (b).

Now let's consider what happens if you move your head towards the image (which induces the apparent rotation). Approaching the image will make the image expand on your retina. If we assume that you are fixating the central dot, this means a downwards shift for the patch at the bottom of the circle (c).

In the figure you see an orange circle which indicates the receptive field of a hypothetical neuron. That is, we assume that there is some neuron which 'sees' only the area inside the orange circle. You can see that the area inside the circle is the same after a rightwards shift (b) and a downwards shift (c). Therefore, our hypothetical neuron will not be able to distinguish a rightwards shift (due to rotation) from a downwards shift (which will occur if we approach the image). This may explain why we confuse a downwards shift with a rightwards shift and, thus, perceive the circles to rotate when we approach the image.

Is this really how the illusion works? I don't know and it must be noted that this does not explain why the illusion is so much stronger if there are two circles, rather than one. But whatever the explanation may be, it's definitely a cool illusion!

Update 29-09-2010: Added a better illustration

References