I've seen a bunch of examples of this artistic effect, but never knew what it was called. After some quick searching, I learned it is called the droste effect. The effect involves a "picture appearing within itself, in a place where a similar picture would realistically expected to appear". In theory it can go on forever, but in reality, it is limited by what can fit inside the image.

The effect is named after the image on the packaging of Droste brand cocoa powder. In it, a nurse is carrying a tray with a box of of the same image. It was introduced in 1904. You can see the original box here.

You can even download a Photoshop plugin that creates the effect for you.