Prevent Views attachment display showing up on anything other than the first page

I have a view (page display) showing thumbnail images in a paginated output. I've set an offset of '1' on the pager, so it hides the first image. I then added an attachment display to the view showing just one image, the first one, and displaying it as a larger thumbnail.

This all works nicely - the view shows a large first image, then a series of smaller images (e.g. ONE, two, three, four, etc.). However, when I click on page 2 of the view, the first image is still displayed (e.g. ONE, five, six, seven, etc.).

How do I remove the attachment display (or simply hide it) from all but the first page of the view?