To provide a device and a program for retrieving a pattern image capable of retrieving an image highly accurately at high speed by conducting the retrieval using a feature quantity in consideration of a predetermined pattern which is a characteristic of the pattern image.

An original image data 120a of a pattern image and an image data 120b of its replication are overlapped for extracting a predetermined pattern image constituting the pattern image in the pattern image retrieval. While the data are shifted in the directions of X and Y, a difference average of brightness in the overlapped part 120 of the two image data is calculated. A size of the pattern image region is decided using a minimum value of the amount of shift showing a minimum value of the difference average from the distribution of the difference average to the amount of shift. Based on the decided size, the image preferably independent of the original image data is extracted as a pattern image.