PURPOSE: To allow accurate recovery of the three-dimensional shape of an object by determining the direction where the variation of feature amount of image is high, moving the viewpoint in that direction, and then determining a correcponding point accurately.

CONSTITUTION: An object is imaged, at an initial position thereof, by means of a television camera and a distributing condition in the direction where the variation of luminance is highest (direction of gradient angle) is determined for all pixels in order to prepare a histogram and then the moving direction of viewpoint is set in the direction corresponding to the peak value of the histogram. When the axis of a columnar object is set in the direction of x-axis, for example, the camera is moved along x-axis or y-axis and the object is imaged before and after movement thereof. In this regard, luminance varies in the direction of y-axis but does not vary in the direction of x-axis because of the columnar structure. Consequently, a corresponding point can be obtained by moving the viewpoint in the direction of y-axis. The camera is moved in the direction thus determined and the objected is imaged and then the image is compared with the image at initial position thus operating the depth of the corresponding point. This method allows accurate recovery of the three-dimensional shape of an object.