To extract only an object that becomes a foreground image with high accuracy even when an input image changes by an imaging state.

A background image storage part 29 stores a reference background image. An object detecting part 25 detects an object from a pickup image picked up by an imaging part 21 and estimates a detected object mask. A background difference image generation part 22 generates a background difference image consisting of a difference between pixels of the pickup image and the reference background image. A feature quantity conversion equation calculation part 84 calculates a conversion equation color consisting of a relationship equation of pixel values between the reference background image and pixels corresponding to the background difference image excluding a region of the object mask. A color conversion part converts the pixel values of the reference background image on the basis of the color conversion equation and generates a color conversion update image. A background image update part 27 performs replacement by the color conversion update image and updates the reference background image. The present invention is applicable to an image processing apparatus.