Background of the invention Expressions and movements of the face and other parts of the body (ie."body language") can often demonstrate the psychological and physical state of a human patient. Facial expressions, for example, can be used to express emotion, pain, happiness, dissatisfaction and many other forms of communication between humans. Additionally, facial expressions may also indicate the emotional state of a human subject in addition to other physiological states such as brain function. Additional clues to psychological and physical state are provided by the patient's choice of words and mode of speaking.

A large number of individuals in society suffer from various forms of mood disorder.

For example, severe melancholic depression often results in the partial incapacitation of individuals. The treatment of mood disorders is a significant burden on the health system which often requires extensive interactions with patients with the associated significant cost. Further, there is often a high level of subjectivity in any assessment system which may mean that it is difficult to measure progress of patients over an extended period of time. The subjectivity of any assessment tends to interfere with its accuracy especially where the assessor is replaced over time and a subsequent assistance provider must rely on notes of assessments etc.

Body language and verbal cues are also vital in the diagnosis of other psychological disorders, drug-induced disorders, and physical disorders (eg. stroke, Parkinson's disease).

Summary of the invention It is an object of the invention to provide a system and method which efficiently assesses a class of disorders in a human subject. These disorders include mood disorders, other psychological disorders, drug-induced disorders and physical disorders.

In accordance with a first aspect of the present invention there is provided a method of determining a disorder measure in a patient comprising the steps of (a) recording temporal data associated with the patient over a predetermined interval; (b) computer processing the temporal data to determine a series of indicator measures associated with the data ; and (c) comparing the indicator measures with those of other patients so as to determine a the mood disorder measure associated with the patient.

The temporal data ideally includes a visual and audio interview with the patient.

According to a further aspect of the present invention, there is provided a method to identify a disorder in a human subject comprising the steps of: recording a series of visual images of a selected body part of a human subject sequentially taken over a predetermined time period; analysing the series of recorded visual images to determine the degree of change to the selected body part over the predetermined time period; comparing the image changes of the selected body part with pre-recorded data to determine whether or not the human subject suffers from a mood disorder.

Optionally the method of the invention may further include the step of using the first visual image of the series of visual images to correlate the position of the body part of the human subject with the body part displayed in the other visual images of the series. This allows the changes in the movement of the body part to be tracked throughout the series of images taken over a given time period.

In one embodiment of the invention, the position of an array of tracking points of the body part are recorded in the first image of the series and the displacement of the array of tracking points from the first image in each subsequent visual image of the series, may be recorded. The degree of displacement of the body part can be recorded as data and compared with the data from a human subject who does not suffer from a mood or emotional disorder (such as for example, melancholic depression), to thereby identify that whether the human subject suffers from the mood or emotional disorder.

The selected body part may be the facial area of a human subject and may include the central facial features of the eyes, mouth and nose.

In another embodiment of the invention, in order to record the changes of the body part of the human subject over a given time period, a tracking area may be selected for the first image and the brightness of the tracking area can then be recorded for that image and each corresponding tracking area in each subsequent visual image of the series. The series of visual images may be pixel images represented in a computer graphic display and the brightness can then be determined by counting the number of non-dark pixels in the tracking area compared to that of dark pixels.

According to a second aspect of the present invention, there is provided a system to identify a mood disorder in a human subject comprising: a visual data recording means to record a series of visual data images of a selected body part of a human subject sequentially taken over a predetermined time period; a data processing means capable of receiving the visual data images from the visual data recording means, the data processing means capable of analysing the series of recorded visual data images to determine the degree of change to the selected body part over the predetermined time period; and a comparative data means in data communication with the data processing means, the comparative data means capable of comparing the image changes of the selected body part with pre-recorded data to determine whether or not the human subject suffers from a mood disorder.

Where in the description of the specification the word"comprising"or"comprised"is used, unless otherwise stated explicitly, the word is to be interpreted inclusively rather than exclusively.

The preferred embodiment can be adapted to be a diagnostic aid in the treatment of other psychological disorders, drug-induced disorders, and physical disorders such as stroke and Parkinson's disease.

Brief description of the drawings Notwithstanding any other forms which may fall within the scope of the present invention, preferred forms of the invention will now be described, by way of example only, with reference to the accompanying drawings in which: Fig. 1 illustrates a flow chart of the steps of the preferred embodiment; Fig. 2 is a graphical plot of data showing the facial displacement of a normal patient compared to a patient with melancholic depression, over a series of visual frame images; Fig. 3 is a graphical plot of data showing the facial brightness changes of a normal patient compared to a patient with melancholic depression, over a series of visual frame images; and Fig. 4 is a graphical plot of data showing a grid deformation measure of a normal patient compared to a patient with melancholic depression, over a series of visual frame images.

Detailed description of the embodiments An embodiment of the invention provides a method and system for identify melancholic depression in a human subject. A series of visual images of a facial area of a patient are firstly recorded for a predetermined time period and the data stored in a computer so that a software application program can analyse visual images of the facial area, to thereby determine the changes which take place to the facial area over the predetermined time period. A comparison is then made with the facial changes of the facial area of the patient, with that of pre-recorded data to determine whether or not the patient could possibly suffers from a mood disorder.

Therefore the steps involved in the preferred embodiment are illustrated in Fig. 1 with a first step being the capture of a series of video images of a patient 1, followed by a temporal processing of the facial portions of the images 2 to derive various measures, followed by the outputting of indicative values 3.

The embodiment of the invention has been applied to the mood disorder of depression and in particular to a sub-type of depression known as melancholic depression. The embodiment of the invention provides an assessment of psychomotor changes associated with melancholic depression in a human patient. That there are several physical changes associated with melancholic depression which includes slow movement, postural slumping and relative immobility of the face and body. These physical changes are used to identify melancholic depression utilising computer automated methods.

The method was developed and tested on six sequences of video taken from human patients. The video sequence was captured at a rate of twelve frames per second with a duration up to ten seconds for each sequence. Each sequence consisted of images of a seated subject with pose varying from looking straight at the camera to looking at approximately 45 degrees from the direction of the camera.

Each of the pictures for each sequence were recorded as data images by a digital camera and the data images input to a file stored on the hard drive of a computer (not shown).

Tracking After the data has been stored in the computer, an assessment was made of the series of data images for each sequence so as to determine the degree of the facial mobility and changes of the patient. This involves tracking the face location through the video sequence. The data images therefore represent a series of image frames which represent the visual images of the patient's face.

One method of face tracking can be as follows: Initially, the tracking in the first frame was achieved by placing a virtual box around the central facial features constituting the eyes and mouth to thereby define a tracking area of the patient. Correlation techniques are then used to track the virtual box location through the subsequent frames by keeping the box size of the sequence constant for each data image in the sequence.

In a first automated measurement facial motion was measured from frame to frame of each sequence by measuring the frame displacement of the tracked face region after correlation processing. Fig. 2 illustrates the resultant displacement of the facial region for each frame which is plotted graphically. The changes are computed with a two frame lag, that is a frame rate of three frames per second.

The plot of Fig. 2 shows the frame number of a frame sequence plotted against the facial displacement of facial features of two sequences, the solid line representing a normal facial displacement (the data being stored in a data base on the computer) and the other plot showing the case for severe facial immobility. The facial motion measure as can be seen in Fig. 2, is significantly higher for the normal sequence than for the immobile sequence of the patient suffering from melancholic depression. Therefore this measure represents making a comparison of these two groups of data it is possible to infer whether a patient is likely to be suffering from melancholic depressions.

Another way to identify facial changes in a sequence of visual frames is to measure the "monitor brightness changes"within the tracked face region on a computer display. One method to monitor the brightness changes within the virtual box is to sum the magnitude of the pixel differences over the area defined by the virtual box. To reduce the noise levels which may be inherent in this method, an alternative brightness monitoring method was developed. This method proceeded by differencing each facial image from a previous image so as to produce a different map image, calculating the standard deviation of the difference map image and counting the number of difference map pixels that are more than 4 standard deviations from zero. The count is then normalised by dividing the area of the face virtual box.

Fig. 3 illustrated a plot of the frame to frame brightness changes using this method for the same sequences used in Fig. 2. The changes are computed with a two frame lag. It can be seen that the sequence of the normal subject shows significantly larger facial brightness changes then the sequence of the severally impaired subject which suffers from melancholic depression and consequently it is possible to identify that the patient suffers from melancholic depression.

A third method for measuring facial activity involves using a series of 5x5 grid points placed on the first of a pair of visual images which are then tracked by correlation to the second visual image within the sequence of images. Constraints are placed on the grid points to ensure that the basic grid structure is retained in the tracked image and points don't"cross-over"other grid lines.

Fig. 4 shows a plot of the amount of grid deformation against frame number using the same data as used in the groups of Figs. 2 and 3. Again there is a clear distinction between the melancholic depression sufferer and the non-sufferer.

A Composite Measure In order to get a simple composite summary of the performance of the proposed techniques on a series of test sequences, the mean value of each measure for a given sequence was calculated to produce a sequence score by taking the mean of the three methods. The scores for each sequence are given in Table 1 below.

TABLE 1 Sequence CORE rating (Facial Sequence Score Immobility) 1 0 2. 8 2 3 1. 2 3 2 1. 5 4 2 1. 3 5 0 2. S 6 0 2. 7 As can be seen from Table 1, the summary scores for each video sequence shows a strong inverse relationship to the core assessments for facial immobility.

As will be appreciated the present embodiment provides a means of identifying a mood disorder in a human subject. This embodiment could be implemented for measuring other body parts of a human subject other than the facial region of the tracking box which includes the eyes, nose and mouth portion of a human subject. For example other tracking areas may include body motion, hand motion, posture, gate, gaze. Furthermore, in other embodiments of the invention, the speaking patterns of the human subjects may be combined with the visual data to determine mood disorders.

This embodiment provides a visual method and system for identifying mood disorders and may be implemented in clinical rooms, internet telepsychiatry, research environments and in hospitals.

It would be appreciated by a person skilled in the art that numerous variations and/or modifications may be made to the present invention as shown in the specific embodiments without departing from the spirit or scope of the invention as broadly described. The present embodiments are therefore, to be considered in all respects to be illustrative and not restrictive.

It would also be appreciated by a person skilled in the art that the technology described in this invention could be adapted to be a diagnostic aid in the treatment of other psychological disorders, drug-induced disorders, and physical disorders such as stroke and Parkinson's disease.