FIELD OF THE INVENTION The invention relates to an audio device for determining biometric characteristics of a user.

The invention further relates to a method of determining biometric characteristics of a user.

The invention further relates to a system for biometric identification. The invention further relates to a method for identification of a user on the basis of aural biometric characteristics of that user.

BACKGROUND OF THE INVENTION

Usually, the term "biometrics" is used to describe methods of recognizing a person based on physiological or behavioural characteristic of that person. Such a characteristic is referred to as a biometric, and can be the person's face, his fingerprints, hand geometry, handwriting, iris, retinal pattern, vein pattern, voice, etc. The use of human biometric features for identity verification and person recognition is well known. Biometric technologies are becoming the foundation of an extensive array of highly secure identification and personal verification solutions. The advantage of using biometrics for access control, rather than using ' other methods such as PIN (personal identification number) codes or identity cards, is that a biometric cannot be lost, given away, or stolen from the person it characterizes. However, not all biometric features are immune to being copied. For example, it is possible to make a copy of a fingerprint and to fraudulently use this copy to gain access to data otherwise only accessible to the true owner of the fingerprint. Furthermore, some types of biometric identification systems fail to gain general acceptance by the public. For example, a person may feel uncomfortable having to expose his eye to a retina or iris scanner.

It is known from prior art that the acoustical properties of the ear can be used to identify people uniquely. This kind of biometric feature cannot easily be copied, and can easily be implemented in a mobile phone for remote identification, thus replacing conventional, less reliable methods of identification such as ^' the PIN code. In the case of acoustic ear canal biometrics, what is of interest is the topology of the ear canal, which is unique for every human. An incoming sound signal is reflected and otherwise modified by the ear canal to give an aurally reflected signal which exits the ear canal. In US 5,787,187 a

sound signal is directed into the ear of a user, and the frequency response of the ear canal is measured and analysed to extract a feature vector unique to this user. However, since the microphone used to detect the response from the ear canal must also pick up any surrounding sound signals, such a measurement system is particularly prone to error owing to background noise. These unwanted background noise signals can really only be excluded from the measurement described by, for example, enclosing the microphone and the ear in headphones of a size large enough to encompass the entire ear. Since such headphones are generally cumbersome to use and awkward to transport, they are impractical for frequent use, and unsuited to user identification for applications such as telephone banking, telephone brokerage, etc, which a user generally wishes to carry out with a mobile phone, whether at home or underway.

SUMMARY OF THE INVENTION

Therefore, an object of the present invention is to provide a robust and user- friendly way of identifying a user on the basis of his aural biometric characteristics.

To this end, the present invention provides an audio device, comprising a loudspeaker for outputting a loudspeaker output signal close to the ear of a user of the audio device and a first microphone arranged in a manner suitable for recording a first microphone input signal including an aurally reflected signal component, an active noise cancellation circuit for . actively reducing any background noise component of the microphone input signal, and a biometric characteristics determination unit for determining the aural biometric characteristics of the user on the basis of the noise-reduced microphone input signal.

An appropriate method for determining biometric characteristics of a user according to the present invention comprises directing a loudspeaker signal into the ear of a user and recording the response as a microphone input signal. The microphone input signal will include other signal contributions besides the loudspeaker signal, such as an aurally reflected signal component contributed by reflections of the loudspeaker signal back out of the ear canal of the user. If the microphone also picks up any other sounds generated in the background - which detract from the quality of the microphone signal and are therefore collectively referred to as background noise - then, according to the invention, such a background noise signal component of the microphone input signal will be reduced by an active noise cancellation circuit. The aural biometric characteristics of the user are then determined using the resulting noise-reduced microphone input signal.

Using the invention, the audio device can be any type of audio device with a

loudspeaker which is intended for use close to the ear of the user, such as a telephone handset, a mobile telephone, a set of headphones, etc., and does not require special headphones to exclude background noise. The loudspeaker incorporated in or on the audio device can issue sound signals essentially directly into the ear canal of the user. Since the first microphone is intended to pick up the loudspeaker signal reflected back out of the ear canal - the aurally reflected signal - the microphone need only be positioned so that it is advantageously placed for picking up these reflected sound signals. Preferably, the microphone might be placed in the direct vicinity of the loudspeaker, but it may even be positioned within the circumference of the loudspeaker itself. Furthermore, using the invention, it is not necessary to use the audio device under ideal conditions. Any unwanted signal components of the microphone input signal - such as traffic noise, conversations being carried out in the vicinity, music etc. - detracting from the quality of the microphone input signal, and collectively referred to as a background noise signal, can essentially be "cancelled" by isolating the background noise signal, phase- inverting it and subtracting it from the loudspeaker output signal to give a noise-reduced microphone input signal. This technique is known as active noise cancellation (ANC). ANC circuits are already to be found in a variety of audio devices, e.g. mobile phones, where they are used to optimise the quality of the loudspeaker output signal, allowing the audio device to be used in a noisy environment, since the background noise is suppressed at the position of the user's ear.

According to the invention, this technique is used to yield a noise-reduced microphone input signal, now comprising essentially the loudspeaker output signal and the aurally reflected signal component, which is forwarded to the biometric characteristics determination unit. Here, with the aid of suitable filters, biometric characteristics of the user can be determined on the basis of the aurally reflected signal component of the microphone input signal, as will be explained in more detail below.

A particularly advantageous characteristic of the present invention is, therefore, that an audio device comprising the necessary loudspeaker, microphone, ANC circuit and filters can easily be also used for biometric user identification without requiring extensive additional hardware. For example, a mobile phone which incorporates an additional microphone in the vicinity of the loudspeaker might be used, whereby it is not necessary to ensure particularly favourable environmental conditions, since the presence of background noise becomes essentially irrelevant when the invention is applied.

The dependent claims and the subsequent description disclose particularly

advantageous embodiments and features of the invention.

A sound signal can comprise a number of frequency components, and can be described in the frequency domain by its frequency spectrum which shows to what extent each frequency is represented in the sound signal. When an acoustic signal is reflected or modified in some way by the surrounding environment, the resulting signal is called the response signal. In travelling from a source (loudspeaker) to a target (microphone) in a non- ideal environment, a sound signal suffers some degree of attenuation. Furthermore, the quality of the loudspeaker and microphone also influence the quality of the received signal. Some frequencies, typically the higher frequencies, will generally suffer more attenuation than others. The frequency response for a recorded response signal can be calculated by performing a Fourier transform on the recorded response signal. Typically, this is carried out digitally on a sampled response signal using a digital filter to perform a discrete Fourier transform, which can be further developed to derive a "feature vector" characterizing the source, target and surrounding environment. Attenuation effects of the microphone are also included in this feature vector. The net changes which the original signal undergoes from source to target can be expressed, generally in the frequency domain, using a "transfer function", which essentially describes the quality of signal transfer from source to target. For example, for a loudspeaker/microphone pair, the transfer function is termed the "loudspeaker to microphone transfer function", and takes into account the frequency response of the microphone, as well as the frequency response of the loudspeaker.

In the case of the ear canal, a sound signal directed into the ear canal will be reflected back out in an attenuated form. The amount of attenuation is unique to each ear canal, so that the feature vector of the frequency response of the ear canal can be used as a reliable biometric characteristic for that ear canal (and ultimately, for the user), and the transfer function from the loudspeaker to the ear is given by the loudspeaker to ear transfer function. Since the noise-reduced microphone input signal essentially consists of the loudspeaker output signal and the aurally reflected signal component, by processing the noise-reduced microphone input signal in a suitable manner it is possible to obtain the loudspeaker to ear canal transfer function, which comprises the biometric characteristics for this user. Regardless of the quality of the loudspeaker output signal, the feature vector calculated from the frequency response of the ear canal will always be the same for a particular user.

In a preferred embodiment of the invention, an adaptive filter arrangement might be used to process the noise-reduced microphone input signal and to determine the biometric characteristics for the user. The adaptive filter arrangement might comprise an analog or

digital filter for processing the loudspeaker signal, and an update part for modifying the filter, for example by modifying the coefficients of an adaptive digital filter or the inductive, capacitive, or resistive parts of an adaptive analog filter. The input signal to the update part might be obtained by subtracting the filtered loudspeaker output signal from the noise- reduced microphone input signal. An adaptive filter arrangement such as the type often used for acoustic echo cancellation (AEC) might be used. Therefore, the audio device can preferably make use of existing circuitry, otherwise used to perform acoustic echo cancellation in the audio device, to also calculate the biometric characteristics of the user's ear canal. In a particularly preferred embodiment of the invention, the adaptive filter arrangement - or biometric characteristics determination unit - of the audio device might be used to indirectly improve the quality of the microphone input signal by modifying the behaviour of the active noise cancellation circuit. Therefore, a control signal connection is preferably implemented between the biometric characteristics determination unit and the active noise cancellation circuit, such that a control signal generated by the biometric characteristics determination unit controls the behaviour of the active noise cancellation circuit. In this way, the noise reduction or cancellation function of the active noise cancellation circuit can be improved to give a better noise-reduced microphone input signal, which in turn provides for a more accurate estimate of the feature vector. In a preferred embodiment of the invention, the audio device might comprise a second ¹ microphone arranged in a manner suitable for picking up or recording a second microphone input signal component, such as a user speech signal, for transmission to a conversation partner. The audio device might therefore be used in the manner of a telephone handset, a mobile phone, a headset, etc. The user of such an audio device can listen to and speak to his conversation partner using the loudspeaker and second microphone respectively.

Since the second microphone might pick up the loudspeaker output signal as well as the desired speech signal component, a preferred embodiment of the invention provides for an acoustic echo cancellation arrangement for reducing any loudspeaker output signal component in the second microphone input signal. This improves the listening quality for the users of such audio devices.

Since the function of the biometric characteristics determination unit may effectively be the same as that of an acoustic echo cancellation circuit, it might be sufficient to implement just one such circuit in the audio device. The single acoustic echo cancellation circuit can be used to minimize the loudspeaker contribution to the second microphone input

signal while one of the conversation partners is talking, and it can also be used to determine the aural bio metric characteristics of the user, for example, during periods of silence. Therefore, in a particularly preferred embodiment of the invention, a switch is implemented for switching an acoustic echo cancellation arrangement between the first microphone input signal and the second microphone input signal. Switching between microphones can be effected once, for example when the user of the audio device initially establishes a connection between that audio device and a remote device, or at intervals, for example during periods of silence. In this way, even the speech signal of the conversation partner can be used in determination of the aural biometric characteristics of the user. The aural bio metric characteristics of the user are preferably calculated when the user himself is not speaking, since his voice would also be picked up by the microphone positioned close to his ear, and might have a detrimental effect on the performance of the biometric characteristics determination unit. Any suitable technique for determining whether or not the user is speaking can be applied to determine when to switch between the two microphones. The aural biometric characteristics of a user might be used to identify the user before granting him access to, for example, an online banking service operating from a remote device. To this end, the aural biometric characteristics might be transmitted for verification to a remote device. Since a user's aural biometric characteristics are to be used for identification and/or for granting him access to applications to which only he should have access, the aural biometric characteristics are preferably encrypted before being transmitted. Alternatively, the user's aural biometric characteristics might be stored locally in the audio device. _' For example, the aural biometric characteristics of the owner of a mobile phone might be stored locally in the phone, authorising only that person to use that phone, and thus offering an effective deterrent against mobile phone theft. If the audio device is being implemented as, for example, a mobile phone, a derivative of the second microphone signal can also be transmitted to a remote device or "far end". Here, the term "derivative" simply means that a microphone signal is not transmitted directly from the microphone, but after being processed in some way. The derivative of the second microphone signal is obtained by performing AEC on the user speech signal. The relative positions of the first and second microphones, and the fact that both microphones pick up the same background noise, albeit to different degrees, can be used to good effect in reducing the noise component of the speech signal. Since the second microphone is favourably positioned to pick up the user's speech, the second microphone signal, which is intended for transmission to a far end, will comprise a relatively greater

proportion of speech than of noise. On the other hand, the first microphone is positioned further away from the mouth, so that the first microphone signal will comprise a relatively greater proportion of noise. Therefore, in a further preferred embodiment, these two signals can be processed together in a signal conditioning unit to improve the signal-to-noise ratio, and therefore the quality, of the speech signal prior to transmission.

An appropriate method of identification of a user on the basis of aural biometric characteristics of that user comprises an enrolment step in which a reference set of aural biometric characteristics of a user are determined as described above, and stored in a storage device to enrol a user for access to an application. In a following identification step, an identification set of aural biometric characteristics of a user are determined as described above, and compared to the reference set of aural biometric characteristics. Here, the comparator may make use of known techniques of pattern recognition to find a reference set to match the identification set of aural biometric characteristics. Identification of the user succeeds if a discrepancy between the identification set of aural biometric characteristics and the reference set of aural biometric characteristics falls below a pre-defined confidence measure.

A "confidence measure" is essentially a measure of the accuracy of the system. For some applications, a very high level of accuracy is required, whereas a lower level might suffice for other applications. The accuracy of the system can be expressed in terms of false acceptance ratio (FAR), where a user is erroneously identified or accepted, and false rejection ratio (FRR), in which a user is erroneously rejected. The audio device, comparator and decision making unit might be realised as a single device. However, since the realisation of a comparator and/or decision making unit might require more space than might be available in a user-friendly audio device, these comparator and decision making units might be realised separately, at the same or at different locations.

A corresponding system for biometric user identification comprises an audio device as described above for determining an identification set of aural biometric characteristics of a user, and an interface or access unit for accessing a reference set of aural biometric characteristics of the user, determined and stored, for example, in a previous enrolment step. The system further comprises a comparator to compare or match the feature vector of the reference set to the feature vector of the identification set, and a decision making unit for deciding whether any discrepancy between the identification set of aural biometric characteristics and the reference set of aural biometric characteristics falls below a pre¬ defined confidence measure.

The storage device used for storage of the identification set of aural biometric characteristics can be, for example, a database on a computer, a SMART card, a SIM card, or any suitable memory device.

For example, the owner of a mobile phone might enrol himself as the authorised user for this mobile phone. To this end, he might initiate an enrolment step by which his aural biometric characteristics are calculated and stored for use in a later verification step, either locally on the SIM card of the mobile phone, or remotely after transmission of the aural biometric characteristics to a remote device or application. The owner of the mobile phone might wish to authorise a further person, or further persons, to use his mobile phone. He might do this by initiating an enrolment step and allowing the person to be authorised to hold the mobile phone to his ear, in order for his aural biometric characteristics to also be stored for use in a future verification step. The mobile phone might include software for checking or verifying whether a person using the phone is also one of the authorised persons. This verification might be carried out before a user of the mobile phone even makes a connection. Therefore, such an application of the invention offers very effective protection of mobile phone theft, since only the authorised persons will be able to use the mobile phone to make calls.

Other objects and features of the present invention will become apparent from the following detailed descriptions considered in conjunction with the accompanying drawings. It is to be understood, however, that the drawings are designed solely for the purposes of illustration and not as a definition of the limits of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will now be described in more detail, by way of example, with reference to the accompanying drawings, wherein:

Fig. 1 is a schematic diagram of an audio device according to a first embodiment of the invention,

Fig. 2 is a more detailed block diagram of the audio device according to the first embodiment of the invention, Fig. 3 is a block diagram of an audio device according to a second embodiment of the invention,

Fig. 4 is a block diagram of an audio device according to a third embodiment of the invention,

Fig. 5 is a block diagram of system for biometric user identification according to the

invention together with an application, and

Fig. 6 is a schematic diagram of a realisation of an audio device according to an embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

In the drawings, like numbers refer to like objects throughout.

Fig. 1 shows a simplified representation of an audio device 1, which might be a mobile phone, telephone handset, headset, etc. The audio device comprises a loudspeaker 2 and microphone 4 and receives an input signal S; which is ultimately to be converted to an acoustic signal S _L by the loudspeaker 2. The loudspeaker 2 is positioned close to the ear 3 of a user in such a way that the acoustic signal S _L , originating in the loudspeaker 2, is directed into the ear canal of the ear 3. For the sake of simplicity, the diagram does not show any output signals, which will be considered in more detail below.

The acoustic loudspeaker signal S _L is reflected back out of the ear canal in the form of an aurally reflected signal S _A - Both the loudspeaker signal S _L and aurally reflected signal S _A are picked up, or recorded, by the microphone 4. Furthermore, any other acoustic signals - besides the loudspeaker signal S _L and aurally reflected signal S _A — collectively regarded as a background noise signal S _N originating from a figurative source of noise 17, are also picked up by the microphone 4. A microphone input signal M therefore comprises a loudspeaker component, an aurally reflected component, and a background noise component.

In its simplest form, the audio device 1 comprises two signal processing blocks 5, 6. One of these, an active noise cancellation unit 5, serves to reduce the background noise component of the microphone signal M by effectively subtracting it from the input signal S; to give a modified loudspeaker output signal, which, when combined with the actual background noise, effectively cancels out the influence of the background noise as perceived by the user.

The other signal processing block, an adaptive filter arrangement 6, processes the loudspeaker and aurally reflected components of the microphone signal M to determine biometric characteristics of the ear canal, and ultimately of the user. Fig. 2 shows the composition of the active noise cancellation unit 5 and the adaptive filter arrangement 6 in more detail.

The active noise cancellation unit 5 comprises a transfer function block 7, which performs a transfer function Hi _s2m i _C on the input signal S;. The transfer function applied is that of the loudspeaker to microphone, giving effectively the same signal that would be recorded

by the microphone if no other signal components were picked up. The output of the transfer function block is subtracted from the microphone input signal M in a summation unit 15. The output of this summation unit 15, comprising basically everything but the loudspeaker signal S _L - i.e. the background noise S _N - is forwarded to an inverse transfer function block 8 and a subsequent low pass filter 9 to modify the noise component S _N , giving an "anti-noise" signal component which is subtracted from the incoming signal Sj to give a modified "anti-noise" version Si ¹ of the input signal Si. This "anti-noise" input signal S;' is converted by the loudspeaker 2 into an acoustic signal S _L that effectively cancels out the background noise, as might be perceived by a listener. The adaptive filter arrangement 6 comprises a filter unit 10, an update block 11 and a summation unit 16. The filter unit 10 receives the modified input signal S;', performs a further modification, and outputs a signal Sx. The summation unit 16 subtracts the signal S _x from the microphone input signal M and passes the result to the update block 11. On the basis of the summation result, the update block 11 continually adjusts the coefficients of the filter unit 10, until they arrive at a good estimate of the loudspeaker to microphone transfer function Hi _s2m i _c . This transfer function Hi _s2m i _C characterizes the aural biometric of the user, and can therefore i be used for reliable user identification. Techniques for deriving such transfer functions are known and described in the literature, for example in H. Moller et al., "Transfer Characteristics of Headphones measured on Human Ears", J. Audio Eng. Soc, Vol. 43, 4, 1995.

Since the filter 10 of the adaptive filter arrangement 6 is adapted continually until it approximates a filter performing the loudspeaker to microphone transfer function Hi _s2m i _C , and, on the other hand, a loudspeaker to microphone transfer function Hi _s2m j _C is required in the active noise cancellation unit 5, the adaptive filter arrangement 6 can preferably be used to improve the performance of the active noise cancellation unit 5. This is shown in Fig.3, which is basically the same as Fig.2 with an additional control signal 22. This control signal 22 "informs" the transfer function block 7 and the inverse transfer function block 8 of the active noise cancellation unit 5 of the "correct" filter performance. The transfer function block 7 and the inverse transfer function block 8 can then be modified accordingly, thereby improving the performance of the active noise cancellation unit 5. Generally the adaptive filter arrangement 6 is realized using digital components. These digital components can be used to control the ANC circuit 5 which is realized generally using analog components. The technique of using digital controllers to modify the performance of analog cancellation electronics is known, and has also been described in US 5,440,642.

Delays may be incurred by the elements of the active noise cancellation unit 5 and the adaptive filter arrangement 6 during signal processing for the determination of the aural biometric characteristics. To compensate for these. delays, appropriate delay elements 12, 13 can be inserted as required into the microphone output signal M before the summation units 15, 16.

The audio device 1 according to the invention can easily be realised in the form of a telephone handset, headset, or mobile phone, as illustrated in Fig. 4. Here, the configuration described in Fig. 2 and Fig. 3 is augmented by a second microphone 14 for picking up a speech signal Sv issuing from the 19 mouth of the user. The adaptive filter arrangement 6 can also be used to reduce unwanted echo effects — perceived as detrimental to the quality of a telephone conversation - by connecting it between the second microphone 14 and the loudspeaker 2. Since aural biometric characteristics are only calculated when the user is not speaking, both first microphone 4 and second microphone 14 can share the single adaptive filter arrangement 6. A switch 34 for switching one of the microphone signals through to the adaptive filter arrangement 6 can be software controlled, so that the first microphone 4 is routed to the adaptive filter arrangement 6 whenever the user is not talking, or when a connection is being set up, or at any other appropriate time. At any other time, the voice input signal My picked up by the second microphone 14 can be switched through to the adaptive filter arrangement 6 and modified to give a speech microphone output signal 37, free of any loudspeaker contribution.

The modified speech microphone output signal 37 is routed to a signal conditioning unit 30 before transmission as an output signal 38 to a far end, where it is used appropriately. For example, the output signal 38 might be transmitted to a remote audio device such as a telephone, and there converted into audible sound by the telephone loudspeaker. A data signal 36, comprising the aural biometric characteristics of the ear 3 of the user, might be sent to an appropriate analysis unit for performing the necessary steps of user enrolment and/or user verification. Such an analysis unit, not shown in the diagram, might be realised in the audio device, or externally, depending on the intended application and on device realisation constraints. Fig. 5 illustrates a system 40 for biometric user identification, where an audio device

1, comprising a loudspeaker 2 and microphone 4, is used to determine aural biometric characteristics for a user in the manner described above, on the basis of a loudspeaker signal S _L and the resulting aurally reflected signal S _A reflected from the ear canal of the user's ear 3. In a first enrolment step, a reference set A _ref of aural biometric characteristics is

determined and transmitted to a remote device 41, which might be in the vicinity of the audio device 1 or at a remote location. Transmission might be effected via a cable link, or in a wireless manner. The reference set A _re f of aural biometric characteristics is forwarded by an interface 42 to a suitable storage device 43 for storage. At a later point in time, in an identification step, an identification set Ai _d of aural biometric characteristics is generated in the audio device 1, and is also transmitted to the remote device 41, where it is forwarded to a comparator 44. The interface 42, operating in this step as an accessing unit 42, retrieves the reference set A _ref of aural biometric characteristics from the storage device 43 and forwards these to the comparator 44. The comparator 44 compares both sets Aid, A _re f of aural biometric characteristics using appropriate techniques of pattern matching or feature vector analysis. The results of the comparison are passed to a decision making unit 45, which determines whether the match is sufficiently close to warrant a positive identification of the user. The decision making unit 45 reports its decision- "accept" or "reject" - in the form of an appropriate identification signal 46 to the application to which the user is attempting to gain access.

The application can use the identification signal 46 to determine whether it should ^■ continue or abort. For example, a positively identified user, attempting to access a telephone brokerage service for which he has previously enrolled by allowing his reference set A _ref of aural biometric characteristics to be stored, is indicated by a corresponding "accept" identification signal 46. The telephone brokerage service can assume that the user is who he claims to be, and responds to the user's wishes, transmitted perhaps as spoken commands over the telephone connection. In the case of a negative decision of the decision making unit and a subsequent "reject" identification signal 46, the telephone brokerage service can report the failure to the user attempting to gain access, via the telephone connection, and terminate the connection.

Fig. 6 shows a possible embodiment of the audio device 1. Here, the audio device 1 comprises a typical mobile phone. The loudspeaker 2 is positioned so that it can be held close to the ear whilst maintaining a suitable distance between the speaker microphone 14 and the mouth of the user. A microphone 4 for use in active noise cancellation and biometric characteristic determination is positioned close to the loudspeaker 2. This illustration demonstrates that the functionality of the methods according to the invention can be realised in an audio device of the type commonly available.

Although the present invention has been disclosed in the form of preferred embodiments and variations thereon, it will be understood that numerous additional

modifications and variations could be made thereto without departing from the scope of the invention. For example, the filters used in the adaptive filter arrangement, acoustic echo cancellation circuit and active noise cancellation circuit can be analog filters, digital filters, or a combination of both. A reference set of aural biometric characteristics for use in user identification can be determined in a different manner to that described above.

For the sake of clarity, it is also to be understood that the use of "a" or "an" throughout this application does not exclude a plurality, and "comprising" does not exclude other steps or elements. A "unit" may comprise a number of blocks or devices, unless explicitly described as a single entity.