Lip reading technology to combat crime

The CCTV camera systems that span the UK can already distinguish gesture and even talk directly to people in the street, but one research group hope to add the ability to read lips to these camera’s capabilities.

The idea is to enable operators to look closely at a persons lips and pick out groups of sensitive words that if detected would trigger an automatic alert message. Officers could then be dispatched to the scene.

The three-year project is headed by Richard Harvey, a senior lecturer in computer vision at the University of East Anglia in Norwich. Government security agencies are interested in the technology as is the Engineering and Sciences Research Council that has awarded the project a grant of over £391,000.

Mr Harvey commented: “We have several methods for extracting what are called ‘features’ from the lips—sets of numbers that vary with the lip shape, but not with anything else. Our technology is very standard. We are using standard speech recognition technology.”

The advances that Mr Harvey’s team hope to make would improve the conditions that can enable cameras to make an accurate assessment of the lip movements of suspects.

“We can lip-read between 10 and 30 utterances at the moment, with an accuracy of around 50 per cent. Given the difficulty of lip-reading that is regarded as pretty good. But obviously there is a huge way to go before we can handle natural speech,” continued Mr Harvey.

If the research project is successful, this technology could not only find its way onto the CCTV network, but may also be used in consumer and business products. If applied to mobile phones with a camera for instance, anyone could communicate in very noisy environments.