Human Visual System: Caucasians and Asians recognize faces and expressions differently

World Science (an online science news site) summarizes work by researchers at the University of Montreal (Canada) into how Asians and Caucasians differ in recognizing faces and facial expressions.  Eye tracking cameras were used to monitor where the subjects were gazing.  Doctoral candidate and researcher Caroline Blais said that

The study confirmed that Caucasians study the triangle of the eyes and mouth, while Asians focus on the nose..

 In a separate study, she also tested how recognition of facial expressions differed between the two ethnic groups. It turned out that Asians were not as accurate at recognizing emotions that required observing the mouth, such as fear, disgust, and anger.

The two studies were published in the journals Current Biology and PLoS One.

Evidence: Listen to the raw audio evidence

If you would care to listen to the audio and read the transcripts of the Governor Blagojevich (IL, USA) wiretaps, here is the link.

Image Recognition: Facial and license plate recognition news items

There are two on-line news articles related to image recognition that I would like to bring to your attention.

The first news article is about law enforcement in Tacoma, Washington, USA using a facial recognition package to match 16 years worth of prisoner mug shots with pictures taken by ATM (Automated Teller Machines) in a forgery and theft investigation to generate the lead needed to solve the case.

The second news article is from New Orleans, Louisiana, USA where local law enforcement used a license-plate recognition system to make 20 arrests and recover 23 stolen vehicles and license plates in a 25 day period.

The common thread here is, of course, automatic image recognition. These software algorithms have come a long way in the last decade. However, one should understand that the conditions are partially or completely controlled in both of these applications - i.e. distance, lighting, exposure, aspect (turned toward the camera), and (possibly) compression all fall within acceptable boundaries. In addition, with the license plate recognition problem, the character set and fonts were known in advance. The controlled conditions and a priori knowledge significantly increase the accuracy of the results tremendously and the chances of a successful investigation and prosecution.

(Hat-tip to JUSTNETNews, USA - I highly recommend this free service of the National Law Enforcement and Corrections Technology Center, or NLECTC, which is part of the National Institute of Justice, USA)

Legal: Recording conversations in the USA

Michael Flynn's podcast, Legal Lad's Quick and Dirty Tips for a More Lawful Life, gives an excellent overview of the legalities of making an audio recording of someone in the USA when you are not a government agency performing lawful surveillance. When is it legal and when is it not? The short answer is: it depends - on whether there was a reasonable expectation of privacy, what state the conversation takes place in, and whether all parties being recorded gave consent.

For instance, if one is having a loud conversation on the street (a public place), how can one expect privacy from being overheard? If one can be overheard, how can he then complain that a record (or recording) was made of what he said? An example from daily life, that we can all probably identify with, is the standard phrase we hear when we dial technical support or customer service - "All calls are recorded for quality assurance." In other words - "We're recording you - you've been warned". By being advised and not hanging up, you will likely been seen as having given implied consent.

Legal Lad, being a lawyer as well as having good communication skills, gives a more thorough and educational overview of this subject.

Enjoy!

CCTV: Work continues on automating CCTV surveillance

Wired News has an article about ongoing research at Ohio State University (USA) in automating surveillance using a network of CCTV cameras. Similar research and product development is occurring at many locations around the world, so I suggest viewing this article as a sample of where things are heading with CCTV technology.

Image Recognition: Military requirements push technology forward

The Strategy Page has an article about how the proliferation of video on the battlefield (e.g. from surveillance cameras and even night vision goggles) is driving technology development. Initially, it was the UK (United Kingdom) that drove image recognition technology with its massive deployment of CCTV in public spaces (e.g. train stations, airports, city centers, and shopping malls). Now, it is the military.

The benefit from computer-assistance in analyzing video comes from the following:

1. Volume (i.e. the sheer number of cameras and, hence, images to be monitored and/or analyzed)
2. Concentration (i.e. the human visual system loses the ability to concentrate effectively on images after about 20 minutes of continuous viewing)
3. Memory (i.e. computers can track and, possibly, predict more things simultaneously than a human can because, generally speaking, computers are not nearly as attention and working-memory limited in the short-term as humans and far outstrip us in their ability to recall video sequences over the long term - they can just play back the video recording off of their hard disks)
   

The article claims that the abilities of computer systems to recognize patterns is rapidly improving and approaching that of humans - that is no small feat as humans are simply amazing in their ability to perform real-time pattern analysis.

One intriguing point made in the article is that the current conflicts are generating a lot of real-world recordings of "bad behavior" that can be used to train and test new pattern analysis and prediction algorithms - training data like this is like gold to us signal processing types!

Enjoy!

Acoustics: Listenging to nature

Using microphones to capture the sounds made by animals in their habitat, minus humans and their man-made sounds. These sounds are known as the "biophony" and are used to sense the quality and character of the animals' habitat. Wired has the story.

Author's comment: It's a good thing that animals don't have privacy rights in the USA (yet!), or these scientists would be in jail for unlawful surveillance activities!

(Via Cross-Spectrum Acoustics)

Biometrics: Fooling an age verification camera

Of course, this comes as no big surprise, but it is amusing - a news reporter in Japan used a photograph clipped from a magazine to fool an age verification camera system on a cigarette machine. Link to the Daily Irrelevant post.

Image Recognition: "Computer, where did I lay the keys?"

The Daily Mail (UK tabloid newspaper) has a gadget article about Smart Goggles, a human wearable image recognition and recall system built into a set of glasses. The user trains the system by focusing its built-in camera on a series of objects, such as car keys, CDs, etc. while speaking the appropriate name of the object. Once trained sufficiently, the system automatically recognizes images it sees and stores the information for later retrieval, again by spoken word. The processor (computer) is worn on the user's back. The system was developed by Professor Kuniyoshi and colleagues at the University of Tokyo.

It isn't much of a stretch to see how these same concepts could be built into other video applications, such as CCTV systems, for instance. Don't be surprised if they don't work this into a future episode of CSI or similar television show (or the next James Bond movie, for that matter).

Miscellaneous: List of links Part 2

How the brain makes a whole out of parts

Use your iPhone to get the name of a song using "Listen"

Humans ARE actually better with one sense than most mammals (namely, fine sound frequency sensitivity)

Adobe Photoshop Elements 6 for Mac is about to be released (and it includes some very neat features for a consumer-grade product, such as Guided Editing and Photomerge)

Telephone companies cancel FBI wire taps over unpaid bills (Not to be a cynic and nothing in particular against The Register, other news outlets, or governmental bodies, but don't be surprised if this hasn't been overblown a bit for political or news making purposes.)

Create professional 3D photographic images in your office

LANL researchers develop low power MRI system (LANL = Los Alamos National Laboratory, USA)

Military: Digital camouflage- now on warplanes

I've posted before about digital camouflage being used on military uniforms.  Now, StrategyPage tells us about how it is being used on warplanes, at least on the tops of the airplanes. On the bottoms, which would be seen by people looking up at the sky, they are still using a sky-colored monochrome.  Interestingly, there are multiple patterns for the top sides to match different terrains, such as arctic and desert.

Image Recognition: License plate scanners

The technology manufacturing cycle is at work with license plate (tag) scanners - decreasing prices lead to more sales and deployments, which lead to further cost decreases and so on.  We've all seen it with personal computers, cell/mobile phones, DVD players, flat screen displays, and a multitude of other devices.  Now this market force is at work with license plate scanners , which are now being deployed on police cars.  The scanners are automating what was before a completely manual process - namely running stolen tag numbers and such - and doing it much faster.

Military: Digital Camouflage Continues to Catch on

Strategy Page has an article about how the Chinese are the latest to adopt digital camouflage patterns for their combat uniforms. Digital camouflage differs from traditional color blotch patterns in that it looks more "pixelized." The digital patterns hide the wearer better against human observers as well as, get this, night vision gear.

Surveillance: Stop sign cameras

It is time to add a new type of "safety camera" to the list - a stop sign camera. The interesting thing about this system is that it detects if the car comes to a complete stop or not. Here is a brief news article about a deployment in California, USA.

Video: Using night vision gear to nab movie pirates

The Reuters Oddly Enough section has an article on how Malaysian cinema staff are catching people video taping new movies in the cinema (presumably to sell pirated copies of). How are they doing it? With night vision googles...

Surveillance: Debate over CCTV cameras - adding audio and concealing cameras

The Daily Telegraph (UK, center-right newspaper) has an article about a political debate in the UK over whether microphones should be added to CCTV camera installations and whether there should be CCTV cameras that were no obvious displayed or, in other words, hidden.

This has to be a quick posting - unfortunately I don't have time for a fully researched and referenced article, but I'd like to point out a few things from completely different aspects.

The first is from a policy angle. According to the article, the UK has about 4.5 million CCTV cameras. Most of these camera systems are not run by governmental bodies but instead by companies and the like, if I understand things correctly. (It should be noted that the British public supports the use of CCTV)

Adding microphones to CCTV installations brings up the issue of whether non-governmental entities would be allowed to "eavesdrop".

Aside: For purposes of this post, by the term "eavesdrop" I mean listening to conversations which are thought to be private.

Just having a microphone with a camera does not mean that it will be possible to understand individual conversations or even that any collected speech can be listened to or understood by the CCTV operators. For instance, sound captured by microphones can be automatically analyzed (i.e. by a computer) to detect alerting sounds such as gunshots (which are uncommon in most of the UK due to gun control laws), screams or threatening tones of speech. A system could be designed such that the human operator has no access to the audio itself, but is instead only alerted by a text message or other indicator.

To summarize my points from the policy perspective, the act of installing a microphone does not necessarily imply that peoples' conversations are going to be bugged and there are some benefits to be had that do not involve any significant 'invasion of privacy' issues, assuming said system is designed properly. A final policy point is that having private companies collecting conversational speech or installing concealed CCTV in public places raises lots of privacy issues (obviously!). The former seems open to serious abuse unless it is heavily and closely regulated (like the police are). I'm withholding judgment on the latter - there just wasn't enough information in this article for me to form an opinion yet as I don't understand what problem is to be addressed and the perceived benefit.

Although I covered some technical issues while addressing policy, there are a few more points to make from that aspect also. The first is that simply mounting a single omni-directional microphone, no matter how sensitive, up on a pole with a CCTV camera is not going to necessarily pick up conversational speech, as any sound engineer can tell you. What is going to be recorded from way up there by an omni mic during busy times? A cocktail party-like blend of sounds, gunshots, sirens, shouts and other loud sounds is about all that will be heard on the microphone output in real urban scenarios. So, my question is what the microphones are supposed to be for? There are obvious benefits that could be derived, but not without cost and technical implications. It seems to me that there is some need here for some consultations with experts knowledgeable about audio, filtering, and the like before this goes too far.

Anyway, that is all for the moment. Please feel free to comment, but please understand that I wrote this very quickly so I was not able to address all the issues that are raised by the political debate. Nothing is implied by my leaving anything out.

Video Filtering: IBM Research on Automatic Face Masking and Privacy

IBM has been moving into the video surveillance services market as of late. Now comes word on some of its research into technologies that might support that. They are not the first to work on automatic face masking but it is the first time I've seen this privacy angle to it - automatically detect, track, and mask (i.e. blur) all faces on an incoming video stream and then selectively unmask as required to identify someone. I've worked with this regularly over the last ten years, so I know that it is a very challenging problem to do reliably, particularly with "real", not laboratory, video.

Beyond Nielsen ratings - now with audio

Imagine getting a free cell phone, but instead of being forced to listen to advertising in exchange for your free-ride, it listens to you instead. That's right. Every so often, it records the activity going on around you and then uses a computer algorithm to create a "signature" from those sounds passes that signature back to a processing center, where another algorithm compares that signature to its master database to recognize what media type and "program" you are listening to - be it a music CD, TV/radio program, Muzak, rock concert, or whatever. Your data is then compiled with many others' to create media ratings. By only passing a signature and not the raw audio itself, no actual conversations get eavesdropped on - which would be illegal in many places.

In this age of TiVO, iPod, and other time- and/or location-shifting devices to allow viewers and listeners to consume media when and where it is convenient, traditional ways of measuring and estimating ratings are not as effective - hence the market's experimentation with this type of technology.

If you are interested in reading more, one company that is selling such a service is IMMI (Integrated Media Measurement Inc). There is a link on their home page to a Wall Street Journal review of several companies doing similar things.

(Hat Tip: Bruce Schneier's CRYPTO-GRAM email newsletter, January 17, 2007 edition)