Computerized Face-Recognition Technology Is Still Easily Foiled by Cosmetic Surgery

In the first test of face-recognition technology vs. cosmetic surgery, face recognition losesFor years, developers of face-recognition algorithms have been battling the effects of awkward poses, facial expressions, and disguises like hats, wigs, and fake moustaches. They’ve had some success, but they may be meeting their match in plastic surgery.

Systematic studies have tested face-recognition algorithms in a variety of challenging situations—bad lighting, for example—”but none of those conditions had nearly the effect of plastic surgery,” says Afzel Noore, a computer science and electrical engineering professor at West Virginia University, in Morgantown. In June, Noore reported the results of the first experimental study to quantify the effect of plastic surgery on face-recognition systems, at the IEEE Computer Society’s Computer Vision and Pattern Recognition conference, in Miami. His team of collaborators is based in West Virginia and at the Indraprastha Institute of Information Technology, Delhi, in India.

Using a database containing before-and-after images from 506 plastic surgery patients, Noore and his colleagues tested six of the most widely used face-recognition algorithms. Even in pictures where the subject was facing forward and the lighting was ideal, the best of the algorithms matched a person’s pre- and postsurgery images no more than about 40 percent of the time. The researchers found that for local alterations—say, a nose job, getting rid of a double chin, or removing the wrinkles around the eyes—today’s systems could make a match roughly one-third of the time. For more global changes like a face-lift, the results were dismal: a match rate of just 2 percent.

”We have to devise systems for security applications knowing that people will aim to circumvent them,” says Noore. In particular, researchers must examine a further complication of the plastic surgery problem—the compounding effects of a series of surgeries over time.

Meanwhile, Noore and his coauthors are testing a game-changing hypothesis: that even after plastic surgery, there are features beneath the skin but still observable that remain unchanged.