Face recognition by machines: Is it an effective surveillance tactic?

How to cite: M. Jagadesh Kumar, “Face recognition by machines:  Is it an effective surveillance tactic?” IETE Technical Review, Vol.30 (2), pp.93-94, March-April 2013.

Can you recall the last time when you tried to recognize a person’s face but failed? Sometimes we recognize the faces quickly and at other times we fail miserably. Among the several visual tasks that humans perform, face recognition is a very complex process since facial features of humans are not very distinct from each other.  How do humans recognize faces?  The right middle fusiform face area (FFA) of the brain is activated when we try to recognize faces [1]. This is an area of the brain located in the temporal lobe. In social interactions, face recognition is essential for us to know the identity, mood, sex and age of the person. It is now well recognized that humans possess highly evolved cognitive and neural mechanisms for face recognition. However, face recognition by humans based on recollection is not always correct.

A question that naturally comes to our mind is can machines be trained to recognize human faces? Human face recognition by machines has several commercial and law enforcement applications [2,3] and has been known since early 1960s. Advances in computer vision and improved sensor techniques have now led to a renewed interest in developing face recognition systems. Face recognition is much more secure since we cannot change our faces unlike a password or a magnetic swipe card which can be misused or passed on to others.

There are different methods of recognizing faces. Face recognition by machines is primarily an image analysis problem and is done either by verification or identification. In verification, you compare a face against a set of faces. In identification, a face is compared against each face in a data base. The reliability of a face recognition system depends on two critical requirements: (i) a large database of facial images and (ii) a testing procedure to evaluate the face recognition systems [4].

Three important criteria decide the effectiveness of face recognition process [5]: (1) The method used to represent the facial image to extract data, (2) Issues related to pose or facial orientation differences and (3) Whether the extracted data is embedded into a statistical shape analysis algorithm.

Faces are continuous three dimensional surfaces and we need to represent them using some effective means and convert into data. The easiest way to represent a face is to define land marks on the face such as eyes, nose and mouth and the geometrical distances and angles between them. The information obtained from land-mark representation is then transferred to a data base as a set of numbers. This method is not very effective if there are pose or illumination variations. In such a case one could use curve-based face representation where after locating a small landmark on the face, facial curves are extracted. This approach is useful in representing even difficult parts of the face such as forehead. However, most face recognition systems today use a complete surface based face representation.  One can use either an image or a mesh for surface representation of a face.

Before we quantify the facial shape using the numerical data extracted by any of the three approaches discussed above, we need to tackle the problem of facial orientation or pose. In one commonly used method, two face orientations are aligned and compared. Using a minimum of three corresponding point locations on both faces, differences in alignment are removed.  The other technique is to use an iterative method to minimize the pose differences.  One can even use data such as distances, angles and areas that are independent of facial orientations [5].

The extracted numerical data is now arranged into a series of numbers called vector descriptions and are embedded into a statistical facial space shape. Statistical shape analysis is required to estimate variability over similarly shaped faces. By calculating the distance and angle between two vector representations, we can then give a score for the similarity between two faces.

Face recognition by machines is no longer a scientific fiction. Face recognition systems have rapidly evolved in the last decade with recognition rates greater than 90 %. With the advent of 3D scanners, face recognition research has now shifted from 2D to 3D shape representation [6,7]. However, many challenges still remain to be tackled to make it robust to occlusions and multiple contexts. For example, expression, illumination and uncontrolled pose changes can result in a significant performance drop of the face recognition systems. As we age, our face changes in a non-linear way. We do not yet know how to tackle these problems effectively. Face recognition is still an evolving and open research area [8-10].

In India, research in the area of face recognition is primarily confined to few Indian Institutes of Technology and the Indian Institute of Science, Bangalore. While China and USA lead the face recognition research efforts, India does not find place in the top 10 countries contributing to this area. Face recognition systems are increasingly becoming important in India in view of the terrorist attacks and the rise in crime in cities particularly against women and children. We need to deploy face recognition systems at all sensitive and crowded places for conducting automated surveillance. This will help us in locating, tracking and profiling fugitives or persons who indulge in vandalism and riots. Use of face recognition systems in tandem with the data collected by the government agencies should enable us to identify any citizen of India in real time. This will not only deter those indulging in criminal activities but will also help the law enforcing authorities in quick dissemination of justice when crimes are committed.

There are of course concerns about privacy issues, such as including innocent citizens in the data bases and accumulating erroneous information about individuals [11]. Due to false positives, harmless people may be harassed if their face resembles that of suspects.  Face recognition can also be misused by commercial entities, for example, to decide which advertisement on the billboard will be of interest to you. Ethical questions too arise. Are we right in capturing facial images from public places such as markets, airports or railway stations and add to the data base without informing the individuals that their facial images are being captured? There is no clear answer. These are issues that one cannot overlook. Appropriate policies and legal provisions should be framed to prevent any private or government agencies from misusing such information to hound innocuous peace loving citizens.

When personal and public security becomes a national concern, we cannot sit back ignoring the threats. Deploying appropriate surveillance technology becomes inevitable. Without further delay, we need to significantly fund research activities in this area for building cost effective and efficient face recognition systems in India.


[1] N. Kanwisher and G. Yovel, “The fusiform face area: a cortical region specialized for the perception of faces”, Philosophical Transactions of the Royal Society B-Biological Sciences, Vol.361, No.1476, pp.2109-2128, Dec 2006.

[2] R. Caldara, X. Y. Zhou and S. Miellet, “Putting Culture Under the ‘Spotlight’ Reveals Universal Information Use for Face Recognition”, PLOS ONE, Vol.5,  Article Number: e9708, March 2010.

[3] W. Zhao, R. Chellappa, P. J. Phillips and A. Rosenfeld, “Face recognition: A literature survey”, ACM Computing Surveys,  Vol.35, pp.399-459, December 2003.

[4] P. J. Phillips, H. Moon, S. A. Rizvi, P. J. Rauss, “The FERET evaluation methodology for face-recognition algorithms”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.22, pp.1090-1104, October 2000.

[5] D. Smeets, P.Claes, D. Vandermeulen and J.G. Clement, “Objective 3D face recognition: Evolution, approaches and challenges,” Forensic Science International, Vol.201, pp.125–132, 2010.

[6] M. De Marsico, M. Nappi, D. Riccio, and H. Wechsler, “Robust Face Recognition for Uncontrolled Pose and Illumination Changes,” IEEE Transactions on Systems, Man and Cybernetics: Systems, Vol.43, pp.149-163, January 2013.

[7] G. Sandbach, S. Zafeiriou, M. Pantic and L. Yin, “Static and dynamic 3D facial expression recognition: A comprehensive survey,” Image and Vision Computing, Vol.30, pp.683–697, 2012.

[8] K. Ricanek, and C. B. Boehnen, “Facial Analytics: From Big Data to Law Enforcement,” Computer, Vol.45, pp.95-97, 2012.

[9] X. Zhang and Y. Gao, “Face recognition across pose: A review,” Pattern Recognition, Vol.42, pp.2876-2896, 2009.

[10] S. Mitra and T. Acharya, “Gesture recognition: A survey,” IEEE Transactions on Systems, Man and Cybernetics, Part C – Applications and Reviews,  Vol.37, pp.311-324, 2007.

[11] A. Nodari, M. Vanetti and I. Gallo, “Digital privacy: Replacing pedestrians from Google Street View images,” 21st International Conference on Pattern Recognition (ICPR), 2012, pp.2889-2893.

This entry was posted in Education and Research and tagged , , , , , , , , , , , , , , . Bookmark the permalink.

2 Responses to Face recognition by machines: Is it an effective surveillance tactic?

  1. Deepak joshi says:

    Very nicely written sir and the areas where it can be exploited for the society are many; it is indeed high time that we realize the potential this technology has, to ease out several aspects of our living. Also it is a boon to the law enforcement agencies if used suitably.


  2. Sarthak kalani says:

    Sometimes it appears to me, are there not many other problems besides such things which ought to be capturing our attention. It seems really bewildering that the number of problems are in themselves numerous, and to add to the fury, the solutions themselves invoke new kinds of problems (like that of ethics and misuse of information as pointed out in this article). In such a case, sometimes I feel: Is it really worth: Finding problems, solving them and once again facing new problems out of the solutions!! What is the real aim where we wish to reach? Do we even have that aim in mind? Is it really clear (surely not to me, but to those much more senior to me and experienced ones)? Or is it a race, in which if America and China are at top, then why not India?

    Surely enough, till the answers are clarified, the best way out is to enjoy the real joys of exploring the top quality technology and invest ourselves in finding the solutions to problems, atleast at technological end!!

I value your feedback. Please feel free to comment.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s