Skip to main content

Facial Recognition, CNNs and the Cost of Being Wrong

· 3 min read
Ross Bulat
Ross Bulat
Full Stack Engineer

Wall's article made me think about facial recognition less as a model-accuracy problem and more as a deployment problem: what happens when an organisation treats a CNN score as objective evidence? A convolutional neural network can return a probability or similarity score, but that score does not, by itself, establish a person’s identity. In a photo app, a wrong match can merely be irritating; but in policing, border control, or military use, it can expose someone to surveillance, exclusion, arrest, or harm (Wall, 2019).

Bias and False Positives​

The main ethical issue for me is the asymmetry of false positives. If a system wrongly identifies someone, that person carries the consequences, while the organisation using the system may frame it as a technical error. This is especially serious when the affected person cannot inspect the model, challenge the evidence, or understand why they were matched.

Wall (2019) also discusses performance differences across skin tone and gender. That made me more cautious about aggregate accuracy. A model can look strong overall while failing more often for specific groups. If training data over-represents white men, or reflects unequal policing, the CNN may reproduce social bias with a technical label.

Public facial-recognition systems can identify and record people as they travel, meet others or attend political events, often without their knowledge or consent. Knowing that this monitoring is possible may discourage people from participating in lawful protests or associating with particular groups. It can also reinforce existing policing patterns: communities under greater surveillance generate more recorded encounters, and those records may then be used to justify even more surveillance in the same communities.

Legally, this raises questions about privacy, consent, proportionality, data retention, and appeal. A "human in the loop" is not enough if the reviewer is likely to defer to the system score. Real safeguards need trained reviewers, clear thresholds, audit records, and a challenge process that ordinary people can use.

Professional Accountability​

For me, deployment is a part of engineering responsibility. Machine-learning teams should ask where the data came from, whose faces are under-represented, which groups have higher error rates, and what happens after a match. They should document limitations, monitor systems after release, and support independent audits. Some use cases may need flat-out refusal to deploy, or at least a moratorium until the social and legal issues are addressed.

Reflection​

My view is that facial recognition may be defensible in narrow, controlled settings, but mass public surveillance is different. The question is not only can the CNN recognise a face? It is also should it be used here, who carries the risk when it fails, and who is accountable?

Reference​

Wall, M. (2019) 'Biased and wrong? Facial recognition tech in the dock', BBC News, 4 July. Available at: https://www.bbc.com/news/business-48842750 (Accessed: 20 June 2026).