Skip to main content

Facial Recognition, CNNs and the Cost of Being Wrong

· 3 min read
Ross Bulat
Ross Bulat
Full Stack Engineer

Wall's article made me think about facial recognition less as a model-accuracy problem and more as a deployment problem: what happens when an organisation treats a CNN score as objective evidence? A convolutional neural network can return a probability or similarity score, but it does not produce truth. In a photo app, a wrong match is irritating; in policing, border control, or military use, it can expose someone to surveillance, exclusion, arrest, or harm (Wall, 2019).

Bias and False Positives​

The main ethical issue for me is the asymmetry of false positives. If a system wrongly identifies someone, that person carries the consequences, while the organisation using the system may frame it as a technical error. This is especially serious when the affected person cannot inspect the model, challenge the evidence, or understand why they were matched.

Wall (2019) also discusses performance differences across skin tone and gender. That made me more cautious about aggregate accuracy. A model can look strong overall while failing more often for specific groups. If training data over-represents white men, or reflects unequal policing, the CNN may reproduce social bias with a technical label.

Public facial scanning affects more than individual cases. It can make protest, association, and everyday movement feel monitored. It can also create a feedback loop: communities that are watched more closely produce more police data, which can then be used to justify more surveillance.

Legally, this raises questions about privacy, consent, proportionality, data retention, and appeal. A "human in the loop" is not enough if the reviewer is likely to defer to the system score. Real safeguards need trained reviewers, clear thresholds, audit records, and a challenge process that ordinary people can use.

Professional Accountability​

A key lesson is that deployment is part of engineering responsibility. Machine-learning teams should ask where the data came from, whose faces are under-represented, which groups have higher error rates, and what happens after a match. Limitations should be documented, systems monitored, and independent audits supported. Some use cases may need flat-out refusal to deploy, or at least a moratorium until the social and legal issues are addressed.

Reflection​

My view is that facial recognition may be defensible in narrow, controlled settings, but mass public surveillance is different. The question is not only can the CNN recognise a face? It is also should it be used here, who carries the risk when it fails, and who is accountable?

Reference​

Wall, M. (2019) 'Biased and wrong? Facial recognition tech in the dock', BBC News, 4 July. Available at: https://www.bbc.com/news/business-48842750 (Accessed: 20 June 2026).