What facial recognition actually does, its real accuracy limits, other biometric signals, and the legal/ethical framework around using it.
Chapter 6 is about whether an image is authentic and where it came from. This chapter is about a different, higher-stakes question: who is the person in it? Facial recognition and other biometric techniques can answer that question fast, sometimes in seconds, but the technology's speed and confidence-sounding output make it uniquely easy to misuse. This chapter covers what the technique actually does, its real accuracy limits, and the ethical and legal boundaries around using it.
Reverse image search (Chapter 6) finds copies or near-copies of the exact same image. Facial recognition is a different technology: it converts a face into a numerical representation (an embedding) and searches for other faces whose embeddings are mathematically close, regardless of the surrounding image, lighting, angle, or context. That is why a facial search can return a completely different photo of the same person, taken years apart, on a different platform, in different clothing, while a reverse image search would not.
Public-facing facial search services generally work by having already scraped and indexed large numbers of photos (social media profiles, public web pages, sometimes leaked datasets), then matching a submitted photo against that index. This raises its own legality and ethics questions, covered in §15.5, separate from the ethics of any specific investigation that uses the tool.
Consumer and semi-professional facial search tools return a ranked list of candidate matches, sometimes with a confidence score attached. Treat both the ranking and the score as a lead, never as a verified identification. Real-world accuracy depends heavily on image quality (angle, lighting, resolution, occlusion), how well-represented the person is in the tool's underlying index, and the tool's own underlying model quality, which varies a great deal between services and changes over time as models are updated.
A pattern that recurs in public write-ups of facial-search mistakes: a partial or low-quality image (a covered mouth, an oblique angle, poor lighting) produces a "confident-looking" top match that turns out to be wrong. The confidence score attached to a match reflects how mathematically close two embeddings are, not how certain the identification is, and a poor input image can produce a high similarity score against the wrong person just as easily as against the right one. Independent technical reviewers have repeatedly pushed back on public identifications made from partial-face images, specifically because a similarity score cannot substitute for basic image-quality checks that a human reviewer would catch immediately.
Faces are the most common biometric OSINT deals with, but several others show up in real investigations, each with narrower and more specific use cases:
Misidentification is not a rare failure mode in this field, it is a recurring, documented one, and the consequences land on a real, uninvolved person. Public write-ups of biometric-OSINT mistakes share a common shape: an investigator gets a confident-looking match from a facial search tool, treats the match itself as sufficient evidence, and publishes or acts on an identification that turns out to be wrong. In more than one documented case, this has led to public harassment of an innocent person who simply happened to have a similar face to the actual subject.
The lesson is not that facial recognition is useless. It is that a facial-search hit is exactly one data point, with a real and non-trivial false-positive rate, and it must be corroborated the same way any other single-source OSINT finding is corroborated elsewhere in this manual (Chapter 13): independent details that only the correctly identified person would have, not just a visual resemblance.
Biometric data carries a heavier legal and ethical weight than most other OSINT source material. Under GDPR and similar frameworks, biometric data used to uniquely identify a person is a "special category" of personal data with stricter processing rules than ordinary personal information. Several jurisdictions separately restrict or ban commercial facial recognition use outright, or restrict it specifically for law enforcement and government use, and the legal landscape here changes faster than in most other areas this manual covers.
Beyond strict legality, weigh cultural and power-dynamic context: identifying a protester, an activist, or a vulnerable individual carries real risk of enabling harm, even where the underlying technique is technically legal. If your work could plausibly be used by a repressive actor to identify and target people exercising a lawful right (to protest, to organize, to speak), that consequence is part of the ethical calculus, not a separate question. If any part of your work might end up supporting a legal case, preserve the original images, your search methodology, and every intermediate result, since an algorithmic match on its own is unlikely to be admissible without that supporting chain of custody and, often, expert testimony.
Never conclude an identification from a facial-search score alone. Use it to generate a candidate, then independently corroborate: does the candidate's other public information (location, associates, timeline, distinguishing marks visible in both images) actually fit what you already know about the subject? Document your reasoning the same way you would any other finding (Chapter 17): what the tool returned, what independent evidence supports or contradicts it, and your resulting confidence level, graded honestly rather than rounded up because the tool's score looked convincing.
For exercise 1, good independent details include things like a distinctive scar or tattoo, a consistent handle or username tying both photos to the same account history, a location or timestamp that fits a known timeline, or a third party who appears in both and can be independently identified. For exercise 2, a strong answer distinguishes between the similarity score (a mathematical property of the comparison) and the actual probability of correct identification (which depends heavily on image quality and how many other people could plausibly produce a similarly high score). For exercise 3, don't be surprised if the answer is genuinely unclear or contested — this is a fast-moving area of law in most jurisdictions, and "the rules are unsettled" is itself a legitimate finding.