Image & Media Forensics

Reverse image search, EXIF/IPTC/XMP metadata, manipulation detection, and video frame extraction.

An image or video is never "just a picture" to an OSINT investigator — it's simultaneously four things: a set of pixels you can search for online, a file carrying hidden metadata, a technical object that might have been altered, and — if it's video — a time-based sequence hiding details visible for only a fraction of a second. This chapter works through all four.

6.1 Reverse Image Search

Reverse image search queries the internet using an image instead of text. In OSINT it's used to: check whether an image has appeared online before, find where it was published, recover older or higher-resolution versions, check whether it's been reused in a misleading context, and support geolocation by recognizing landmarks or objects.

Critical limitation: it is not an automatic authenticity test. A match gives you an investigative lead — the matching image, the page, the publication history, and the surrounding context still need examining. A failed search doesn't prove originality either — the matching copy might be private, deleted, recently uploaded, heavily altered, or simply outside that particular engine's index.

How it actually works: four layers of technology

Modern reverse-image engines (Google Lens, TinEye, Yandex) work like a funnel, from basic file recognition to deep semantic understanding:

1. Exact / near-exact matching: recognizes the same underlying file even if it's been resized, compressed, or slightly color-adjusted. Fails on major crops, added text, mirroring, or heavy editing.
2. Perceptual hashing: unlike a cryptographic hash (SHA-256, which changes completely with a one-byte difference), a perceptual hash represents visual appearance — similar-looking images produce similar hashes even if size/compression/format differ. It can occasionally produce false matches, or miss heavily modified versions.
3. Feature matching / visual embeddings: recognizes local details (corners, shapes, textures) that survive moderate resizing, and higher-level embeddings that return different photos of the same landmark or object — useful for identification, not proof that the same file was reused.
4. Object and text recognition: identifies objects and extracts visible text (a logo leads to a company, a sign reveals a location) — always verify auto-extracted text manually.

Exact copy, derived copy, similar image

Three categories that get confused constantly. An exact copy has the same visual content (possibly the same file). A derived copy comes from the same image but has been resized, compressed, cropped, mirrored, or watermarked. A visually similar image shows the same subject or location but comes from a different photograph entirely (two photos of the same monument, taken at different times — these are not the same file and shouldn't be treated as reused copies). To confirm two results derive from the same image, compare small, distinctive details: the exact position of people, vehicles, clouds, shadows, reflections, surface damage.

Worked example: reverse image search, Pisa

Imagine an image showing a bridge, a clock tower, and a cathedral, traced as follows. Step 1 — a full-image reverse search finds similar photos in the Pisa area. Step 2 — searching just the cropped background (bridge, clock, cathedral) via Bing Visual Search confirms the right clock tower, but on the opposite side of the bridge from expected. Step 3 — Google Maps/Street View turns up two candidate bridges. Step 4 — Street View on the correct bridge reveals the original photo had been mirrored — which would explain why background graffiti had looked "backwards." Lesson: always check for mirroring when landmarks look right but sit on the wrong side.

A cautionary pattern: visually similar, geographically wrong

This exact pitfall is worth walking through with a hypothetical. Imagine a reverse-image search for a set of brightly painted stairs returning a strong, visually convincing match to a well-known rainbow staircase in a different country entirely — bright rainbow paint, a similar stair angle and framing. The match feels confident. It would still be wrong if the original photo was actually taken somewhere else, at an entirely different structure that happens to share the same distinctive visual signature. This is a realistic failure mode, not a rare edge case: brightly painted or visually distinctive public installations (rainbow staircases, colorful murals, striking public art) tend to recur in similar forms across many cities, which is exactly the kind of visual signature that fools a confident-looking match. The fix is the same discipline described above: don't stop at the first visually-similar result just because it looks confident — compare specific distinguishing details (railing style, surrounding architecture, exact color transitions) before treating a strong-looking engine match as an identification rather than a lead still waiting to be verified.

A practical workflow

Preserve the original image and record filename, source, and claimed context. Search the whole image first, in more than one engine. If results are weak, create working crops around distinctive features on a copy — never the original — and try different sizes, rotation, and a mirrored version. Open promising results and compare small details, never just a thumbnail. Examine the source page for captions, dates, credits. Use "earliest confirmed appearance" rather than "the original source" unless you can actually demonstrate that.

Common mistakes

• Treating a photo of the same subject as "the same image" without comparing specific details
• Trusting a thumbnail without opening the actual result page
• Assuming the first result is the original source, or that a failed search proves originality

6.2 Metadata: EXIF, IPTC, XMP, and Beyond

Metadata is data about data. One key source of confusion: "metadata" and "EXIF" are not synonyms — EXIF is only one category.

Multiple date fields — they don't all describe the same event

Example: DateTimeOriginal: 2025:08:02 18:43:15, XMP ModifyDate: 2025:08:05 11:20:04, FileModifyDate: 2026:07:10 15:32:41. This can mean: captured Aug 2, 2025; processed Aug 5, 2025; downloaded or touched on the current machine Jul 10, 2026. Differences don't automatically mean deceptive manipulation — the file may simply have been copied, exported, or legitimately edited. Also watch for timezone fields (OffsetTime, OffsetTimeOriginal, GPSDateStamp) — a timestamp with no offset might reflect a camera clock that was simply set wrong.

GPS and reverse geocoding

Coordinates show up as decimal (49.6116, 6.1319) or degrees/minutes/seconds. They must be checked in a mapping tool and compared against the visual content — a coordinate in metadata is a lead, not final proof (it can be inaccurate, copied from another file, manually added, or associated with a spot near the camera rather than the exact subject). Reverse geocoding converts coordinates into a readable place name — but the nearest label isn't always the right answer; examine the exact point and administrative boundaries before deciding which name is relevant.

Detecting possible modification through metadata

Clues: a software field (Software/CreatorTool) appearing only in the "after" version; an editing timestamp later than the original capture; metadata missing from one of two compared versions (possibly stripped during editing, compression, or social-media processing); conflicting fields (EXIF date vs. XMP date, GPS that doesn't match the visible environment). Every one of these is a lead, not proof — the right conclusion is "the metadata indicates the file was processed," not "the metadata proves the image is fake."

ExifTool: practical commands

Workflow: preserve the highest-quality image (no screenshots, no re-saves) → compute a cryptographic hash before analysis (sha256sum image.jpg) to prove the file didn't change during your investigation → extract full grouped metadata → compare it against the claim attached to the image → cross-check against reverse image search, maps, landmarks, shadows, weather.

Two practical points worth knowing before concluding metadata is simply absent. First, how a file was transferred matters: a plain email attachment or certain cloud-upload paths commonly strip EXIF data in transit, even when nothing about the process looks unusual — if you need to preserve metadata for a handoff, compress the file into a zip archive or base64-encode it first rather than sending the raw image directly, and confirm the metadata survived on the receiving end. Second, don't stop at the date/GPS fields: comment, description, and caption fields can carry arbitrary text that a device or editing tool never generates on its own — someone can plant a false caption, a fabricated location name, or a misleading note directly into these fields, so check them specifically rather than assuming all textual metadata is machine-generated and therefore automatically trustworthy.

6.3 Cryptographic vs. Perceptual Hashing

A point that genuinely distinguishes technical understanding: cryptographic hashes (SHA-256) change completely with a single-bit difference — good for proving byte-for-byte identity, useless if the image was resized. Perceptual hashes aim at the opposite: similar values for visually similar images. That distinction explains why reverse image search finds "variants" of an image while a SHA-256 hash wouldn't recognize them as related at all.

6.4 Image Manipulation Detection

Processing (technical operations: resizing, compression), editing (a broader term), and manipulation (changes that alter the visible content or meaning) aren't synonyms. The right question isn't "is this fake?" but "what actually changed, and does that change affect the meaning or evidential value of the image?"

Types of manipulation

• Object removal: erasing a person/object/inscription, replaced with cloned pixels or AI-generated fill. Clues: repeated patterns, interrupted lines, unusual smoothing, missing shadows.
• Object addition: inserting content from another image — may differ in perspective, scale, sharpness, lighting, noise, shadows.
• Copy-move: copying part of the same image elsewhere in it — compatible color/noise because it's from the same file; look for identical textures, repeated defects.
• Splicing: combining content from two or more source images — inconsistent edges, sharpness, lighting, scale.
• Text/inscription modification: check font shape, spacing, alignment, perspective, surface texture, shadows.
• Cropping/context manipulation: the remaining pixels can be genuine while the removed area held essential context (a person, a sign, a source credit).
• AI-generated/AI-assisted editing: inconsistent geometry, repeated objects, malformed text, implausible reflections — but high-quality AI content can have almost no visible mistakes. AI-detection services produce false positives and negatives — treat their output as one indicator, never final proof.

Visual inspection techniques

Edges/boundaries: inserted or removed content can create edges that are too sharp, too soft, or haloed — but depth of field, focus, and compression create natural differences too. Repeated patterns: cloning tools duplicate groups of bricks, leaves, clouds — natural scenes have repetition, but identical arrangements with matching flaws are more suspicious. Texture inconsistencies: an edited area can look unusually smooth or noisy compared to its surroundings. Broken lines/structures: architectural edges, text baselines, and horizons should continue logically. Color/tonal differences: check whether they're physically plausible, not just present.

Perspective/geometry/scale: objects in the same scene should follow compatible perspective — size, angle, relationship to the horizon. An inserted object might look wrong-sized or angled relative to the surface beneath it (remember: a wide-angle lens distorts objects near the frame edge, and a telephoto lens compresses distance — don't mistake these for manipulation).

Lighting/shadows/reflections: objects lit by the same main source should show compatible highlights and shadows. A missing, mispositioned, or wrongly-soft shadow is a real clue. Reflections in mirrors, glass, or water must follow the geometry of the reflective surface — a missing or inconsistent reflection is strong evidence.

JPEG artifacts, Error-Level Analysis, and noise

Repeated JPEG saving creates block patterns and "ringing" around sharp edges. Error-Level Analysis (ELA) visualizes differences in recompression behavior — bright areas in an ELA result can come from text, sharp edges, fine texture, noise, or just normal JPEG behavior, not only from manipulation. It's less useful on heavily compressed or repeatedly-saved images. Free tools: Forensically and FotoForensics. Camera sensor noise patterns vary by device/ISO/lighting — an inserted object might carry a different noise signature than its surroundings, but only compare regions with similar lighting and focus.

When pixels alone can't settle it: research as forensics

Visual inspection sometimes can't resolve a case on its own — a professionally removed or altered inscription can look completely clean at the pixel level, with no detectable cloning artifact, unusual edge, or texture mismatch at all. When that happens, treat the question as a research problem rather than only a pixel-analysis one: search historical photographs, archived records, or secondary written descriptions of the same object or location from before the image in question was taken. Independently confirming what an inscription used to say, through a source entirely outside the image itself, can settle a case that pixel-level forensics alone cannot — because in a genuinely clean edit, the evidence of manipulation lives outside the file, not inside it.

Reporting manipulation findings

Separate observation ("a repeated stone pattern appears on both sides of the suspected area"), interpretation ("consistent with cloning or content-aware fill"), and conclusion ("the image shows several indicators consistent with content removal, supported by comparison with an earlier complete version") — with an explicit confidence level (low/moderate/high). Record the filename and hash, source, tools/commands, screenshots, and alternative explanations.

Worked example: geolocating from a sign

Imagine a photo of a road intersection: text on a utility-pole sign looks Dutch, the environment is desert-like, palm trees are visible. A "South Africa" hypothesis (Afrikaans, derived from Dutch) gets confirmed by searching the exact sign text, which returns an Afrikaans-language Facebook page. A second sign is searched together with "South Africa" and returns a specific city directly. Verification on Street View reveals that the second street name is actually shared by two different streets in different areas of that city — only one of the two candidate intersections matches every other clue (the terrain, the first sign, the vegetation). Lesson: one common place name is never enough on its own — you need a second, independent point to rule out alternatives.

6.5 Environmental Clues: Language as the Strongest Category

Among all the categories of environmental clue (language, landmarks, architecture, roads, terrain, vegetation, weather, sun/shadow — fully developed in Chapter 7), written information is usually the strongest: even if you can't read the full text, the writing system alone (Latin, Greek, Cyrillic, Arabic, Hebrew) narrows things considerably — though a writing system is not a language (Cyrillic is used by several countries), so you'll need additional evidence (spelling patterns, diacritics, flags, currency) to narrow further.

A clue's strength depends on how distinctive it is, how many places could have something similar, whether it's independently verifiable, and whether it agrees with the rest of your evidence. A landmark also needs to be checked against its surroundings — many cities have similar towers or churches; only the combination of landmark + surrounding buildings + terrain + camera angle is convincing. Identifying a landmark doesn't automatically reveal a building's function either (a historic complex might now operate as a museum or hotel) — that needs a separate check against an official source.

Sun and shadows: the sun's position is described by solar azimuth (compass direction) and solar altitude (height above the horizon). To translate a shadow's direction inside a photo into a real-world compass direction, you first need to align the scene with a map (roads, building facades). A sun-position tool (SunCalc) gives an approximate hour or range, not an exact minute — uncertain camera orientation, uneven ground, and lens distortion all reduce precision.

6.6 Video Analysis and Frame Extraction

A video isn't a single image — it's a sequence of frames, and a detail (a sign, a face, a reflection) might be visible for only a fraction of a second. Frame extraction lets you pause and examine specific moments in detail — but it has to be done properly to be reproducible.

The technical basics

A video container (MP4, MOV, MKV, WebM) is not the same as its codec (H.264, H.265, VP9, AV1) — an MP4 file might hold H.264 video plus AAC audio. Frame rate can be constant (CFR) or variable (VFR) — common in smartphone footage, screen recordings, and social media exports — where frame 300 doesn't necessarily correspond to exactly 10 seconds. For reliable timing, use the stream's actual presentation timestamps (PTS), not a simple frame-number/fps calculation.

Many codecs don't store every frame as a full image: I-frames (keyframes) are complete, independently decodable images; P-frames store changes relative to an earlier frame; B-frames depend on both earlier and later frames. When a player jumps to a timestamp, it often seeks to the nearest earlier keyframe first — which is why an accurate seek (placed after the input file in an FFmpeg command) is slower but more reliable than a fast one.

FFmpeg extraction — practical commands

Use PNG, not JPEG, to avoid extra lossy compression when analyzing. Watch for rotation metadata — smartphone video often reports landscape dimensions while displaying vertically via a rotation flag; make sure your extraction tool applies it correctly.

Choosing useful frames and verifying them

Not every frame has equal value — a useful one has a readable sign, a clear landmark, a reflection, a stable and sharp view. Check neighboring frames, since motion blur and obstruction vary frame to frame; the clearest frame might be just before or after the "obvious" moment. Extracted frames can feed reverse image search, OCR (always verify auto-OCR output manually on stylized/angled/blurry text), duplicate/cut detection (a frozen frame isn't automatically suspicious — it might be a static camera or a frame-rate conversion), and audio-video sync checks (they may not perfectly align due to recording distance or editing).

Screenshot vs. proper extraction: a player screenshot can include UI controls, subtitles, and scaling artifacts — a direct FFmpeg extraction is more reproducible and preserves more of the original visual information.

6.7 Extracting Text from Images (OCR)

A growing share of relevant content online never exists as searchable text at all — it's shared as an image. A screenshot of a chat conversation, a meme, an infographic, a photographed document, a slide from a presentation: all of this carries real information, but a normal keyword search can't find any of it, because there's no indexable text for a search engine to match against. If the claim, the conversation, or the figure you care about only exists as pixels, you need a different tool: OCR (Optical Character Recognition), software that reads the visible text in an image and converts it into actual text you can search, copy, and quote.

Accessible starting points: Google Lens has a built-in text-extraction feature (select text directly from a photo and copy it out), and many phone camera apps now include on-device text recognition as a standard feature — you don't need specialized software to get started.

OCR accuracy breaks down in predictable ways: low resolution, stylized or handwritten fonts, an angled or rotated capture, low contrast between text and background, and text overlaid on a busy or textured background all reduce accuracy — and memes in particular are often deliberately stylized in exactly these ways, since a striking font or a busy background image is part of the format.

The critical discipline point: OCR output must always be manually checked against the original image before you rely on it or quote it in a report. A misread character or word can silently produce a wrong quote — the same caution already given for auto-extracted text in Chapter 11 §11.4, and for auto-OCR on extracted video frames earlier in this chapter (§6.6). Never paste OCR output directly into a report without opening the source image next to it and reading the actual text yourself.

Once you've extracted the text from a meme or infographic, the rest of this chapter still applies: the claim it makes is subject to the same manipulation-detection thinking as any other image (§6.4), and the image itself can be reverse-searched (§6.1) to find where it first appeared and whether it's been altered or recontextualized since.