exiftool in CTF: extracting file metadata part 2

Depov

Moderator
Staff member
MODERATOR
ULTIMATE
SUPREME
PREMIUM
MEMBER
Joined
Feb 18, 2025
Messages
646
Reaction score
1,454
PDF Metadata Analysis: Hidden Objects and Streams

PDF is one of the most underrated formats in CTFS. Everyone drives EXIF in pictures, and PDF files in the meantime hide more: embedded files, JavaScript code, hidden objects and invisible text.

ExifTool on PDF. Launch exiftool document.pdf shows the basic metadata: Author, Creator, Producer, Create Date, Modify Date, Page Count. The CTF flag can be in any of these fields. The Producer field contains the name of the PDF generator program - a hint to the format or method of creating a file.

Deep analysis. ExifTool reads only top-level metadata. For the internal structure of PDF, Didier Stevens utilities are needed. The first - pdfid document.pdf – outputs statistics on PDF keywords: /JS (JavaScript), /OpenAction (auto start at opening), /EmbeddedFile (input files), /Encrypt (encryption). If the pdfid shows /EmbeddedFile 1 – a file is hidden inside the PDF. The second - pdf-parser document.pdf – disassembles PDF by object. Each object is numbered and can contain a data stream. Team pdf-parser --object 5 document.pdf will show the contents of object No. 5, and with the flag --filter decompresses streams (FlateDecode, ASCIIHexDecode). Trail of Bits CTF Field Guide explicitly recommends pdf-parser for PDF analysis in CTF.

Workflow for PDF-Task: launch pdfid challenge.pdf and look for non-zero values at /EmbeddedFile, /JS, /OpenAction. If /EmbeddedFile more than zero – find an object with an investment through pdf-parser --search /EmbeddedFile challenge.pdf, then pull it out by the team pdf-parser --object N --filter --dump embedded.bin challenge.pdf, where N is the number of the object found.

Invisible text. In PDF, the text can be written in white on a white background or placed outside the visible area of the page. ExifTool won't show that. Use pdftotext document.pdf - to extract all text content, including invisible. If pdftotext gives out more text than can be seen when you open, the flag is in the invisible part.
Metadata Office documents: .docx as a ZIP archive

Microsoft Office files of OOXML (2007+) are ZIP archives. .docx, .xlsx, .pptx files are unpacked with a standard command unzip. To analyze the metadata of Office documents, this is a key knowledge.

Step 1 – unpacking. Team unzip -d docx_contents report.docx creates a directory with the following structure:

docx_contents/
├── [Content_Types].xml
├── _rels/
├── docProps/
│ ├── core.xml (автор, дата, ревизия)
│ └── app.xml (приложение, время редактирования)
└── word/
├── document.xml (основной текст и Track Changes)
└── media/ (встроенные изображения)

Step 2 – core.xml. This file contains Dublin Core metadata: dc:creator (author), dc:title, dc:description, cp:lastModifiedBy (last editor) cp:revision (review number), dcterms:created and dcterms:modified. The flag can be in any of these fields. Often dc:creator contains a name that is the answer to the task, and dc:description stores base64-line.

Step 3 – app.xml. Here: Application (e.g., Microsoft Office Word), AppVersion, TotalTime (total editing time in minutes), Pages, Words, Characters. The TotalTime field with a suspicious value like 31337 minutes (leet speak) - greetings from the compilers.

Step 4 – Track Changes. If the author of the document included tracking changes, the deleted text is saved in document.xml within the tags <w:del>. This text is not visible at the usual opening, but is easily pulled out of XML. Looking for <w:delText> through grep -oP '<w:delText>[^<]+' word/document.xml – remote fragments may contain a flag or part of the flag.

Step 5 – built-in media files. Catalogue word/media/ contains all images from the document. Each can be driven through the exiftool - the CTF compilers sometimes hide the second flag in the metadata of the built-in image. Do not forget about this level of nesting.

ExifTool also reads Office document metadata without unpacking: exiftool report.docx will show Author, Last Modified By, Create Date. But to access Track Changes and media files, unpacking through unzip is necessary.
ExifTool alternatives for metadata CTF

ExifTool is the main tool for extracting metadata, but not the only one. Depending on the task, other utilities are doing better.

mat2 (Metadata Anonymization Toolkit) – a metadata removal utility that shows at the same time that there is in the file. Launch mat2 -s challenge.jpg (flag -s – show metadata) displays a list of fields. Useful as a cross-check: If exiftool missed something, mat2 can detect metadata in exotic formats.

Python + Pillow for software EXIF-retrieval. When a task requires processing a pack of files or extracting a specific field over a script, Python with a Pillow library solves the problem. Trail of Bits CTF Field Guide calls Python the main scripting language for forensics. According to the EN-source (Medium, foysol60s), “Both ExifTool (Perl) and Pillow (Python leverage) to access and parse the EXIF data” – approaches are complementary.

from PIL import Image
from PIL.ExifTags import TAGS

img = Image.open("challenge.jpg")
exif_data = img._getexif()
if exif_data:
for tag_id, value in exif_data.items():
tag_name = TAGS.get(tag_id, tag_id)
print(f"{tag_name}: {value}")

The script displays all EXIF tags with human-readable names. For CTF automation, wrap it in a loop by file from the directory and add keyword filtering.

Metagoofil for OSINT intelligence. Metagoofil downloads public documents (PDF, DOCX, XLSX) from the target domain through Google and automatically pulls out metadata from them: usernames, email addresses, paths to files. In pure CTF forensics, it is rarely needed – files are already given. But in tasks with the OSINT component, Metagoofil is indispensable for osint file analysis.

strings + grep as the last frontier. When exiftool shows nothing interesting (metadata is cleared or damaged), go back to the basic: strings -n 8 challenge.file | grep -iE "flag|ctf|key|pass". Rough method, but on CTF it works regularly. Trail of Bits confirms: strings - "a first-pass time that listens that plaintext flags or encoded data."
Task The instrument Why exactly him
First Pass by Any File exiftool Maximum coverage of formats
Mass File Processing Python + Pillow Scriptability
Deep PDF analysis pdf-parser, pdfid Access to objects and flows
Office history revision unzip + grep by XML Direct access to Track Changes
Cross Check mat2 Alternative Parser
OSINT by domain Metagoofil Automatic collection from the web
Purified Metadata strings + grep Works when everything else didn't work
Steganography in metadata and anti-forensic

CTF drafters are not limited to the standard Comment and Artist fields. The data is hidden in non-standard EXIF tags, and knowledge of these techniques distinguishes an experienced participant from a beginner.

MakerNote – metadata unit specific to the camera manufacturer. ExifTool parsits MakerNote most cameras (Canon, Nikon, Sony and dozens of others). The custom data inside can look like garbage — but if the MakerNote contains a line similar to base64 (ends with = or ==), decode: echo "строка" | base64 -d.

XMP arbitrary fields. The XMP standard allows you to create custom namespaces. Tag like <ctf:secret> will not show any viewer, but exiftool with flag -G he'll get him out. Look for atypical groups of tags in the output – anything that does not apply to standard EXIF, IPTC, or XMP groups.

ICC Profile – color profile of the image. Rarely, but data can be implemented in the ICC profile. ExifTool shows ICC tags in the group ICC_Profile.

Base64, hex and other encodings. Frequent reception of steganography in metadata: the flag is encoded in base64 and placed in a text field. The line ZmxhZ3t0ZXN0fQ== in the Comment field, this flag{test}. There are also hex-lines: 666c61677b... – ASCII symbol codes flag{...}. A less typical but possible option is ROT13. If the field contains unreadable but printed text, try all the standard decoding. Didn't guess the first time? Try CyberChef – drop the line and click “Magic”, it will pick up the encoding itself.
Timestomp: change of time tags (T1070.006)

In real-world investigations, attackers use the Timestomp (T1070.006, MITRE ATT&CK) technique to replace timestamp files. On the CTF, this technique works in both directions: the drafters can intentionally put suspicious dates (the file “created” in 1970 or 2099) as a hint. If the exiftool shows Create Date, sharply different from Modify Date or File Modification Date, it is a signal to dig deeper.

ExifTool can not only read, but also write timestamps: exiftool -AllDates="2023:01:01 00:00:00" file.jpg Re-records all dates. In the CTF context, this means that the date in EXIF is an unreliable indicator, it is easy to replace. In Atomic Red Team there are three tests for T107.006: change access timestamp, modification timestamp and development timestamp through commands on Linux and macOS.

To detect Timestomp in real investigations, there are Sigma rules from the SigmaHQ repository: proc_creation_lnx_touch_susp.yml monitors suspicious calls touch in Linux, posh_ps_timestomp.yml detects manipulation of file attributes through PowerShell, win_security_susp_time_modification.yml record suspicious time changes in Windows Security Log. In total, 6 Sigma rules for three platforms are available by T1070.006: Linux, Windows, macOS.

Metadata and transmission platforms. The study, "Forensic Value of Exif Data" (SciEPublish, 2025, 15 citations), found that different platforms handle metadata differently when transferring files. Messengers like WhatsApp and Signal squeeze pictures and remove a significant part of EXIF. Transmission via email attachments or USB usually saves metadata in its entirety. For CTF, this is context: if in the assignment legend the file “was downloaded from the server” or “transmitted by email” – metadata on the spot. If “obtained through the messenger” – some of the data could be deleted, and this is part of the puzzle.
Practical checklist for CTF forensics

This algorithm works for any metadata extraction task:

file challenge.ext – define the real format by magic bytes
exiftool challenge.ext – read all metadata, look for Comment, Author, GPS, non-standard tags
exiftool -b -ThumbnailImage challenge.ext > thumb.jpg – pull out the thumbnail (for images)
For PDF: pdfid challenge.pdf – looking for /EmbeddedFile, /JS — then pdf-parser for extraction
For DOCX: unzip -d out challenge.docx – read docProps/core.xml, looking for <w:delText> in document.xml
strings -n 8 challenge.ext | grep -iE "flag|ctf|base64" – if the metadata is empty
Found base64 — echo "строка" | base64 -d
Retrieved the nested file - back to step 1
 
Top Bottom