Biometrics Under Attack

Overview

Face biometric systems are increasingly exposed to manipulated media. This tutorial looks at two sides of that problem along the full recognition pipeline — capture, processing, enrollment and decision: attacks, represented by face morphing, and defenses, represented by invisible watermarking for provenance (SSLWatermarking, RoSteALS). The central question is how to evaluate both jointly: how much a watermark costs in perceptual quality and biometric similarity, how robust it is to benign transformations, and how vulnerable a system is to morphs at its real operating point.

Materials

Hands-on activity

How does increasing watermark strength (α) change, at the same time, visual quality, watermark reliability and biometric similarity?

1. Setup — fetch facetrust.zip, data.zip and morphs.zip, copy the morphs into data/morph/, install the package with the ArcFace extra. Run this before the session.
2. Embeddings — ArcFace via insightface, with a fallback extractor so the notebook never blocks.
3. Dataset — load clean, watermarked and morphed samples from data/.
4. α sweep — PSNR/SSIM and embedding drift as a function of watermark strength.
5. Robustness — JPEG, resize, blur, crop and rotation.
6. Four scenarios — Clean↔Clean, Clean↔WM, WM↔Clean, WM↔WM; EER and TAR@FAR.
7. Morphing attack potential — MMPMR, FMMPMR, ProdAvg-MMPMR and RMMR at calibrated FARs, per morphing algorithm.
8. Discussion — is there a single optimal α, or a trade-off surface?

Watermarked images were generated offline with SSLWatermarking (iterative optimisation per image), so the live session only evaluates them. A GPU runtime speeds up ArcFace; CPU works for the provided data.

Running locally

1. Download facetrust.zip, data.zip and morphs.zip.
2. Extract the code and the data, and install the package:

   unzip facetrust.zip -d src && unzip data.zip
                   pip install -e "src/facetrust[arcface]"     # add ,lpips for LPIPS

3. Extract the morphs and point copy_frll_data.py at the folder that holds the morph_* subfolders. It copies them to data/morph/<algorithm>/<a>+<b>.png:

   unzip morphs.zip -d morphs_src
                   python src/facetrust/scripts/copy_frll_data.py --skip-clean \
                   --morph-src morphs_src/<folder with morph_*> --datadir data \
                   -t morph_stylegan2 morph_stylegan3 morph_diffae   # the subfolders you have

   Add -n for a dry run.
4. Run the full pipeline:

   python src/facetrust/examples/demo_full_pipeline.py

Expected data layout:

data/clean/<subject_id>/<sample_id>.png
            data/watermarked/<subject_id>/<sample_id>__alpha<alpha>.png
            data/morph/<algorithm>/<contributor_a>+<contributor_b>.png

What does the code measure?

Data and third-party code

• Clean images: Face Research Lab London Set (CC BY 4.0).
• Morphs: FRLL-Morphs (AMSL, FaceMorpher, OpenCV, StyleGAN2).
• Watermarking: SSLWatermarking (CC BY-NC) and RoSteALS.
• MMPMR, ProdAvg and RMMR checked against the reference implementation by Scherhag et al.

Citation

@inproceedings{schardong2026biometrics,
  title     = {Biometrics under attack: On provenance, trust and threat robustness},
  author    = {Schardong, Guilherme and Novello, Tiago and Schirmer, Luiz and Gonçalves, Nuno},
  booktitle = {SIBGRAPI 2026 Tutorials},
  year      = {2026}
}