Skip to content
aidataset.broker home

Checklist / buyer side

What to require from de-identified enterprise data

Before accepting de-identified enterprise data, a lab should require a written policy, a description of how identifiers were found, evidence that it worked (leak scans, a reviewed sample, measured recall), its own sample review, and written answers on residual risk. The checks below are grouped in that order, each with the evidence to ask for.

Two reviewers, an older man and a younger woman, comparing a printed checklist with highlighted passages on a monitor in a dark office

Part 1

Scope and policy

Settle these before any data is processed. They decide what the rest of the checklist is measured against.
  1. 01A written de-identification policy with a version numberEvidence: the policy document, referenced in the datasheet
  2. 02A list of identifier types the policy covers, and the token for eachEvidence: token table, as on the schema page
  3. 03Categories removed outright rather than tokenised, such as health, union membership and credentialsEvidence: removal list with counts
  4. 04Sources and mailboxes excluded before export, with the reasonEvidence: exclusion list signed by the licensor
  5. 05Whether tokens are numbered per conversation or across the datasetEvidence: a sentence in the datasheet

Part 2

Method

How identifiers were found. Good methods combine sources, because each one misses different things.
  1. 06People lists built from the source system's own directory, headers and member listsEvidence: description of list building, with counts
  2. 07Name variants covered: nicknames, initials, handles, misspellingsEvidence: examples of variants matched
  3. 08Pattern rules for structured identifiers: email, phone, IBAN, order and account numbersEvidence: rule list
  4. 09A trained entity recogniser for the languages in the setEvidence: model name and language coverage
  5. 10Metadata handled: file properties, comments, tracked changes, signaturesEvidence: list of metadata fields stripped
  6. 11Dates shifted by a constant offset and quasi-identifiers generalisedEvidence: rule stated, offset withheld

Part 3

Evidence to request

What the supplier should hand over with the sample. Claims without evidence are not a basis for sign-off.
  1. 12Leak-scan results on the final files, by identifier typeEvidence: counts before and after, per type
  2. 13A human-reviewed random sample, with its size and error rateEvidence: review sheet
  3. 14Recall measured on a labelled test set in each languageEvidence: test set description and scores
  4. 15Records dropped for risk, with counts and reasonsEvidence: drop log summary
  5. 16Who had access to the raw data, and when the raw copy is deletedEvidence: access list and deletion date

Part 4

Your own sample review

Run these yourself on a random sample. They take a day and catch most problems a supplier's report hides.
  1. 17Search for raw email addresses, phone numbers and URLs with personal paths
  2. 18Read greetings, sign-offs and quoted lines, where names hide most often
  3. 19Check that one person keeps one token within a conversation
  4. 20Look for client and project names in labels, channel names and subjects
  5. 21Try to identify the licensing organisation from the text alone
  6. 22Confirm that removed categories are really absent, not just rarer

Part 5

Residual-risk questions

Nothing is risk-free. Ask the supplier to answer these in writing, in the datasheet.
  1. 23What could someone who knows the organisation still learn from the data?
  2. 24Which roles are held by one person, and how were they generalised?
  3. 25Could records be linked to public sources such as press releases or court filings?
  4. 26Was a re-identification test run, by whom, and with what result?
  5. 27What happens if an individual asks for their data to be withdrawn after delivery?

The legal frame, briefly

Under the GDPR, replacing names with tokens is pseudonymisation: the data remains personal data for anyone who can reasonably link it back to people. Truly anonymous data falls outside the regulation, but the bar for anonymity is high. The European Data Protection Board has published guidance on pseudonymisation (Guidelines 01/2025), and the Court of Justice looked at the recipient's position in EDPS v SRB, C-413/23 P: whether pseudonymised data is personal data for a recipient depends on whether that recipient can reasonably re-identify the people in it.

For a buyer, the practical consequence is that evidence and contract terms carry weight. What the supplier did, what you can and cannot do with the data, and what the licence forbids (re-identification, linking with other sources) together determine the risk you take on. Treat this page as a technical checklist, not legal advice, and have your privacy counsel set the bar.

Readability is part of the requirement

De-identification that destroys the conversation is a failure too. Over-redacted text, where every capitalised word became a token, teaches a model nothing about how people work. When you review a sample, check that roles, products, tools, amounts and the order of events survived. The schema uses typed tokens numbered per conversation for exactly this reason.

Where this goes in the paperwork

Section 6 of the datasheet template records the method, the evidence and the residual risk. The licence should reference that section, ban re-identification, and say what happens when a leak is found. The procurement page covers the clauses.

A privacy engineer examining highlighted text segments on a large monitor in a dark office at night

FAQ

De-identification questions

Is de-identified enterprise data still personal data under the GDPR?

It can be. Pseudonymised data is personal data for anyone who can reasonably re-identify people. Whether it is personal data for you as the recipient depends on what you and others can realistically do with it, which is why evidence and contract terms matter.

What evidence should a lab ask for?

Leak-scan results on the final files, a human-reviewed random sample with its error rate, recall measured on a labelled test set per language, the drop log, and a written residual-risk statement.

Should names be removed or replaced with tokens?

Replaced with typed tokens in most cases. Removing them breaks the structure of conversations. Tokens keep who said what to whom while hiding who the people are.

How large should a buyer's own review sample be?

Large enough to cover each source, language and record type. A few hundred records spread across those groups is a practical start; increase it where the first pass finds problems.

Which data should be removed rather than tokenised?

Content that harms people even without a name: health, sexual life, union membership, disciplinary and HR matters, and credentials. Legal privilege is usually excluded too.

Who is responsible if a leaked identifier is found after delivery?

The licence should say. Typical terms require the buyer to report it, the supplier to issue a corrected version, and both to delete the affected records from working copies.

Can an off-the-shelf PII detector do the job?

It is a useful layer, not a method. Generic detectors miss nicknames, internal identifiers, client names and language-specific patterns. Lists built from the source system's own directory catch much of what they miss.

Keep reading

Datasheet templateSection 6 records the method and evidence.The JSONL conversation schemaTyped tokens, numbered per conversation.EU AI Act training dataWhere personal data appears in the summary.

Next step

Need a dataset that comes with this evidence?

Name the data type, languages, period, volume and intended use. We answer by email with what can be assembled and how it would be documented.

Request a datasetBrowse spec sheets