Spec sheet / Email
Enterprise email dataset spec
An enterprise email dataset is a set of business threads rebuilt from mailbox exports: each object is one thread with ordered messages, sender and recipient tokens, the subject, and an outcome where one exists. Quoted history and signatures are handled before delivery. This spec covers threading, header fields, email-specific de-identification traps, size drivers and uses.
Common sources
OutlookOne line holds
Format
Volume
Record structure
How this type uses the shared conversation schema. Fields not listed here follow the general definition.
| Field | What it holds |
|---|---|
| source.kind | email_thread |
| labels.subject | Subject line, de-identified, taken from the first message |
| turns[].to | To and Cc as tokens; Bcc is dropped |
| turns[].direction | inbound, outbound or internal, decided before names were replaced |
| turns[].text | Only the new text of each message; quoted history removed |
| turns[].attachments | File types and sizes, never the files |
Threads have to be rebuilt
A mailbox export is a pile of individual messages, each carrying copies of earlier ones in quoted form. The useful unit is the thread, rebuilt from the Message-ID, In-Reply-To and References headers, with subject matching only as a fallback. Each message in the rebuilt thread should carry only what was new in it. Otherwise a ten-message thread contains the first message ten times, and a model learns to repeat itself.
Inline replies are the trap. People answer point by point inside the quoted text, and a naive quote stripper deletes their answers along with the quote. Ask how the supplier handled this and look for it in the sample.
Export routes
Organisations on Google Workspace can export mail with Google's Data Export tool (Google's documentation). Microsoft 365 tenants usually go through a Purview eDiscovery export (Microsoft's guide). Older archives arrive as mbox, EML or PST files. The route matters for diligence: it should be run by the organisation's administrator, on mailboxes it has the right to license.
Where identity hides in email
Email repeats identity in more places than any other source: From, To and Cc headers, greetings, sign-offs, signatures with phone numbers, legal disclaimers naming the company, forwarded headers inside the body, and display names in quoted lines. The headers make a good starting list of people and organisations. The body still needs a full pass, because customers and suppliers are often named only in the text. The checklist lists what to test.
Writing the spec
Say which functions you care about (sales, purchasing, finance, customer service), whether external threads are required, which languages, and what period. Set a limit on thread length if you need it, and say whether threads that drift between languages are welcome. They are common in companies that trade across borders, as the non-English data page explains.
Size, sample checks and questions to ask
What drives size
- Number of mailboxes and the roles behind them
- Years of mail kept on the server or in archives
- Share of newsletters, notifications and receipts filtered out
- Share of threads with outside parties
- Mailboxes excluded at source, such as HR, legal and board
Checks on a sample
- Threads rebuilt from message headers, checked by hand on a sample
- Quoted text removed without losing replies written inline
- Signatures, disclaimers and footers gone
- Automated mail filtered, with the rule documented
- No addresses left in headers, bodies or forwarded blocks
- Direction labels correct on a sample
Ask the supplier
- Which mailboxes were exported and which were excluded, and why
- The export route, such as Google Workspace Data Export or Purview eDiscovery
- How threads were rebuilt and how forwarded chains were split
- How personal mail inside work mailboxes was found and removed
- Whether calendar invites and auto-replies were filtered
Where email data fits in training
Email agents
SFT on business writing
Evals
RL environments
FAQ
Email datasets: questions
How are email threads rebuilt from a mailbox export?
From the Message-ID, In-Reply-To and References headers, with subject matching only as a fallback. Each message then keeps only its new text, so earlier messages are not repeated.
What happens to quoted text and signatures?
Quoted history is removed from each message because the thread already holds it. Signatures, disclaimers and footers are removed. Replies written inline inside a quote must be kept, which is worth checking on a sample.
Are attachments included in email datasets?
Usually as a list of file types and sizes. If attachments matter, ask for them as a separate document set, converted and scrubbed like any other document.
Which mailboxes should be excluded?
Mailboxes whose content is mostly sensitive: HR, legal, board and medical functions. Personal mail inside work mailboxes should be found and removed too.
How do I know if a thread is internal or external?
Ask for a direction field per message, decided from the email domains before they were replaced with tokens. After replacement it can no longer be worked out.
Keep reading
Next step
Need email data built to this spec?
Send the languages, period, functions and intended use. We reply by email with what can be assembled.