For data acquisition and post-training teams
What should an enterprise dataset contain, how should it be documented and de-identified, and how do you source one? This site answers that from the buying side, with spec sheets, an open JSONL schema and a datasheet template. When you have a spec, we assemble the dataset, de-identify it and deliver it under a licence.
> spec / schema / datasheet / checks / licence
How it works
Common source systems
Format
Evidence
Paperwork
Spec sheet
| Type | Common sources | One line holds | Format |
|---|---|---|---|
| Workplace chat | SlackMicrosoft Teams | One thread or one channel-day per line | JSONL in the published schema |
| Support tickets | ZendeskFreshdeskIntercomJira | One ticket per line | JSONL in the published schema |
| Source code | GitHubGitLabBitbucketJira | Repositories as git bundles, reviews and issues one per line | git bundles plus JSONL |
| GmailOutlook | One rebuilt thread per line | JSONL in the published schema | |
| Documents | SharePointOneDriveGoogle DriveNotionConfluence | One Markdown file per document, one index line each | Markdown plus JSONL index |
| CRM and process records | SalesforceHubSpotPipedrive | One deal, case or process instance per line | JSONL or tables, specified per request |
Top-level fields
Full definition on the schema page.
Example line (fictional)
{
"schema": "aidb.conversation/1.0",
"id": "c-0001",
"dataset": "fictional-example",
"example": "Fictional record written for illustration. No real company, person or ticket.",
"source": {
"system": "zendesk",
"kind": "support_ticket",
"channel": "email"
},
"language": "nl",
"period": {
"start": "2023-03-06T08:14:00Z",
"end": "2023-03-07T10:02:00Z"
},
"participants": [
{
"ref": "<PER_1>",
"side": "customer",
"role": "office manager"
},
{
"ref": "<PER_2>",
"side": "internal",
"role": "support agent"
}
],
"labels": {
"priority": "normal",
"category": "invoice"
},
"turns": [
{
"n": 1,
"at": "2023-03-06T08:14:00Z",
"from": "<PER_1>",
"kind": "message",
"visibility": "public",
"text": "Goedemorgen, op factuur <ID_1> staan 40 dozen printpapier, wij hebben er 4 besteld. Kunnen jullie dit corrigeren?"
},
{
"n": 2,
"at": "2023-03-06T09:40:00Z",
"from": "<PER_2>",
"kind": "note",
"visibility": "internal",
"text": "Orderregel handmatig ingevoerd, typefout. Creditnota aanmaken."
},
{
"n": 3,
"at": "2023-03-07T10:02:00Z",
"from": "<PER_2>",
"kind": "event",
"event": {
"field": "status",
"value": "solved"
}
}
],
"outcome": {
"status": "solved",
"code": "credit_note",
"reopened": false,
"closed_at": "2023-03-07T10:02:00Z"
},
"links": [],
"deid": {
"policy": "v1",
"tokens": {
"PER": 2,
"ID": 1
},
"date_offset": "constant per dataset",
"review": "automated + human sample"
},
"provenance": {
"licensor": "L-EXAMPLE",
"route": "admin export",
"licence": "LIC-EXAMPLE"
}
}Written for illustration. In the file each object sits on one line.
Pseudonym tokens
<PER_n>A person: name, nickname, handle, initials or display name<ORG_n>A customer, prospect or the licensing company itself<EMAIL_n>An email address<PHONE_n>A phone number<ADDR_n>A street address<ID_n>Account, order, invoice or customer number<URL_n>A link that points to a person or an internal system<SECRET>A credential or key, never numbered5
Spec sheets
Workplace chat, email, support tickets, source code and documents, each with checks and uses.
13
Top-level fields
In the open conversation schema, with a fictional example file you can load today.
8
Token types
Typed pseudonyms that keep text readable while people and organisations stay unnamed.
Before you sign
Training data deals stall on rights and evidence more often than on the data itself. These are the points a lab's counsel and privacy lead will raise, so settle them in the first draft.
How labs procure enterprise data
Two ways to use this site
Open reference
Everything on this site is free to use inside your own team, with or without a request.
Built to your spec
Send the spec. We say plainly whether we can assemble it, and in what shape.
Guides
FAQ
A reference site for teams that buy enterprise data for model training: spec sheets per data type, a JSONL schema, a datasheet template, a de-identification checklist and pages on the AI Act, RL environments and procurement. You can also request a dataset assembled to your spec.
Records that organisations produce while working: team chat, email, helpdesk tickets, source code with history, business documents and CRM records. None of it is public, which is why it has to be licensed from the organisation that holds the rights.
Yes. The schema, the example file, the datasheet template and the checklist are published for anyone to use and adapt inside their own team, whether or not you ever send a request.
You describe the data type, languages, period, volume, intended use and licence. We reply by email with what can be assembled and how. Datasets are assembled to spec, de-identified, documented and delivered under a licence with a datasheet.
No. Each dataset is assembled for a specific request, so there is no standing catalogue with volumes. The datasheet of a delivered set states its record and token counts.
Yes. The specs, schema and checklist apply anywhere. The AI Act page matters for any provider placing a general-purpose model on the EU market, wherever it is based.
Next step
Name the data type, languages, period, volume and intended use. We answer by email with what can be assembled and how it would be documented.
