Guide / languages
Non-English enterprise data for LLM training
Non-English enterprise data is work writing in languages such as Dutch, German, French, Polish, Spanish, Italian and Swedish: chat, email, tickets and documents produced by organisations that operate in them. It fills a gap public web text leaves, because business register and jargon are rare online. Buy it with per-language coverage figures and per-language de-identification evidence.
What public text misses
For most languages other than English, the open web is thin and uneven. It holds news, government pages, forums and a large share of translated material. What it rarely holds is the way people write at work: a purchasing assistant asking a supplier for a split delivery, an IT team arguing about a release in a mix of Dutch and English, a Polish accountant explaining a correction to a client. Models trained mostly on web text handle those registers less well than English ones, and the gap shows up as stiff phrasing, wrong forms of address and missed jargon.
Enterprise records fill that gap because they were written by native speakers doing real tasks. They also carry the conventions an assistant must follow to be useful in a business: when to use the formal address, how a complaint is phrased, how a German invoice query differs from a Spanish one.
Coverage questions to ask
- Volume per language. Tokens and records, after filtering, not before.
- Regional variants. Belgian Dutch and Netherlands Dutch, Austrian and Swiss German, Latin American and peninsular Spanish behave differently.
- Register mix. Internal chat, formal email and customer-facing replies in proportion.
- Sectors and functions. One language from one sector teaches that sector's jargon, not the language at work.
- Code-switching. The share of conversations that move into English, tagged per turn.
- Machine translation. Any translated text flagged, since it carries the source language's structure.
Per language
Language-specific checks
| Language | Tags | Name trap | ID to scan for | Register |
|---|---|---|---|---|
| Dutch | nl, nl-BE | Lower-case particles inside surnames (van, de, van der) are easy to miss | BSN, 9 digits | u versus je; Flemish and Netherlands usage differ |
| German | de, de-AT, de-CH | Names inside compounds and hyphenated project names | Steuer-ID, 11 digits | Sie versus du; Swiss spelling without ß |
| French | fr, fr-BE, fr-CH | Elided forms before names, as in d'Anne or l'équipe de Marc | NIR, 15 digits | vous versus tu; formal letter conventions |
| Polish | pl | Names inflect by case, so one surname appears in several forms | PESEL, 11 digits | Pan and Pani forms of address |
| Spanish | es, es-419 | Two surnames and compound given names | DNI and NIE | usted versus tú; regional vocabulary |
| Italian | it, it-CH | Surnames that are also ordinary words | Codice fiscale, 16 characters | Lei versus tu |
| Swedish | sv, sv-FI | Patronymic endings and very common surnames | Personnummer, date plus 4 digits | du is near universal; formality lives in tone |
Sourcing it properly
Non-English enterprise data comes from organisations that work in the language, licensed by the party that holds the rights and exported with each tool's own admin export. Most of it sits in the EU, so the GDPR shapes how it is processed: de-identification before it leaves the licensor's control, documented residual risk, and a licence that forbids re-identification. The checklist applies in every language, but the recall figures should be reported per language, since a method tuned on English text can miss half the names in Polish.
Tagging and splitting
Ask for a BCP 47 tag on every conversation and on any turn in another language, as the schema provides. For evaluation, split by organisation rather than by record, so the test set does not share people, products and jargon with the training set. For low-volume languages, a smaller set with clean documentation is worth more than a large one with unclear origin.
Writing the spec
Name the languages and variants, the minimum tokens per language after filtering, the data types (email and chat are the richest for register), the sectors, and how much English mixing you accept. Ask for per-language recall in the de-identification report and per-language counts in the datasheet.
FAQ
Non-English data questions
Why does non-English enterprise data matter for LLM training?
Because public text in most languages skews towards news, forums and translated pages. Work writing in Dutch, German or Polish has its own register, jargon and conventions, and models that serve businesses in those languages need to see it.
Which coverage questions should a buyer ask per language?
Tokens and records per language, regional variants, the mix of registers, sectors and functions, how often people switch to English mid-conversation, and whether any text was machine-translated.
How is language tagged in enterprise datasets?
With a BCP 47 tag per conversation, and a tag per turn where a turn is in a different language. Regional tags such as de-AT or nl-BE matter when your evaluation does.
Is de-identification harder in some languages?
Yes. Polish names change form by grammatical case, Dutch surnames carry lower-case particles, Spanish names have two surnames. Detection lists and models must be built for each language and tested on it.
Should mixed-language conversations be kept?
Usually yes. Switching to English for technical terms or with foreign colleagues is how many European teams actually write. Keep it, tag it, and measure its share.
Does non-English data have to stay in the EU?
Not because of its language. If it still contains personal data, GDPR transfer rules apply when it leaves the EU. Well de-identified data with documented residual risk changes that analysis; your counsel decides.
Keep reading
Next step
Looking for work data in a specific language?
Name the data type, languages, period, volume and intended use. We answer by email with what can be assembled and how it would be documented.