Machine Translation 5 August 2026
Statistical Machine Translation (SMT) and the Age of Data
Giving up rule-writing for learning from data was the biggest change of direction in translation technology. The logic of the statistical approach, its strengths and the legacy it left behind.
The sharpest change of direction in the history of machine translation came in the 1990s. The field gave up trying to describe language through rules and turned instead to learning probability from translated text. This approach is called statistical machine translation, abbreviated to SMT.
Contents
The origin of the idea is in fact far older. The Weaver memorandum of 1949 had already suggested that the statistical methods used in code-breaking could be applied to language (Pietrzak & Kornacki 2021, Chapter 3). What the idea needed in order to become workable was enough computing power and enough text.
In this article we look at how the statistical approach works, what it changed in the industry, and at what point it gave way to neural models.
What does statistical machine translation rest on?
The starting point of the statistical approach is this: if you do not know the correct translation of an expression, look at how people have translated it before. Given enough examples, the most frequently used rendering is in all likelihood the right one.
The system therefore knows no grammar. It does not recognise the subject or the predicate of a sentence. It only counts how often one sequence appears alongside another, and builds probability tables from those counts.
When a new sentence comes in, the system generates possible translation candidates and gives each one a probability score. The highest-scoring candidate is chosen as the output. This is translation recast as a search problem.
The appeal of the approach is that it is language-independent. The same method can be applied to any language pair for which you have enough data. The expert work that rule-based systems required from scratch for every language pair disappears here.
The fuel: the parallel corpus
The only thing a statistical system needs is a parallel corpus: the versions of the same text in two languages, matched sentence by sentence. The larger and the cleaner the corpus, the better the system learns.
This requirement also determined which languages the field would advance in. The official documents produced by multilingual institutions became the source of the first large corpora. The texts that European Union institutions and the Canadian parliament produce in two or more languages were unmatched in this respect.
The result was an inequality between languages. Languages with a heavy volume of institutional multilingual output advanced quickly; languages with no such source were left behind. This divide is discussed in the literature under the heading of “resource richness” and still holds today.
The cleanliness of the corpus matters at least as much as its size. Wrongly matched sentences lead the system to learn the wrong patterns. Alignment is therefore one of the most critical steps in a statistical translation infrastructure.
How it works: three components
The phrase-based models in common use have three components. The first is the translation model: it holds which source phrase corresponds to which target phrase, and with what probability.
The second is the language model. This component takes no interest at all in the source language; it only knows which word sequences are natural in the target language. Its job is to pick the most fluent of the candidates the translation model produces.
The third is the reordering component. Because element order differs from language to language, the order in which the phrases will appear in the target sentence is calculated separately. This is the component that struggles most with structurally distant languages.
| Component | What does it learn? | Where it stays weak |
|---|---|---|
| Translation model | Phrase correspondences and their probabilities | Terms that do not occur in the corpus |
| Language model | Natural sequences in the target language | Long-distance dependencies |
| Reordering | The order of the phrases in the target sentence | Structurally distant language pairs |
Hybrid systems: the two approaches combined
The spread of the statistical approach did not wipe out rule-based systems overnight. For a long period, hybrid set-ups that combined the two approaches were in use.
The most common set-up was this: the statistical engine produces the output and a rule-based layer corrects it. Term renderings are enforced, number and date formats are fixed, and formatting tags are put back in place.
Another set-up worked the other way round: the source text was put through rule-based pre-processing so that the statistical engine would cope with it better. Long sentences were split up and ambiguous pronouns were replaced with explicit nouns.
The shared lesson of these set-ups still holds today: changing the engine is not the only route to a solution. Tidying up the input and checking the output in a rule-governed way often brings a bigger gain than changing the engine, and costs far less.
What did it change in the industry?
The most visible effect of the statistical approach was that it put machine translation back on the agenda. The interest that had faded after ALPAC came back with systems that produced usable output.
In industry terms this meant machine translation entering the translator’s workflow. In a 2012 industry survey, the proportion of companies stating that they had at least tried machine translation was measured at 44.02%, and the proportion of freelance translators at 55% (DePalma 2012).
The second change concerned the job description. Correcting machine output became a separate service line. The skill this work calls for differs from translating from scratch, and in time it developed its own criteria and its own pricing logic.
The third change was data culture. Organisations realised that their past translations were an asset. The value of translation memories and archives rose not only for reuse but also for engine training. That realisation brought the debate over data ownership with it.
Weaknesses
The best-known flaw of statistical systems was that they processed a sentence by breaking it into pieces. Even when each piece was translated sensibly on its own, the join could come out broken. In long, nested sentences the problem became obvious.
The second flaw was dependence on data. A term or a field that does not occur in the corpus is invisible to the system. An engine trained on general text performs far worse than expected in a specialist field.
The third flaw was long-distance dependencies. The relationship between an element at the beginning of a sentence and one at the end is usually lost in a phrase-based model. In languages such as Turkish, which place the predicate at the end, this limit is felt particularly keenly.
The fourth flaw was morphology. The system counts words as indivisible units. In languages where a single stem yields a large number of inflected forms, every form counts as a separate word, so the data thins out quickly and learning becomes harder.
From SMT to neural models
From the middle of the 2010s onwards the industry moved to models based on artificial neural networks. Because these models represent the sentence as a whole rather than piece by piece, the problem of joining phrases together largely disappeared.
The transition was not abrupt, and the legacy of the statistical era was not lost either. Parallel corpus collection, alignment, data cleaning and evaluation practices all matured in this period and still make up the infrastructure of today’s systems.
Another legacy is a way of thinking. The statistical era established the idea that translation quality is something measurable. Comparing engines against one another, demonstrating improvement in numbers and the habit of regular evaluation all come from here.
A third legacy amounts to a warning. The statistical era showed that data quality can be more decisive than the choice of model. An advanced model trained on a dirty corpus can fall behind a simple model trained on a clean one.
This observation holds at organisational level too. If your own archive is disorganised, which tool you choose matters less. The first investment in translation technology is usually not software but order in the data.
The lasting legacy of this transition is the idea that translation quality is now determined not only by knowledge of language but also by the breadth of the data pool the system has seen. When data is scarce, even the most advanced method remains limited; this is the basic factor that explains why your language pair gives different results.
The Türkiye context
Two basic assumptions of the statistical approach come under strain in Turkish. The first is the assumption that the word is an indivisible unit. Turkish is an agglutinative language; the layers of suffixes added to a stem can produce a great many forms. When each form counts as a separate word, a corpus of the same size yields far fewer examples than it would in English.
The second is the assumption that element order is relatively fixed. In Turkish, element order can change according to emphasis and the predicate usually comes last. For a component that reorders phrases, this means a task that is both longer-range and more variable.
On top of this comes a resource limit. In terms of the volume of processed parallel text, Turkish lags behind the largest languages. In Türkiye a significant share of corporate translation is kept as individual files and never turned into a corpus. Putting the in-house archive in order therefore makes a bigger difference in Turkish projects than it does on the international average.
What does this mean for you?
The most concrete thing the statistical era has left you is this: your past translations are an asset. An archive sitting around as scattered files both lowers cost and raises consistency once it is put in order.
- Gather the translation files of the past three years in one place, with the source and target pairings intact.
- Have the matching file pairs aligned and turned into a translation memory.
- Write into your supplier contract that ownership of the memory belongs to you.
- Share the memory in advance on every new project and ask for a discount on the repeated sections.
These four steps are usually a few days’ work and their effect is lasting. We can assess together whether your archive is in an alignable state; you can see how pricing is affected by the repetition rate on our pricing page, and you can send us your files from here.
Frequently asked questions
What is statistical machine translation?
It is the approach that learns from large volumes of translated text which expression is matched with which rendering and how often, and then calculates the most likely translation for a new sentence. It rests on counting rather than on rule-writing.
What does parallel corpus mean?
It is the versions of the same text in two languages, matched sentence by sentence. Statistical systems learn from these matches; the larger and the cleaner the corpus, the better the result.
Why did SMT output sometimes produce broken sentences?
Because the system split the sentence into pieces and matched each piece separately, the combination could come out broken even when the pieces were individually correct. In long, nested sentences the problem was more visible.
Is SMT still used today?
The mainstream approach now is neural models. But the infrastructure the statistical method brought with it, particularly the practice of collecting and aligning parallel corpora, still forms the foundation of today’s systems as well.
Sources
- Koehn, P. (2010). Statistical Machine Translation. Cambridge University Press, pp. 3–35, 127–160.
- Pietrzak, P. & Kornacki, M. (2021). Using CAT Tools in Freelance Translation. Routledge Focus, Chapter 3.
- DePalma, D. A. (2012). Translation Productivity Benchmarks. Common Sense Advisory.