The CAT Tool Productivity and Quality Dilemma: The Real Balance Sheet

The oldest question in the debate about translation technology is this: does the tool speed the translator up, or does the translator slow down in order to adapt to the tool? CAT tools do genuinely change the balance between productivity and quality, but the change is not as one-directional as the advertising makes it sound.

Contents

The answer varies with the type of text, the repetition rate of the project, the experience of the team and the pricing model. The same tool may deliver a clear gain on a technical manual and contribute almost nothing to a piece of advertising copy.

This article looks at the data behind the productivity claim, at the findings on the quality side, at who ends up keeping the gain, and at how the buyer of translation can make use of this balance sheet.

Where does the productivity claim come from?

The productivity claim made for CAT tools rests on a single mechanism: when a sentence you have translated before comes round again, you do not translate it a second time. That mechanism produces a gain in proportion to how repetitive the content is.

Repetition rates are higher than people assume. According to DePalma’s 2012 study, roughly 59 per cent of translated content consists of entirely new words, while the remaining 40 per cent or so can be reused in one form or another (2012 data). This proportion is very similar in in-house translation departments and among freelance translators.

The second source of gain is the disappearance of typing and formatting work. The tool preserves the formatting of the source file, so the translator does not have to wrestle with tables, bullet points and style settings. In long documents this is a larger item of time than people imagine.

Considered together, these two items show that part of the gain has nothing to do with translation at all. The tool is productive to the extent that it frees the translator from non-linguistic work, which means the gain varies with the complexity of the document.

The third source is the automation of quality control. Numerical inconsistencies, empty segments, term violations and double spaces are all scanned by machine. Done by hand, these checks take hours and are still left incomplete.

What do the figures say?

Industry data show that productivity depends not on the individual but on the way work is organised. In DePalma’s 2012 measurements, the hourly output of freelance translators averaged 443 words, while the hourly output of translation companies stood at 5,728 words (2012 data). The difference comes not from the speed of a single translator but from several people and automated steps working together.

Daily output presents a more measured picture. The same study puts translators’ average daily output at 2,684 words, against the widely accepted industry benchmark of 2,500 words (2012 data). Tools have therefore not multiplied daily output; they have lifted it to a level slightly above the industry benchmark.

Read together, these two sets of figures produce the following picture: the real gain from CAT tools lies not in individual speed but in scalability. Delivering a high-volume job in a short time becomes possible not because one translator writes faster, but because the work becomes divisible while remaining consistent.

Another indicator is the proportion of translators who make a living from the profession. In the case study the book carried out with 141 Polish translators, 75 per cent of those using a CAT tool earned their living from translation alone, while among non-users the figure stopped at 41 per cent (2021 data).

Is there really a CAT tool productivity and quality dilemma?

The best-known study on the quality side belongs to Lynne Bowker. In a pilot study published in 2005, Bowker showed that unquestioning trust in the translation memory multiplies errors (2005 data). Flawed records held in the memory had been carried over into the participants’ translations to a considerable extent.

This finding does not prove that there is a necessary trade-off between productivity and quality. What it shows is subtler: quality falls when the gain in speed takes the place of the checking steps; when those steps are kept, the gain is obtained without paying a price in quality.

In practice the damage usually comes through pricing. When full matches are counted as discounted or free, there is no longer any economic reason to read those segments carefully. The cause of the drop in quality is not the tool but the commercial model built around it.

To this a cognitive factor is added. The 2017 study by O’Brien and colleagues compiled the ways in which irritating features of tool interfaces draw the translator’s attention away from the text (2017 data). Part of the time saved goes back into dealing with the tool.

Who keeps the gain?

The aspect of the productivity debate most often skipped is how the gain is shared. The tool licence is usually bought by the translator; in the book’s case study, 91 per cent of translators own their own licence (2021 data). Against that, match discounts pass part of the saving on to the client and to the intermediary.

This sharing is not unjust in itself. When the discount scale is reasonable, the translator protects their income by delivering more work in less time, and the client does not pay full price a second time for repeated content. The balance depends on the scale being realistic.

The problem begins at the point where the discount disregards the cost of checking. If a full match is going to be passed over unread, the discount makes sense; but if a context check is needed, that work has been done unpaid. In the long run this leads to checking being quietly abandoned.

This is why serious quotations define the discount scale and the scope of quality work together. Which type of match carries which discount, and which checks will be carried out in spite of those discounts, are written out on separate lines.

The balance sheet varies with the type of text

Expressing the productivity gain as a single figure is misleading. The table below summarises the gains that can be expected on different types of job and the risks that need watching.

Type of job	Repetition rate	Productivity gain	Main risk
Technical manual update	Very high	High	Outdated records being carried over
Set of contracts and legislation	High	High	A single changed word being missed
Product catalogue	Medium	Medium	Loss of context in short segments
Corporate website	Medium	Medium	Style drifting apart
Press release	Low	Limited	Overstating the gain
Advertising and brand copy	Very low	Next to none	Mistaken price expectations

The bottom rows of the table matter. On texts without repetition it is not realistic to expect a match discount; on that kind of job the tool contributes not speed but terminological consistency and automated checks.

The repetition rate is a quantity that can be measured at the start of a project. When the document is loaded into the translation tool, the tool produces an analysis report showing internal repetitions and memory matches. If that report is shared as an annex to the quotation, the argument about expectations is closed before it starts; the client sees in figures where the gain comes from.

How should the balance sheet be read?

A balanced reading requires three distinctions to be maintained. The first is the difference between speed and scale: tools make a single translator somewhat faster, but they scale teams up far more.

The second is the difference between the first project and the ones that follow. When the memory is empty there is no gain; the first project may even take longer because of the work of preparing a term list and templates. The gain becomes visible on the second and third projects. If you are met with a promise of productivity for a one-off job, you are right to question it.

The third is where the gain is invested. If the time saved is passed on entirely in the price, no resource is left for the quality steps. The healthy model is to put part of the gain into the price and part into final review and context checking.

In short, CAT tools neither raise nor lower quality; they make the process that determines quality cheaper. How that cheaper process is used is a management decision. You can compare the differences in scope on the pricing page.

The Türkiye context

The first factor that complicates the productivity debate in the Turkish market is the unit of pricing. Established practice is a page count based on characters, whereas CAT tools work on words and segments. On the conversion measure the book gives, 1,800 characters with spaces correspond to roughly 240 words (Pietrzak & Kornacki 2021, p. 83). Where no conversion is made between the two units, match discounts become meaningless.

The second factor is the structure of the language. Because Turkish is agglutinative, the same content is expressed in fewer words than in English. A price per word gives a misleading result when the target language is Turkish and the count is not made on the source text. A quotation must therefore state clearly which language it counts.

The third factor is document structure. In Türkiye a significant proportion of the documents that come up for translation are official papers, and most of them arrive as scans. In such documents a memory gain only appears once the text layer has been extracted properly; otherwise part of the productivity gain is spent at the preparation stage.

What does this mean for you?

For the buyer, the right expectation is this: CAT tools do not make every job cheaper, they make repetitive jobs cheaper. Placing your order with that distinction in mind keeps both your budget and your quality expectations realistic.

1. Gather similar documents into a single project; ordering piece by piece wipes out the repetition gain.
2. Ask for the match discount scale and the checks that will be carried out despite those discounts to be set out on separate lines of the quotation.
3. Do not expect a discount on one-off creative texts; the value there is sought not in speed but in fitness for purpose.
4. Confirm that your memory is kept separately in your name; a gain that accumulates in a shared memory does not come back to you.
5. Rather than expecting a gain on the first project, set time aside for the term list and for decisions about style; the return comes on later jobs.

To see the repetition rate of your document set and the scope that corresponds to it, you can share your files through the quotation page.

Frequently asked questions

Do CAT tools really speed translation up?

On texts with a high repetition rate they speed it up markedly. On one-off, creative texts with no repetition the gain is limited; on that kind of job the tool’s real contribution is not speed but terminological consistency and quality control.

Does the increase in speed lower quality?

There is no necessary drop. Bowker’s 2005 pilot study showed that errors are carried over when the memory is trusted without question. When the checking steps are kept, the increase in speed can be obtained without any loss of quality.

Why are match discounts applied?

They rest on the assumption that full matches coming from the memory require less work. But full matches also need to pass a context check; the discount scale should therefore be kept within reasonable limits and stated clearly in the quotation.

As a client, how do I benefit from the productivity gain?

By gathering document sets with a high repetition rate into a single project and by making sure your memory is stored in your name. On the second and third projects the gain becomes far more visible than on the first.

Sources

• Pietrzak, P. & Kornacki, M. (2021). Using CAT Tools in Freelance Translation: Insights from a Case Study. London: Routledge Focus, pp. 71–95.
• Bowker, L. (2005). Productivity vs quality? A pilot study on the impact of translation memory systems. Localisation Focus, 4(1), pp. 13–20.
• DePalma, D. A. (2012). Translation Output Metrics. Common Sense Advisory industry research.
• O’Brien, S., Ehrensberger-Dow, M., Connolly, M. & Hasler, M. (2017). Irritating CAT tool features that matter to translators. HERMES, 56, pp. 145–162.