Translator Research: The Method and Scope of a Case Study of 141 People

Most writing on translation technology opens with “translators do this” and never says where the claim comes from. The translator research behind this series is open about it: a survey carried out in Poland over the 2019 period with 141 participating translators. The number, the country and the date are all on the record.

Contents

That clarity is good news, but it brings a responsibility with it. Percentages drawn from a sample of 141 people do not count as evidence about the translation market in Türkiye. At most they help us ask the right questions.

Below we set out how the study was built, who took part, what was measured and, just as importantly, what could not be measured.

What question was the translator research built on?

The study set out from a simple observation: computer-assisted translation tools have been in the industry for thirty years, yet few studies measure systematically the difference between the professional circumstances of translators who use them and those who do not. Industry reports usually describe translation companies; the freelance translator is a scattered population and is rarely asked anything one by one.

Freelancers, however, make up the main body of the industry: a weighted average of studies from different countries shows that roughly 78.4% of translators work freelance (Pym et al. 2012). If you want to understand the effect of technology on the industry, then, you have to look at this scattered population rather than at in-house translation departments.

The research set out to fill that gap. Its aim was not to establish a norm but to place the working practices of two groups, tool users and non-users, side by side and see where the differences became pronounced. It is a descriptive case study that makes no claim to statistical representativeness.

Being descriptive is not a weakness of the study but a deliberate design choice. That choice does, however, decide the shape of the sentences we can build when we report the findings. We cannot say “this proportion produces that result”; we can say “these two groups part company at this point”. That distinction is the basic rule we shall keep recalling throughout the series.

The sample: 90 users, 51 non-users

Of the 141 translators who took part in the survey, 90 said they used a computer-assisted translation tool and 51 said they did not (Poland, 2019). This division is the backbone of the study: almost every finding was produced by comparing the two groups.

Group	Participants	Share	Limit of the comparison
CAT tool users	90	roughly 64%	Cells shrink in subgroup breakdowns
Those not using CAT tools	51	roughly 36%	Carries no national representativeness on its own
Total	141	100%	Poland, 2019; voluntary participation

It would be wrong to read the 64% here as “two-thirds of translators use CAT”. The figure is the distribution of the people who chose to take part in the survey; it is not the distribution of all the translators in a country. On a survey about technology, people who are interested in technology can be expected to answer more readily.

The control group of 51 needs to be read with particular care. Once that group is broken down by age, field of specialisation and years in the profession, ten to twenty people are left in each cell. At that scale, differences of a few percentage points can easily arise by chance.

The groups are not equal in size either. In a comparison of 90 against 51, the proportions measured in the smaller group are more volatile; a handful of people answering differently shifts the percentage by several points. In the comparison tables the figures for the larger group are steadier and those for the smaller group more fragile. This asymmetry has to be kept in mind throughout the reading of the findings.

How was the data collected?

The data was collected through a structured questionnaire put to translators. Participation was voluntary and there was no compulsory sampling frame; the researchers did not work from a randomly selected list, and the translators who came across the survey answered it of their own accord.

The method is common in the social sciences and it is legitimate, but the constraint it brings has to be named: a self-selected sample. The results describe the community that reached the survey and chose to answer it. What those who did not answer think cannot be measured.

A second constraint is that the data is self-reported. We learn about translators’ incomes, the tools they use and the problems they run into from their own statements. Nobody produces an invoice and nobody’s screen is monitored. In self-reported surveys, participants tend to present their professional circumstances in a somewhat more favourable light; this does not invalidate the data, but it does mean leaving a margin when reading it.

The third point is question design. In a survey, the way a question is framed directly affects the answer it receives. A yes to “do you use a CAT tool?” puts the translator who spends eight hours a day inside the tool and the person who opens the program a few times a year out of necessity in the same box. Intensity of use cannot be measured with binary questions of this kind.

What was measured?

The survey did not focus on a single topic; it scanned the translator’s relationship with the tool along several axes. The rest of this series takes up each of those axes separately.

1. Which tool is used: SDL Trados 43.3%, memoQ 29.1%, Wordfast 10.6% (Poland, 2019).
2. Whose name the licence is in: 91% of participants said they held their own licence (Poland, 2019).
3. Compatibility problems: 90% said they ran into problems, and 70% of those said they ran into them frequently (Poland, 2019).
4. Source of livelihood: those living on translation alone were 75% among CAT users and 41% among non-users (Poland, 2019).
5. Reasons for not using them: no need 50%, price roughly 40%, no client demand roughly 40%, complexity 25% (Poland, 2019).

All five headings are descriptive. None of them builds a cause-and-effect claim such as “using a tool raises income”; they only show what the two groups looked like at the same moment.

Each item is the subject of a separate article. What matters here is to see that the whole list rests on a single survey: the same 141 people, the same period, the same country. These findings are therefore not independent pieces of evidence but different faces of one observation. Several findings pointing the same way does not multiply their evidential force.

What does voluntary participation give, and what does it take away?

What voluntary participation gives is depth. Someone who answers a survey they were not obliged to answer usually does so because the subject interests them, and writes fuller answers to open questions. This is where the qualitative richness of case studies comes from.

What it takes away is representativeness. A survey circulated through professional networks, online groups and association mailing lists reaches the translators inside those networks. People who do not join professional networks, who work alone or who translate as a secondary occupation are systematically left out.

The same problem is sharper still in the control group. A translator who says “I do not use CAT” often sees a survey about technology as a chance to state an opinion. That can lead to an over-representation of people with firm views within the group of non-users; the undecided and those with no interest in the subject stay silent.

The practical upshot is this: the percentages in this research should be read not as estimates of prevalence but as signs of a relationship. “91% of translators in Poland buy their own licence” is too strong a sentence. “In this sample the cost of the licence appears to fall mainly on the translator” is the weight the data can carry.

2019: a snapshot from before artificial intelligence

The time dimension is the most critical limit of this study. The survey belongs to the 2019 period. Neural machine translation was widespread by then, but large language models had not yet entered the freelance translator’s daily workflow.

That difference is not trivial. The question “do you use machine translation?” did not mean in 2019 what the same question means today. Another measurement from the same period showed that 55% of freelance translators had at least tried machine translation (DePalma 2012, CSA); that figure is in all likelihood far higher today, but we have no current comparison produced by the same method.

Some findings, by contrast, can be expected to hold up better over time. Whose name the licence is taken out in, or the difficulty file formats have in speaking to one another, are not matters that change quickly as one generation of models gives way to the next. Questions about attitude and expectation are in all likelihood the part that dates fastest.

This research therefore has to be treated not as a measurement of a trend but as a dated snapshot. The value of the snapshot is that it offers a reference point against which we can measure what has changed since. We set out this whole framework in detail in the article where we take up the limits of the research.

The Türkiye context

There is no publicly available translator research in Türkiye carried out on this scale and by this method. We cannot give a comparative figure, because no such figure exists. Saying so is more honest than guessing.

The structure of the market in Türkiye also makes the sample hard to transfer directly. Poland is surrounded by a high-volume public demand for translation for European Union institutions, and that demand makes CAT tools compulsory in practice. In Türkiye, a significant part of the workload consists of document translation tied to notarisation, commercial correspondence and web content. On a short notarised document the return from a translation memory is limited.

Language structure produces a difference too. Turkish is an agglutinative language; a single word can carry information that other languages convey in a clause. This makes word-based measurements directly incomparable for Turkish. The volume and productivity figures in the Polish data cannot be transferred one to one to Turkish projects.

What does this mean for you?

For the buyer of translation the real lesson here is methodological: behind every statistic your supplier puts in front of you there is a sample, and you have every right to ask what that sample was.

1. Ask which year, which market and how many projects the productivity or saving figure you are given rests on.
2. Ask for results measured on your own document type rather than general industry averages.
3. Produce your own data with a small pilot translation; a ten-page trial is more binding than a survey of a hundred people.
4. For every quotation that promises a saving, ask in writing which repetition rate it rests on.

That is why we base our pricing not on general averages but on a real repetition analysis of your own file. You can see how we work with each document type on our services page, and if you would like a figure measured for your own file you can use the quotation form.

Frequently asked questions

How many translators took part in the research?

It was carried out with 141 translators in total (Poland, 2019). Of those participants, 90 were translators who used a CAT tool and 51 were translators who did not.

Do the findings hold for Türkiye?

Not directly. The sample is in Poland and does not represent the market in Türkiye. The findings can be used to generate questions, but not as a benchmark.

Is a sample of 141 people enough?

It is reasonable for a case study and small for generalisation on a national scale. When the sample is divided into subgroups it comes down to cells of 51 people and the margin of error grows.

Is the data still current today?

The survey belongs to the 2019 period; it falls before large language models spread through the industry. Some of the attitudes recorded then may have changed since.

References

• Pietrzak, P. & Kornacki, M. (2021). Using CAT Tools in Freelance Translation: Insights from a Case Study. Routledge Focus, Chapter 6.
• Pym, A., Grin, F., Sfreddo, C. & Chan, A. L. J. (2012). The Status of the Translation Profession in the European Union. European Commission.
• DePalma, D. (2012). Translation Productivity Benchmarks. Common Sense Advisory.