PAUL GREGORY +++ LANGUAGE FOR CLARITY

SOFTWARE to MEASURE the LINGUISTIC COMPLEXITY of TEXTS

There is an urgent need for such software. It beggars belief that, in the age of translation memories and machine translation, there is none.

I. The present situation

Increasingly, those requiring translations of any kind and length demand that a price be agreed in advance. There is a failure or unwillingness to distinguish between a fixed price and an estimate. It may be assumed that, often, the customer will then make price comparisons. The less frequent situation is where prior information is needed about the cost in order to decide whether it is worth having the translation done.

The problem faced by those who will perform the service, or possibly act as intermediary (agency, “service provider”) is how to estimate a price that is commensurate with the work that will be involved. There are always “known unknowns” in a text, not all of which would be detected by any software, but many could. A quotation may be made by including a crude margin to accommodate these “stumbling blocks”, much as a hired chauffeur paid by mileage alone might guess at how many traffic jams etc. they may encounter. But this is an unreliable method and calculating the margin too generously will mean not winning the assignment. Calculating too tightly may mean a transfer of risk to the translator, who may respond by delivering work which is less good than it needs to be. Depending on how carefully the costing is done, it involves not a little work, usually under time pressure, which often comes to nothing since the bids are unpaid (something that needs changing, possibly by statute since this affects wide areas of professional and trade work.)

A custom has established itself of making a crude count of the text quantity and multiplying this by some monetary figure. This quantity is often stated or expected on invoices even where there has not been any bidding process or estimate. A framework contract will usually specify a rate per quantity. The quantities are stated in number of words, in lines, or number of characters, with variations around counting spaces, rounding number of words to a nearest thousand, etc. There are statistical surveys on this basis.

Where it is accepted that this method is unreasonable there is recourse to number of hours. Most translation work is done by freelancers working alone. It is usually impossible to check the time devoted to a task, and in any case this will vary greatly depending on the “form” of a translator quite apart from their experience, ability to handle computer assisted translation (CAT) techniques, etc. Nor is everyone good at counting the time they spend on a task, the inspiration that comes while walking the dog, or is honest about it (as if they had any reason to be).

Stating a number of hours is at best a guide, a kind of check on another estimate. Often it will be a PR exercise.

Counting by quantity alone (e.g. 14,399 words, 1770 lines) for such a complex task is clearly absurd, and the translation profession can rightly be indicted for using a method better suited to simple typing. It comes from the typewriter age, of course, and probably won through because it made life easy for the intermediaries, i.e. business people rather than professionals.

There are, unsurprisingly, programs available which claim to count the words or lines better than the run-of-the-mill software (e.g. Microsoft Word), sometimes canvassing their purchasers with claims of being more accurate, as if a difference of one percent would make their users rich.

II. So what would a serious software for measuring linguistic complexity look like?

To begin with it would list all the words in the source text in the order of their frequency (e.g. 1000 x “the”) and with an indication of their frequency in the general language. In MS-DOS days I had a software which did the first half of this task.
https://www.collinsdictionary.com/dictionary/english
provides a dual classification of words into the 30,000 most used, and those used less frequently. What would be needed is a much finer categorisation, and this for the most frequently translated languages. (As I am based in Europe I restrict my considerations to this market, although much is happening particularly in Asia.)

A second quantitative analysis would categorise the sentences similarly. Legal texts can easily have intricate sentences each running to many more than fifty words. Disentangling then re-assembling any one of these involves much more work than just translating the same word quantity in several discrete sentences. So one needs information on the median length of sentences. The software could also generate lists of the outliers. The (potential) translator could then inspect these individually. (The text length varies enormously, but our concern here would be texts of ten, twenty or hundreds of pages.)

Already such a software would represent a major advance for all concerned (excepting, of course, those who profit from the present lack of transparency). It should command a respectable price.

To the shame of the laggardly translation associations and “industry”, progress has already been made in another area.

At https://storytoolz.com/readability
a text can be analysed for various metrics including
* Characters per word
* Syllables per word
* Number of sentences
* Words per sentence
* Number and percentage of short sentences
* Number and percentage of long sentences
* Number of passive sentences
* Longest sentence, shortest sentence

Of particular interest here is the identification of passive sentences. A more sophisticated software for the use of translators would include analysis of grammatical complexity as well as of terminological variety and structure. (My gratitude here to Lukas Felsberger of CERN, Geneva, who found this software for me as well as another for German.)

Once various statistics were available there would be an ongoing process of observation and experimentation to convert those statistics into consensual prices.

III. The concept and the pricing in context

The translation “industry” is changing rapidly. The last twenty years were dominated by the spread of Translation Memory systems. Initially these meant individual or associated translators creating a database of sentences they had once translated. The same or a similar sentence would be recognised on reappearance, differences highlighted (for example, “increase” instead of “decrease”), and much work saved. Used carefully, these systems also provide a quality control and enable consistency.

They cost high three-figure sums. Market leaders include SDL (formerly Trados) and MemoQ. On the basis of my experience and observations I am not personally well-disposed to these companies. Among other things, their software has also gotten too good for its own sake. As a remote comparison, think of GPS, a godsend, but it still needs a navigator with eyes, a brain and a map.

Machine translation is progressing enormously. Forget Google Translate. Try DeepLhttps://www.deepl.com/en/translator

This program draws on a mega-database of translations available on the Internet. But it also uses “neural-network” technology so as to adapt those available translations which only provide fuzzy (inexact) matches. The results are impressive, but they only work at the level of individual sentences. The sentences do not fit together (there is no “flow”) and the use of terminology is inconsistent. The translations are highly error-prone. So a whole new category of work has emerged which is called Post Editing (PE). This work is frustrating for any experienced translator (who can usually translate better and quicker than they can post-edit). With PE it is difficult to spot subtle but decisive translation errors (shifts, even reversals, of meaning, as well as omissions). In many cases it will involve more work (input) than traditional translation (probably with CAT), but it is expected to cost less. This said, PE may not be a bad way for novices to learn.

It is truly astonishing that so much know-how and research has been devoted to this sophisticated development, but none to the comparatively modest task of measuring linguistic complexity and enabling more precise pricing. My own mild overtures in the direction of fellow translators of German and German translator associations have met with little resonance, although I have not given up. It is a predominantly female profession.

As a makeshift solution I have introduced the concept of
“line value”, which involves counting (visually, “manually”) double or more the “lines” of involved sentences and terminology, while discounting repetitions and lists of names, etc. (but the discounting already happens).