The Multidimensional Quality Metrics, is a translation quality assessment framework enables users to customize their own metrics for quality evaluation. It was developed to create shared quality metrics system for both human and machine translation and to improve automatic translation quality estimation
Terminology – errors arising when a term does not conform to normative domain or organizational terminology standards or when a term in the target text is not the correct, normative equivalent of the corresponding term in the source text
Accuracy – errors occurring when the target text does not accurately correspond to the propositional content of the source text, introduced by distorting, omitting, or adding to the message
Linguistic conventions – errors related to the linguistic well-formedness of the text, including problems with grammaticality, spelling, punctuation, and mechanical correctness
Style – errors occurring in a text that are grammatically acceptable but are inappropriate because they deviate from organizational style guides or exhibit inappropriate language style
Locale conventions – errors occurring when the translation product violates locale-specific content or formatting requirements for data elements
Audience appropriateness – errors arising from the use of content in the translation product that is invalid or inappropriate for the target locale or target audience
Design and markup – errors related to the physical design or presentation of a translation product, including character, paragraph, and UI element formatting and markup, integration of text with graphical elements, and overall page or window layout
Client requirements - errors related to Client specific requirements and criteria
How can such a framework be used by a company evaluating written output?
Everybody knows that total quality is expensive, and that machines cannot attain it, irrespectively of their capabilities. So, based on the specific text type, such a framework allows you to understand which criteria and requirements to prioritise over the other and where to focus more resources to attain what gives you best returns in the specific circumstance.
Product translation, UIs definition, Marketing materials or Legal documents could be optimised by balancing the most important criteria for each textual asset against those that are less important.
To make it less theoretical: let's say you have a marketing text that you want to optimise for another country. So not only translation, but culturalisation is also very important. Audience appropriateness, is the most important criteria in this case, followed by local conventions, specific brand speak (terminology), Specific tone (attained through a calibrated style) and so on.
On the other hand, an engineering product manual will need to be absolutely accurate, spot on on terminology, avoid synonyms, adhere to an austere and precise style, with correct rendering of diagrams, industrial designs and tables.
I guess, now it is time to play: The following decision tree is a guide for how to determine under which error type dimension to put the error
* As published in the MQM website
And to obtain a graphical representation by textual asset of where to put your resources, you could use a radar graph, like the one below.
Pretty soon you'll notice that different textual assets have very different geometrical forms. And that cost is a function of the criteria you consider, and the degree of their automation.
Ready to try it out for your company?