In today's globalized world, effective communication across language barriers has never been more important. Translation can help businesses expand into new markets and communicate more effectively with non-English speaking customers, but quality is key. Poor quality translation can have the opposite effect, causing confusion, misinterpretations, and costly errors.
There are a number of different metrics and tools for assessing translation quality and ensuring that your translated content has the intended effect on your target audience. Here’s what you need to know.
What is translation quality?
Translation quality refers to the degree to which a translated text effectively achieves its intended objectives. It takes into account fidelity to the source material but also the extent to which meaning is accurately conveyed. A high-quality translation maintains the essence of the original text while making it understandable and coherent for the target audience. Depending on the industry, compliance with regulations may also come into play.
Translation quality should not be confused with a mere word-for-word translation. Because there is so much variation in terms of cultural references and expressions, it’s sometimes necessary to depart from the original text to make sure that the ideas come through clearly. Quality translation captures the overall meaning and context of the source material while keeping these cultural expressions and nuances in mind to effectively communicate the intended message.
Measuring translation quality
There are a number of frameworks to evaluate translation quality. Some are specialized for certain industries, like the SAE J2450 metric for the automotive and manufacturing industries. This standard measures characteristics like correct spelling and terminology but does not consider writing style.
Others are more general. The Multidimensional Quality Metrics-Dynamic Quality Framework (MQM-DQF) framework can be used to analyze both human translation and machine translation and is integrated in the most popular computer assisted translation (CAT) tools.
Important objective metrics for translation quality include:
- Accuracy: Is the meaning of the original text communicated clearly?
- Fluency: Are standard spelling, punctuation and grammar used in the translation?
- Terminology: Are keywords and phrases translated accurately and consistently?
- Style: Is the translation easy to read and understand, and does it match the style of the original?
- Design/formatting: Is everything formatted correctly in the translation?
Errors and issues can be ranked as critical, major, minor or neutral, depending on their impact on the overall message.
There are also special metrics for machine translation (MT) quality. The most common one that you’ll see is the BiLingual Evaluation Understudy (BLEU) score. BLEU scores are calculated automatically by comparing a piece of machine-translated content with a human-translated version of the same text.
These scores are useful for comparing different MT systems and for evaluating the quality of these systems as a whole. Evaluating a specific translated document still requires a human touch.
Subjective evaluations from human linguists may also be used to catch issues that are not easily quantified with the metrics listed above. For example, in-country reviews by local reviewers can be used to make sure that the content will resonate with consumers in the target market.
Translation quality standards
Frameworks are systems for evaluating the quality of translation, standards are the processes a language services provider (LSP) follows to ensure consistently high-quality translations. Compliance with these standards is evaluated and certified by a third party.
Here are some common standards that LSPs can be certified to:
- ISO 17100: provides requirements for the core processes, resources, and other aspects necessary for the delivery of a quality translation service that meets applicable specifications.
- ISO 9000 family of standards: addresses quality management and quality assurance
- ASTM F2575-14 Standard Practice for Language Translation: provides a framework for agreement on specifications for translation projects that will meet the needs and expectations of the end user.
Our ULG team is aligned with ISO 9001:2015 and ISO 17100:2015 to ensure quality.
Methods and tools for measuring translation quality
Organizations use a variety of methods and tools to measure translation quality. These can include metrics that don’t directly refer to translation, including:
- Customer feedback, such as reviews, customer satisfaction (CSAT) scores, and Net Promoter Scores. These scores don’t specifically assess translation quality but may reveal translation issues.
- Internal feedback from employees who speak the target language.
- Engagement and performance analytics can reveal translation quality issues in websites.
One way to evaluate translation quality more directly is to hire a third-party language provider to provide independent assessments.
Automatic quality assurance tools can be incorporated into translation management systems and workflows. These tools work like a spelling and grammar check but for translated content. They automatically analyze the text and flag issues like repeated words, mismatches between words used in the translation and relevant terms from a glossary, capitalization errors, and other issues.
Automated quality estimation tools can also be used to evaluate machine translation output without requiring a human-translated reference text. These tools use machine learning (ML) to estimate machine translation quality.
While automated tools can speed the process up, human reviewers remain essential. Language is complex, and human reviewers are best equipped to catch the nuances that machines miss.
AI: A new revolution in translation quality
In the near future, artificial intelligence (AI) Large Language Models (LLMs) such as ChatGPT may be able to assist in quality assurance in translation.
For example, a research paper from Microsoft released in May 2023 describes a new metric to assess translation quality: GEMBA, which stands for GPT Estimation Metric Based Assessment. According to the researchers, GPTs demonstrate “state-of-the-art capabilities” for assessing translation quality under certain conditions.
However, for the models to perform accurately, they need clear instructions. Also, the models have only been shown to perform well with high-resource languages (languages that already have a lot of translated content available to train on.) For under-resourced languages, the results may not be as accurate.
Translation industry news site Slator’s 2023 Language Industry Market Report highlights the potential advantages of using LLMs in quality assurance:
- Translation error identification and annotation — previously a manual, time-intensive task performed by expert linguists — can now potentially be performed at scale;
- Translation error detection can be equally applied to human translation as to MT
- Once detected, errors can be corrected by humans, or by the LLM itself.
According to language services industry expert Stefan Huyghe, the future of measuring translation quality is a collaboration between human linguists and LLMs. He explains, “While humans and LLMs each have their strengths and weaknesses in this area, the study uncovers how their collaboration could revolutionize translation assessment. Humans excel at grasping context, idiomatic expressions, and cultural nuances, while LLMs offer lightning-fast processing and consistent evaluations, devoid of human biases or fatigue.”
Benefits of measuring translation quality
Measuring translation quality offers benefits that go beyond spending your translation budget wisely. Quality assurance for translated content safeguards your brand reputation and maintains consistency in both your brand voice and the overall customer experience.
Accurate, high-quality translations help you avoid potential brand damage that may arise from translation errors. These errors can negatively impact even large, well-established brands, as the cautionary tale of HSBC Bank demonstrates.
HSBC’s slogan “Assume nothing” was mistakenly translated as “Do nothing” in several countries. Customers were understandably wary of putting their money in a bank that advertised doing “nothing” with their investments.
Regaining that trust came with a cost: a global rebranding campaign that cost $10 million.
Errors can also increase liability and cause harm to customers. A translation error in Germany caused 47 failed knee replacement surgeries, all because the phrase “non-modular cemented” was mistranslated as “without cement,” which caused surgeons to install them incorrectly.
Measuring translation quality gives you a benchmark for continuous improvement, so you can confidently communicate with your global audience while mitigating these potential risks.
Challenges of measuring translation quality (and how to overcome them)
Measuring translation quality presents a variety of challenges, but they can be effectively addressed with the right strategies. One of the main challenges lies in the fact that there can be multiple ways to say the same thing in any language. The solution is to employ style guides and glossaries as a roadmap, ensuring consistency throughout the translation process.
Another challenge arises when requirements are not clearly defined from the outset, which can result in even highly skilled linguists producing translations that are technically correct but still fail to meet client expectations. This can be addressed by carefully defining the requirements at the beginning of the project.
It can also be challenging to navigate the many different translation metrics and measurement tools to ensure you’re collecting relevant data. Your LSP can help with this.
Additionally, in-country review can become a bottleneck in some organizations, especially when the reviewers haven’t been told what to look for ahead of time. Clearly defining project goals and establishing good communication between linguists and reviewers can help to avoid this problem and ensure successful in-country reviews.
A final challenge is time, especially as MT allows for more and more content to be translated. The best way to effectively scale the QA process to keep up with the pace of MT is to invest in more technology and automation. New tools are coming out all the time.
Tips to ensure high-quality translations
Beyond measuring translation quality, there are some actions that organizations can take to ensure high-quality translations every time.
First, understand that high-quality translations start with the source content. Optimize it whenever possible to make it easier to translate and reduce the chance of errors slipping in. Stick to short, clear sentences, avoid jargon, slang, and humor, and avoid references that are culturally specific.
Develop a glossary and a style guide ensures your brand voice remains consistent across all markets. Your LSP can help you develop and deploy these assets.
Speaking of LSPs, choosing the right partner is key. In fact, it’s probably the single most important action you can take to ensure your translations are of the highest quality. Look for a provider with a proven track record and documented quality control processes.
Make sure that the linguists your LSP works with are subject matter experts with experience translating for your industry. This is especially important for specialized fields like medical and legal translation.
Our experts are ready to help you with all of this and more. Contact us today for a free consultation to see how we can assist you with high-quality translations, subject matter expertise and efficient workflows.
