Twenty years ago, measured translation quality was a budding topic in linguistic services. Now, as we navigate through the era of Large Language Models (LLMs), it’s time to forecast the future of measured translation quality. While working at ForeignExchange Translations, I co-authored a book on measured translation quality with Jason Arnsparger. The core theme centered around proactively managing and measuring your quality, ultimately reducing time-to-market and cost. Our clients assumed the need for quality but had begun to focus heavily on cost and time. This paradigm has only become more acute in the last 20 years.
With the emergence of LLMs, our traditional translation processes are being put in question. The advent of LLMs challenges traditional translation processes, raising pivotal questions about the future role of human translators and the nature of content creation. There are more questions than answers; more theories than fact. Everyone has a theory on what this means for the future of the industry and of translators. Will translation go away? What will the role of a translator be? Will we continue to create and then localize source content, or will there be a shift towards multilingual content creation?
The Enduring Necessity of Human Oversight
Despite the huge potential of LLMs, they are not without quality risks. We’ve all seen plenty of high profile cases in the news that highlight the hallucinations of these chatbots, cases where AI systems generate inaccurate or even made up content. One example is the Levidow case, where attorneys relied on a chatbot, which generated non-existent cases such as Martinez v. Delta Air Lines.
Vectara recently published statistics on AI hallucinations. Hallucination ranged from 3.0% for GPT 4.0 to 27.2% from Google’s Palm 2 Chat based on Nov. 1, 2023 results.
Image credit Vectara / GitHub, https://github.com/vectara/hallucination-leaderboard
Despite the vast quantity of uncontrolled training data, the instances of hallucinations, errors, and bias should come down over time. I believe most would agree that they will not be eliminated. As a result, humans will remain a key component of producing high quality translations for many types of content. Whether we are using LLMs to translate content, to automate post-editing, to automate quality control steps, or even to create multilingual content, the working theory is that there will be a need for humans in quality control.
What is the future of measured translation quality?
I asked ChatGPT what the future of Measured Translation Quality is. I found the response quite intriguing. Many of the points GPT 4.0 presents are spot on, such as: “Ethical and Bias Mitigation: Future technologies will focus on reducing biases to ensure fair and inclusive translations.” I find it interesting that there is no mention of the importance of terminology management, whether that be in proactively generating high quality AI-translated content or in the measurement of quality levels.
The journey of measured translation quality is far from over. I will be continuing my deep dive into the future of measured translation quality over the coming months. I’d be interested in hearing your thoughts on the evolving landscape of measured translation quality.
Sign-up to be notified when new content is released
Sonia brings 30+ years of leadership experience in the localization industry. Having worked with a diverse range of clients and industries, Sonia brings a wealth of knowledge and a proven track record of success in delivering exceptional localization solutions.