Why is Machine Translation quality relative?

Every technology has its proponents and detractors, and machine translation is no exception. 

In our industry, I have come across countless old school translators who swear that machine translation is useless, that it will never be able to produce anything close to what they can do, and that translation companies should increase translator salaries rather than investing in machine translation tools and research.

Conversely, I have met a lot of computer scientists convinced that it is a matter of time before machine translation is good enough to eliminate all human translation besides a few very special cases such as advertising.

As both a software engineer and a translator, I can understand both sides of the argument and the underlying emotion. Because I stand somewhere in between, I have even authored a paper on the subject and given a speech at the AMTA conference a few years ago.

So who is right? The mad scientists or the entrenched translators?

What if they are both right, and sometimes even simultaneously right.

Let me explain.

You are browsing the Internet looking for some particular type of product. You click on a link on an English website page and suddenly you are on the Japanese page of a company manufacturing exactly the type of product you are looking for. 

The only trouble is that the page is all in Japanese and you only understand the pictures! As you are using Google Chrome, a button offers to automatically translate the page in English. You click on it, and the page is magically turned into English, thanks to Google Translate operating in the background. 

It’s a strange English though, and some sentences don’t make sense, but overall, you get it. You understand the product specifications, most of the company history and a few more details, but you find the style awkward and maybe one paragraph remains close to gibberish. 

Overall, you are pretty satisfied and you mentally award an 8/10 grade both to the products and to the translation.

Now, imagine that the Japanese company has decided to translate their website into English, but used Google Translate to do so, instead of working with professional translators (such as us!).

This time, you are sent to the English page of the Japanese website. Interestingly, because they have used Google Translate, what you see is exactly the same as what we described earlier.

However, this time, you notice the bad quality of the English, sentences that just don’t make any sense, and you wonder whether their products are as bad as their website. Overall, you give it a 6/10 for the products and a 2/10 for the translation, and you immediately close the webpage.

What did we learn from this story, is machine translation quality good or bad? Personally, I’d say it’s a matter of expectation and context!