Playing Around with Machine Translation

Article comparing ChatGPT vs. specialized translation products.

There’s an old, old joke about machine translation. Supposedly, in the early 1960’s, IBM unveiled a computer program that could translate between English and Russian. A general from the Pentagon asked if he could try it out. “Give me a phrase in English,” the IBM technician told him. “Out of sight, out of mind,” the general replied. The technician typed it in, and a few seconds later the computer printed out a phrase in Russian. But the general didn’t speak Russian. “Have the machine translate it back into English,” he suggested. The technician complied. A moment later, out came the result: “Invisible idiot.”

Well, all I can say is that the technology has improved a great deal. Below, to start, are four passages. The first is from the recent, excellent French translation of my book Men on Horseback . The second is a translation of that passage back into English by Google Translate. The third is a translation of the passage back into English by Chat GPT. The fourth is the original passage in English.

1 Like


This is how I test most of the new language models for Argos Translate for languages I don’t speak.

Yes the back-translation approach is a great method to test accuracy, yet doesn’t beat a linguist’s evaluation.

I’ve been thinking of launching an initiative to create a community group of native speakers that could help evaluate models. I just have to think of the right incentive model because I doubt people will just volunteer time to do this (and if they do, evaluations might not be done in a timely manner).

1 Like

This is a neat idea! LibreTranslate has been able to attract many open source contributors to translate the interface on Weblate and I think people would volunteer to do this too. I agree that the volunteer contributions probably won’t be as timely though. To get people to rate the translations quickly and reliably you probably need to pay them.

1 Like

Right, perhaps via a bounty/reward system.

1 Like