Machine Learning in Linux: Argos Translate is an Offline Translation Library

Our Machine Learning in Linux series focuses on apps that make it easy to experiment with machine learning.

Machine Learning extends in many directions. We’ve not covered neural networks in this series, so let’s rectify that. A neural network is a method in artificial intelligence that teaches computers to process data in a way that’s inspired by the human brain. It is a type of machine learning process, called deep learning, that uses interconnected nodes or neurons in a layered structure that resembles the human brain.

Google Translate is a popular multilingual neural machine translation service that translates text and websites from one language into another. How about a self-hosted open source alternative?

Argos Translate is state of the art neural machine translation software. It’s written in Python and published under an open source license. Argos Translate uses OpenNMT for translations and can be used as either a Python library, command-line, or GUI application.

Modern machine translation goes beyond simple word-to-word translation to communicate the full meaning of the original language text in the target language. It analyzes all text elements and recognizes how the words influence one another.

Installation

As we’ve explained in previous articles in this series, we don’t recommend using pip to install software unless it’s within a virtual environment. A good solution is to use a conda environment as it helps manage dependencies, isolate projects, and it’s language agnostic.

1 Like

If your machine has a dedicated graphics card, you’ll want to enable GPU support. This is enabled by setting the ARGOS_DEVICE_TYPE env variable to cuda or auto.

I actually think this is bad advice, at least for now. I recommend running Argos Translate on a CPU. For whatever reason CTranslate2 doesn’t seem to really run faster on a GPU than a CPU. Libretranslate’s performance on high end CPUs is very good though.

In my tests the GPU also did not show much increase in performance (for single translations).

I think there’s a good use case for using the GPU to speed-up batch inference via ctranslate (where it’s actually faster), but it’s hard to scale it right, as large batch sizes quickly fill up the GPU memory.

1 Like

I just ran some benchmarks.
We do ~200K documents/day with Argostranslate on all supported languages, running pretty large batches.
For an identical batch I’m averaging ~5.8 documents/second with H100, on CPU (2x Intel(R) Xeon(R) Gold 6248R CPU) I’m getting ~2.7 documents/second.

Hope this helps

1 Like

Awesome thanks for sharing!

For an identical batch I’m averaging ~5.8 documents/second with H100, on CPU (2x Intel(R) Xeon(R) Gold 6248R CPU) I’m getting ~2.7 documents/second.

I looked up prices on Ebay and by my math the CPU is cheaper per the throughput performance but they’re within an order of magnitude:

$4000 - 2x Intel(R) Xeon(R) Gold 6248R CPU
$30000 - NVIDIA H100 GPU

1481 $/(doc/s) for the CPU
5172 $/(doc/s) for the GPU

We do ~200K documents/day with Argostranslate on all supported languages, running pretty large batches.

:fire::fire::fire:

Wow, that’s a lot of docs.

1 Like