Performance, MiniSBD and GPU

Argos Translate
1

I am using LibreTranslate on a 12-core Epyc VM with Nvidia A4000 GPU. I tested the translation from German to English by pasting the wiki source of the German language Wikipedia article on the Riemannsche Vermutung (https://de.wikipedia.org/w/index.php?title=Riemannsche_Vermutung&action=edit) into the web UI. With ARGUS_DEVICE_TYPE=cpu it takes 4.3 minutes while with ARGUS_DEVICE_TYPE=cuda it takes about 2.5 minutes.

There’s still significant CPU usage, and I cannot confirm if MiniSBD uses the GPU. I installed onnxruntime-gpu but it would be nice to know if it really uses the GPU. Of course, I set ARGOS_CHUNK_TYPE=MINISBD.

The article is 2864 sentences with 31850 words or 240147 characters. That’s about 20 sentences per second. Is this perfomance to be expected? How much time is MiniSBD to take?

2

MiniSBD will use the GPU, if it’s available and ARGOS_DEVICE_TYPE is set to cuda and onnx-runtime[gpu] is installed and drivers are configured. Looks like you have a typo. “ARGUS_DEVICE_TYPE”.

1 Like
3

Thanks. I had it right in my systemd service file but typed this without looking at it. :slight_smile:

And thanks for your response.