Language model training for argos-translate/LT: Locomotive

General
6

I was able to run Locomotive on a Vast.ai Linux server with a RTX4090 GPU and the demo mostly worked out of the box.

python3 train.py --config model-config.json

I did get this error from CTranslate2 failing to use CUDA but I fixed it by setting the CTranslate2 device to “cpu”:

[email protected]:~/Locomotive$ python3 eval.py --config model-config.json 
Starting interactive mode
(en)> Hello this is a test
Traceback (most recent call last):
  File "/root/Locomotive/eval.py", line 239, in <module>
    translation_obj = data["model"].translate_batch(
RuntimeError: Library libcublas.so.11 is not found or cannot be loaded

I also decreased the number of train steps to make the training run shorter:

# train.py
-    'valid_steps': 5000, 
-    'train_steps': 50000, 
+    'valid_steps': 1000, 
+    'train_steps': 5000,

Despite the short training time and solely using the TildeMODEL dataset the model seems to generate decent quality translations.

Interactive mode

[email protected]:~/Locomotive$ python3 eval.py --config model-config.json 
Starting interactive mode
(en)> Hope this can be useful to others! Some quirks might still be present, I haven’t tested on Linux/macOS, but that testing/adding support for those platform is next. I’m also looking at the possibility of having an easy “install.py” command to add the model to the local argos-package directory so that it can be used in argos-translate/LT. :tada:
(it)> E' possibile che ciò possa essere utile per altri!, che potrebbero ancora essere presenti, hontoto sulla Linux/macOS, ma che le prove/astendono il sostegno a questa piattaforma sono prossime. I'm cerca anche di avere un semplice "impianto di comando" per aggiungere il modello al repertorio locale argos-pacchetto, affinché possa essere usato in unrgos-trans/LTda:
(en)> ^C

BLEU

[email protected]:~/Locomotive$ python3 eval.py --config model-config.json --bleu
Downloading flores200 dataset...
Tokenizer 'spm' has been changed to 'flores101', and may be removed in the future.
BLEU score: 52.79772
1 Like