Add option for fallback model when stanza doesn't exist

LibreTranslate API
,
1

@christopherpickering christopherpickering

The stanza models don’t exist for all languages, and in this case Locomotive failes with a Cannot download stanza model … It would be nice to be able to specify a fallback local (currently I hardcode it) to a similar language.

See train.py:

try:
            os.makedirs(stanza_dir, exist_ok=True)
            stanza.download(stanza_lang_code, dir=stanza_dir, processors="tokenize")
            break
2

There’s a mechanism in train.py for a fallback mechanism (it’s hard-coded), one simply needs to add to the mapping variable source lang => stanza lang to use instead, perhaps it could be useful to make it a command line parameter instead.

2 Likes