What's the future of the argos-translate dependency in LibreTranslate

Argos Translate
1

I’m trying to update the LibreTranslate package in NixOS and see that this package now depends on a fork of argos-translate (argos-translate-lt). Is this a long-term fork or a temporary fork? I’d rather not keep multiple versions around.

2

It’s temporary. The two are virtually the same. There’s just a fix for Do not set compute type by pierotofy · Pull Request #511 · argosopentech/argos-translate · GitHub merged and I’ve removed the dependency on torch (and stanza) in the fork, in order to make LibreTranslate lighter to download (and fix installation issues that plague torch).

1 Like
3

Thanks! In that case, I’ll keep the dependency on the upstream argostranslate (for the moment).

4

I merged this change.

I’d like to better support installing Argos Translate without PyTorch but I don’t know a great way to do this for a Python package. I don’t think there’s a good mechanism in Pip for optional dependencies. Having a forked version of Argos Translate for LibreTranslate seems like a good solution for now. I might just fully remove Stanza (which depends on PyTorch) at some point which would make this unnecessary.

1 Like
5

I’m hardly an expert on this, but it looks like you can mark a dependency as optional by appending [some_feature_name] after the dependency. E.g.:

ctranslate2>=4.0,<5
sentencepiece>=0.2.0,<0.3
spacy [spacy]
stanza [stanza] == 1.10.1
packaging
sacremoses>=0.0.53,<0.2
minisbd

Then you’d run pip install argos-translate[stanza] to install it with stanza support.

2 Likes
6

Oh good tip thanks. That feature would work perfectly for this usecase.

I made a GitHub ticket to look into this when I get a chance

1 Like
7

Hi, pytorch comes from the stanza dependency.
Some of the leads we’ve been working on for the SBD :

  1. spacy and pysbd are ok for many languages
  2. for many languages (Armenian, Amharic, Chinese, Hindi, Japanese, Khmer, Lao, Thai etc…) you need a special library. That’s where stanza kicks in.
  3. @pierotofy has developed an ONNX equivalent to stanza: MiniSBD (there’s a post about it in this community forum)

I think I’ll come back to this matter in the next months: I have to fix some SBD issues on our corporate platform.

2 Likes
8

I implemented Stanza as an optional dependency in Argos Translate!

I want to test this a bit more before I release it on PyPI but it seems to work well. The default install is so much faster without having to install Stanza and PyTorch.

1 Like
9

This is great news; anything connected to pytorch always requires a huge download and this will make it easier to integrate argos-translate in downstream applications. :partying_face:

1 Like