Documentation on LibreTranslate Environmentals

LibreTranslate API
1

Is there any good documentation on LibreTranslate environmental variables, for example: LT_THREADS? The default is 4.

I’m trying to host on AWS, with q set to an array. So, how big can the array be? What about LT_THREADS? Are they connected? Is LT_THREADS set per core, or is it the total number of threads regardless of the number of cores?

With q set to 6 strings and LT_THREADS set to 32, I need 8 AWS EC2 vCPUs to stop it falling over. Any help would be very much appreciated.

Thanks

2

I think LT_THREADS just configures the number of threads LibreTranslate uses. CTranslate2 also creates its own threads and processes.

3

So, is that a max of LT_THREADS concurrent connections for LibreTranslate? I’m coming from a node.js background, so not sure what LibreTranslate LT_THREADS means?

4

That value is passed to waitress:

1 Like
5

Thanks a million, that’s super info. I’m not a python head, but I’ll get there!

6

Some users have complained that LibreTranslate uses too much RAM. Should decreasing LT_THREADS decrease memory usage?

There are also outstanding memory leaks[1][2] we haven’t been able to find that could be causing problems too.

7

I haven’t done detailed profiling, but I would guess that LT_THREADS probably will not affect memory usage much.

Decreasing the size (number of parameters) of the ctranslate models will have the largest impact, but obviously that affects quality.

I’m not sure how much memory stanza uses, but it might be interesting to make a comparison with spacy in that regard.

1 Like