GPU support on ARM

Good morning,

Is it possible to use the GPU & NPU on the Arm64 platform (ie orange pi 5 pluses) for acceleration like it is with Nvidia GPU’s on the x86_64 platform for Argos & Libretranslate?

Are you trying to run inference on the models to use LibreTranslate or train new models? If you’re running inference look at CTranslate2 to see what hardware they support.

tbh I have no idea how this all works. It looks like I can use rknn-toolkit2 to deploy the model. But ctranslate2 is specific about just supporting CUDA, so not sure if that would work.

But the orange pi 5 plus has a 6 tops NPU and it would be nice to use it for accelerating libretranslate.