Good morning,
Is it possible to use the GPU & NPU on the Arm64 platform (ie orange pi 5 pluses) for acceleration like it is with Nvidia GPU’s on the x86_64 platform for Argos & Libretranslate?
Are you trying to run inference on the models to use LibreTranslate or train new models? If you’re running inference look at CTranslate2 to see what hardware they support.
tbh I have no idea how this all works. It looks like I can use rknn-toolkit2 to deploy the model. But ctranslate2 is specific about just supporting CUDA, so not sure if that would work.
But the orange pi 5 plus has a 6 tops NPU and it would be nice to use it for accelerating libretranslate.