Critical: GPU recognition failing in nvidia-tensor-based branch despite providing all CUDA 11.0/11.8 libraries

[vgazioglu]

Issue Body:

Describe the bug The GPU classification mode in the nvidia-tensor-based branch fails to initialize. Even after manually extracting and providing the complete set of required CUDA 11.x and cuDNN 8 libraries (totaling ~1.6GB) and correctly setting the LD_LIBRARY_PATH, TensorFlow in the nextcloud:stable image cannot link these libraries.

Environment:

Base Image: nextcloud:stable (Debian Bullseye/Bookworm)

Builder Image for libraries: nvcr.io/nvidia/tensorflow:22.12-tf2-py3 (Contains CUDA 11.8)

Recognize Version: Latest (using TensorFlow.js / tfjs-node-gpu)

GPU: NVIDIA (Verified via nvidia-smi on host and container)

Steps Taken:

Extracted the following libraries from the NVIDIA builder image into lib_gpu:

    libcudart.so.11.0, libcublas.so.11, libcublasLt.so.11, libcufft.so.10, libcurand.so.10, libcusolver.so.11, libcusparse.so.11, libcudnn.so.8.

Built the custom image using the provided Dockerfile logic, copying these into /usr/local/cuda/lib64/.

Set LD_LIBRARY_PATH=/usr/local/cuda/lib64 and ran ldconfig.

Attempted to run php occ recognize:classify.

Error Logs: Despite the files being present and the paths being set, TensorFlow repeatedly logs:
Plaintext

W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublas.so.11'; dlerror: libcublas.so.11: cannot open shared object file: No such file or directory
... (repeated for all listed .so files)
W tensorflow/core/common_runtime/gpu/gpu_device.cc:1850] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU.

Observations:

The current build-all.sh in the repo is broken as it pulls CUDA 12 images which are incompatible with Recognize's requirement for CUDA 11.

The strip command in the original script corrupts the NVIDIA binaries (file format not recognized).

Even with clean, unstripped CUDA 11.8 libraries, the Debian-based Nextcloud container seems unable to link the Ubuntu-sourced NVIDIA libraries correctly.

Request: Could you please provide a working Dockerfile or a verified build-all.sh that addresses these linking issues? It seems there is a mismatch between the environment the libraries are pulled from and the final Nextcloud environment.

[bugsyb]

What is the command used to create the image?
Please provide:

• output of docker image ls|grep -e jitsi -e debian -e nvidia -e nextcloud,
• command used to create the image,
• output of image creation steps.

I've just created on Ubuntu (which doesn't matter at all, since we create container image) Nextcloud 32, earlier Nextcloud 31 and it worked. It could potentially be that maybe earlier stage of creation I have files as were created earlier (the underlying "FROM" images) and building it today, just fails - hence interested to see the odds on your system.

[vgazioglu]

What is the command used to create the image? Please provide:

* output of `docker image ls|grep -e jitsi -e debian -e nvidia -e nextcloud`,

* command used to create the image,

* output of image creation steps.

I've just created on Ubuntu (which doesn't matter at all, since we create container image) Nextcloud 32, earlier Nextcloud 31 and it worked. It could potentially be that maybe earlier stage of creation I have files as were created earlier (the underlying "FROM" images) and building it today, just fails - hence interested to see the odds on your system.

./build_all.sh -g -n 32

[bugsyb]

Without the rest of requested details, it will be hard to troubleshoot and understand why it doesn't work at your end and it works at my end.

[vgazioglu]

Without the rest of requested details, it will be hard to troubleshoot and understand why it doesn't work at your end and it works at my end.

I respect. The issue is I've spent a whole day with build_all.sh -g -n32 executing hundres of commands and getting corresponding errors (mostly missing libraries, version mismatch etc) then deleting all :) next whole day tried to discover America again but that time trying to act as recognition installer by myself and installing things. With the help of Gemini, I succeed but the next day we together :) couldn't reproduce it again. Today i tired with -n 31 but another x doesnİ't like Y version, y doesn't like z version stories...

below is my docker-compose at the end and what kind of errors i got is the subject of a book :)

services:
app:
# Build ettiğin en güncel ve kapsamlı imajı kullanıyoruz
image: local/nextcloud:latest-gpu
container_name: nextcloud-app
restart: unless-stopped
environment:
- MYSQL_HOST=db
- MYSQL_DATABASE=nextcloud
- MYSQL_USER=nextcloud
- MYSQL_PASSWORD=example
# NVIDIA sürücülerinin PHP ve Node tarafından görülmesi için kritik:
- NVIDIA_VISIBLE_DEVICES=all
- NVIDIA_DRIVER_CAPABILITIES=all
- LD_LIBRARY_PATH=/usr/local/nvidia/lib64:/usr/lib/x86_64-linux-gnu
volumes:
- nextcloud_data:/var/www/html
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu, compute, utility]
depends_on:
- db

db:
image: mariadb:10.11
container_name: nextcloud_db
restart: unless-stopped
environment:
- MYSQL_ROOT_PASSWORD=example
- MYSQL_DATABASE=nextcloud
- MYSQL_USER=nextcloud
- MYSQL_PASSWORD=example
volumes:
- db_data:/var/lib/mysql

web:
image: nginx:alpine
container_name: nextcloud-web
restart: unless-stopped
ports:
- "8080:80"
volumes:
- nextcloud_data:/var/www/html:ro
- ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
depends_on:
- app

volumes:
nextcloud_data:
external: true
db_data:
external: true

BR

[bugsyb]

Does this mean that, the built finished successfully and you've node installed in the image?

www-data@gpunode1-app:~/html/custom_apps/recognize$ bin/node ./src/test_libtensorflow.js 
2026-01-14 00:01:25.778056: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2026-01-14 00:01:25.794625: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

Can't run it on the system with GPU at the moment, hence first things first that image is built properly?

Below is from fresh Ubuntu (25.04 - doesn't matter) installation and image built:

root@recognize:~/stuff/docker# docker image ls
                                                                                                                                                                                                                         i Info →   U  In Use
IMAGE                                                      ID             DISK USAGE   CONTENT SIZE   EXTRA
debian:bookworm-slim                                       8242eaf53627       25.3GB         8.76GB        
local/nextcloud:32-fpm-tensor                              eb213c2b4826       2.08GB          523MB        
local/nextcloud:32-fpm-tensor-custom                       06121b3982a8       5.39GB         1.88GB    U   
local/nextcloud:32.0.3-fpm-tensor                          eb213c2b4826       2.08GB          523MB        
local/nextcloud:32.0.3-fpm-tensor-custom                   06121b3982a8       5.39GB         1.88GB    U   
local/nextcloud:32.0.3-fpm-tensor-custom-20260113-220101   06121b3982a8       5.39GB         1.88GB    U   
local/nextcloud:latest                                     06121b3982a8       5.39GB         1.88GB    U   
local/nextcloud:latest-gpu                                 06121b3982a8       5.39GB         1.88GB    U   
local/php:8.2-fpm-bookworm                                 ba7eee9c1e8f       29.6GB         10.1GB        
nc-web:latest                                              1946ca0ed36d       92.2MB         25.9MB    U   
nvcr.io/nvidia/tensorflow:24.10-tf2-py3                    8242eaf53627       25.3GB         8.76GB        
php:8.2-fpm-bookworm                                       ba7eee9c1e8f       29.6GB         10.1GB        

Add also output of

• bin/node ./src/test_gputensorflow.js run from inside the recognize folder
• node -v output
• nvidia-smi output dump

[vgazioglu]

Does this mean that, the built finished successfully and you've node installed in the image?

www-data@gpunode1-app:~/html/custom_apps/recognize$ bin/node ./src/test_libtensorflow.js 
2026-01-14 00:01:25.778056: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2026-01-14 00:01:25.794625: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

Can't run it on the system with GPU at the moment, hence first things first that image is built properly?

Below is from fresh Ubuntu (25.04 - doesn't matter) installation and image built:

root@recognize:~/stuff/docker# docker image ls
                                                                                                                                                                                                                         i Info →   U  In Use
IMAGE                                                      ID             DISK USAGE   CONTENT SIZE   EXTRA
debian:bookworm-slim                                       8242eaf53627       25.3GB         8.76GB        
local/nextcloud:32-fpm-tensor                              eb213c2b4826       2.08GB          523MB        
local/nextcloud:32-fpm-tensor-custom                       06121b3982a8       5.39GB         1.88GB    U   
local/nextcloud:32.0.3-fpm-tensor                          eb213c2b4826       2.08GB          523MB        
local/nextcloud:32.0.3-fpm-tensor-custom                   06121b3982a8       5.39GB         1.88GB    U   
local/nextcloud:32.0.3-fpm-tensor-custom-20260113-220101   06121b3982a8       5.39GB         1.88GB    U   
local/nextcloud:latest                                     06121b3982a8       5.39GB         1.88GB    U   
local/nextcloud:latest-gpu                                 06121b3982a8       5.39GB         1.88GB    U   
local/php:8.2-fpm-bookworm                                 ba7eee9c1e8f       29.6GB         10.1GB        
nc-web:latest                                              1946ca0ed36d       92.2MB         25.9MB    U   
nvcr.io/nvidia/tensorflow:24.10-tf2-py3                    8242eaf53627       25.3GB         8.76GB        
php:8.2-fpm-bookworm                                       ba7eee9c1e8f       29.6GB         10.1GB        

Add also output of

* `bin/node ./src/test_gputensorflow.js` run from inside the recognize folder

* `node -v` output

* `nvidia-smi` output dump
* 

www-data@cc6a00643b53:/html/custom_apps$ cd recognize/
www-data@cc6a00643b53:/html/custom_apps/recognize$ bin/node ./src/test_libtensorflow.js
bash: bin/node: No such file or directory
www-data@cc6a00643b53:/html/custom_apps/recognize$ bin/node ./src/test_libtensorflow.js
bash: bin/node: No such file or directory
www-data@cc6a00643b53:/html/custom_apps/recognize$ node -v
v20.19.2
www-data@cc6a00643b53:~/html/custom_apps/recognize$ node ./src/test_libtensorflow.js
node:internal/modules/cjs/loader:1215
throw err;
^

Error: Cannot find module '@tensorflow/tfjs-node'
Require stack:

• /var/www/html/custom_apps/recognize/src/test_libtensorflow.js
  at Module._resolveFilename (node:internal/modules/cjs/loader:1212:15)
  at Module._load (node:internal/modules/cjs/loader:1043:27)
  at Module.require (node:internal/modules/cjs/loader:1298:19)
  at require (node:internal/modules/helpers:182:18)
  at Object. (/var/www/html/custom_apps/recognize/src/test_libtensorflow.js:1:12)
  at Module._compile (node:internal/modules/cjs/loader:1529:14)
  at Module._extensions..js (node:internal/modules/cjs/loader:1613:10)
  at Module.load (node:internal/modules/cjs/loader:1275:32)
  at Module._load (node:internal/modules/cjs/loader:1096:12)
  at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:164:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/var/www/html/custom_apps/recognize/src/test_libtensorflow.js' ]
  }

Node.js v20.19.2
www-data@cc6a00643b53:/html/custom_apps/recognize$ bin/node ./src/test_gputensorflow.js
bash: bin/node: No such file or directory
www-data@cc6a00643b53:/html/custom_apps/recognize$ nvidia-smi
Wed Jan 14 07:27:34 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.105.08 Driver Version: 580.105.08 CUDA Version: 13.0 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4060 ... On | 00000000:01:00.0 On | N/A |
| N/A 39C P8 4W / 60W | 64MiB / 8188MiB | 35% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+

ps1: I installed node bymyself as root@cc6a00643b53:/var/www/html# apt install -y nodejs npm
ps2: host info: volkan1@herakleia:~/Projects/Recognition-migration/GPU_victory$ nvidia-smi --query-gpu=driver_version --format=csv,noheader && nvcc --version | grep release && cat /usr/include/cudnn_version.h | grep -E 'CUDNN_MAJOR|CUDNN_MINOR|CUDNN_PATCHLEVEL' -A 2
580.105.08
Cuda compilation tools, release 12.0, V12.0.140
#define CUDNN_MAJOR 8
#define CUDNN_MINOR 9
#define CUDNN_PATCHLEVEL 7

#define CUDNN_VERSION (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL)

/* cannot use constexpr here since this is a C-only file */

ps3: images

IMAGE ID DISK USAGE CONTENT SIZE EXTRA
debian:bookworm-slim 8242eaf53627 25.3GB 8.76GB
local/nextcloud:32-fpm-tensor 38bf16099e3b 2.08GB 523MB
local/nextcloud:32-fpm-tensor-custom 321801667192 5.39GB 1.88GB U
local/nextcloud:32.0.3-fpm-tensor 38bf16099e3b 2.08GB 523MB
local/nextcloud:32.0.3-fpm-tensor-custom 321801667192 5.39GB 1.88GB U
local/nextcloud:32.0.3-fpm-tensor-custom-20260114-094558 321801667192 5.39GB 1.88GB U
local/nextcloud:latest 321801667192 5.39GB 1.88GB U
local/nextcloud:latest-gpu 321801667192 5.39GB 1.88GB U
local/php:8.2-fpm-bookworm 27a6e2fea14a 29.6GB 10.1GB
mariadb:10.11 8763a63f00ec 465MB 111MB U
nginx:alpine 66d420cc54ef 93.1MB 26.8MB U
nvcr.io/nvidia/tensorflow:24.10-tf2-py3 8242eaf53627 25.3GB 8.76GB
php:8.2-fpm-bookworm 27a6e2fea14a 29.6GB 10.1GB

ps4: as of this morning they were 31 but i rebuilt for 32 as per your request
BR

p5: my docker-compose

services:
app:
# Build ettiğin en güncel ve kapsamlı imajı kullanıyoruz
image: local/nextcloud:latest-gpu
container_name: nextcloud-app
restart: unless-stopped
environment:
- MYSQL_HOST=db
- MYSQL_DATABASE=nextcloud
- MYSQL_USER=nextcloud
- MYSQL_PASSWORD=example
# NVIDIA sürücülerinin PHP ve Node tarafından görülmesi için kritik:
- NVIDIA_VISIBLE_DEVICES=all
- NVIDIA_DRIVER_CAPABILITIES=all
- LD_LIBRARY_PATH=/usr/local/nvidia/lib64:/usr/lib/x86_64-linux-gnu
volumes:
- nextcloud_data:/var/www/html
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu, compute, utility]
depends_on:
- db

db:
image: mariadb:10.11
container_name: nextcloud_db
restart: unless-stopped
environment:
- MYSQL_ROOT_PASSWORD=example
- MYSQL_DATABASE=nextcloud
- MYSQL_USER=nextcloud
- MYSQL_PASSWORD=example
volumes:
- db_data:/var/lib/mysql

web:
image: nginx:alpine
container_name: nextcloud-web
restart: unless-stopped
ports:
- "8080:80"
volumes:
- nextcloud_data:/var/www/html:ro
- ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
depends_on:
- app

volumes:
nextcloud_data:
external: true
db_data:
external: true

BR -Volkan