Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libcublas.so.*[0-9] not found in the system path on CPU-only machine #19

Open
0x62 opened this issue Sep 13, 2023 · 0 comments
Open

libcublas.so.*[0-9] not found in the system path on CPU-only machine #19

0x62 opened this issue Sep 13, 2023 · 0 comments

Comments

@0x62
Copy link

0x62 commented Sep 13, 2023

I'm trying to make use of this package on a fly.io machine. The image is built with Docker:

# Using the python:3.11-slim image
FROM python:3.11-slim

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PIP_NO_CACHE_DIR=1 PIPENV_VENV_IN_PROJECT=1 CUDA_VISIBLE_DEVICES=""

# Update and install dependencies
RUN apt-get update && \
    apt-get install -y --no-install-recommends gcc libffi-dev libssl-dev && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /app

# Install pip and pipenv
RUN pip install --upgrade pip && \
    pip install pipenv

# Copy Pipfile and Pipfile.lock
COPY Pipfile Pipfile.lock ./

# Install dependencies using pipenv
RUN pipenv install --deploy --ignore-pipfile

# Copy the rest of the application files
COPY . .

ENTRYPOINT ["pipenv", "run"]

The image builds successfully, but when run it I get an error from Torch:

   INFO Preparing to run: `pipenv run python worker.py` as root
   INFO [fly api proxy] listening at /.fly/api
  2023/09/13 17:25:40 listening on [fdaa:1:291c:a7b:173:c2e5:763f:2]:22 (DNS: [fdaa::3]:53)
  Loading .env environment variables...
  Traceback (most recent call last):
    File "/app/.venv/lib/python3.11/site-packages/torch/__init__.py", line 168, in _load_global_deps
      ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
    File "/usr/local/lib/python3.11/ctypes/__init__.py", line 376, in __init__
      self._handle = _dlopen(self._name, mode)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  OSError: libcurand.so.10: cannot open shared object file: No such file or directory
  During handling of the above exception, another exception occurred:
  Traceback (most recent call last):
    File "/app/worker.py", line 5, in <module>
      from workers.classification import process as classify
    File "/app/workers/classification.py", line 2, in <module>
      import tweetnlp
    File "/app/.venv/lib/python3.11/site-packages/tweetnlp/__init__.py", line 2, in <module>
      from .loader import load_model, load_dataset
    File "/app/.venv/lib/python3.11/site-packages/tweetnlp/loader.py", line 3, in <module>
      from .text_classification.model import Sentiment, Offensive, Irony, Hate, Emotion, Emoji,\
    File "/app/.venv/lib/python3.11/site-packages/tweetnlp/text_classification/model.py", line 7, in <module>
      import torch
    File "/app/.venv/lib/python3.11/site-packages/torch/__init__.py", line 228, in <module>
      _load_global_deps()
    File "/app/.venv/lib/python3.11/site-packages/torch/__init__.py", line 189, in _load_global_deps
      _preload_cuda_deps(lib_folder, lib_name)
    File "/app/.venv/lib/python3.11/site-packages/torch/__init__.py", line 154, in _preload_cuda_deps
      raise ValueError(f"{lib_name} not found in the system path {sys.path}")
  ValueError: libcublas.so.*[0-9] not found in the system path ['/app', '/usr/local/lib/python311.zip', '/usr/local/lib/python3.11', '/usr/local/lib/python3.11/lib-dynload', '/app/.venv/lib/python3.11/site-packages']

Looking at #18, it appears to support CPU only, and the machine won't have any GPU's available so I'm not sure why it's not following the CPU init path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant