Failed to compile cuda_ndarray.cu: libcublas.so.7.5: cannot open shared object file

I am trying to import theano library in an aws instance to use GPU. I have written a python script using boto to automate aws setup which will essentially do an ssh to the instance from my local machine and then start a bash script where I do "python -c 'import theano'" to start the GPU. But I get the following error:

ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: libcublas.so.7.5: cannot open shared object file: No such file or directory

When I tried to import theano module directly in the instance command shell it automatically starts using GPU.

Using gpu device 0: GRID K520 (CNMeM is disabled)

I guess I am missing some other import that has to made while importing through my automation python script. What could possibly be the solution?

Answers 1

  • I will try to solve this problem clearly and concise, as I found not really good answer for people which are starting using unix or are not familiar with compilation and linking.

    The problem has to do with dynamic linking and it can be solved in two ways. First one is by setting LD_LIBRARY_PATH enviroment variable. Assuming cuda is installed in /usr/local/cuda/, just add in your enviroment file /etc/enviroment:

    LD_LIBRARY_PATH=/usr/local/cuda/
    

    Or simply in your bashrc:

    export LD_LIBRARY_PATH=/usr/local/cuda/lib64/
    

    This solution is not recommended by unix gurus (i am not one i have just read that on the internet and i follow linux gurus). So the solution I found is simple, modify the path where the linux ld software search for libraries by default. To do that just do (you have to do it as root):

    cd /etc/ld.so.conf.d/
    

    Then pick for example and edit:

    vi libc.conf 
    

    Inside this file just add the path to the lib64 root like:

    /usr/local/cuda/lib64/
    

    You would get something like this in the file:

    \# libc default configuration
    
    /usr/local/lib
    
    /usr/local/cuda/lib64/
    

    And then just run:

    sudo ldconfig
    

    Hope this answer helps people which are starting seen programming, or using high level languages such as python that uses C code below (like theano does) and are not familiar with compilation, linkig...


Related Articles