[GIS] Installing python GDAL with pip in local (user) space on RHEL6 server: failure due to missing headers

gdallinux

There are similar themed questions for other platforms, but the solutions out there do not work for me.

(e.g. https://stackoverflow.com/questions/11336153/python-gdal-package-missing-header-file-when-installing-via-pip)

I am using a virtualenv to have a user-space implementation of python on a RHEL GNU/Linux server.

I have installed lots of favourite packages into my local python directory with little problem, using pip.

pip install gdal, on the other hand, fails, with many missing header file errors.

I have tried

pip install –no-install gdal

followed by navigating to (my python path)/build/GDAL and then

python setup.py build_ext –include-dirs=/usr/include/gdal

but that fails with lots of other errors:

running build_ext
building 'osgeo._gdal' extension
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/gdal/ -I/usr/include/python2.7 -I/home/projects/sprawl/pythonSprawl/lib/python2.7/site-packages/numpy/core/include -I/usr/include -c extensions/gdal_wrap.cpp -o build/temp.linux-x86_64-2.7/extensions/gdal_wrap.o
extensions/gdal_wrap.cpp:3000: error: ‘VSILFILE’ has not been declared
extensions/gdal_wrap.cpp: In function ‘int wrapper_VSIFReadL(void**, int, int, int*)’:
extensions/gdal_wrap.cpp:3035: error: cannot convert ‘int*’ to ‘FILE*’ for argument ‘4’ to ‘size_t VSIFReadL(void*, size_t, size_t, FILE*)’
extensions/gdal_wrap.cpp: In function ‘int PyProgressProxy(double, const char*, void*)’:
extensions/gdal_wrap.cpp:3237: warning: the address of ‘_Py_NoneStruct’ will never be NULL
extensions/gdal_wrap.cpp: In function ‘void PyCPLErrorHandler(CPLErr, int, const char*)’:
extensions/gdal_wrap.cpp:3373: error: ‘CPLGetErrorHandlerUserData’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘CPLErr PushErrorHandler(void (*)(CPLErr, int, const char*), void*)’:
extensions/gdal_wrap.cpp:3387: error: ‘CPLPushErrorHandlerEx’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘void PopErrorHandler()’:
extensions/gdal_wrap.cpp:3394: error: ‘CPLGetErrorHandlerUserData’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘int wrapper_VSIStatL(const char*, StatBuf*, int)’:
extensions/gdal_wrap.cpp:3488: error: ‘VSIStatExL’ was not declared in this scope
extensions/gdal_wrap.cpp: At global scope:
extensions/gdal_wrap.cpp:3496: error: ‘VSILFILE’ has not been declared
extensions/gdal_wrap.cpp: In function ‘int wrapper_VSIFWriteL(int, char*, int, int, int*)’:
extensions/gdal_wrap.cpp:3503: error: cannot convert ‘int*’ to ‘FILE*’ for argument ‘4’ to ‘size_t VSIFWriteL(const void*, size_t, size_t, FILE*)’
extensions/gdal_wrap.cpp: At global scope:
extensions/gdal_wrap.cpp:3832: error: ‘GDALAsyncReaderH’ does not name a type
extensions/gdal_wrap.cpp:3838: error: ‘GDALAsyncReaderH’ does not name a type
extensions/gdal_wrap.cpp: In function ‘void DeleteAsyncReaderWrapper(void*)’:
extensions/gdal_wrap.cpp:3857: error: ‘struct GDALAsyncReaderWrapper’ has no member named ‘hAsyncReader’
extensions/gdal_wrap.cpp: At global scope:
extensions/gdal_wrap.cpp:3867: error: ‘GDALAsyncReaderH’ was not declared in this scope
extensions/gdal_wrap.cpp:3868: error: expected primary-expression before ‘void’
extensions/gdal_wrap.cpp:3868: error: initializer expression list treated as compound expression
extensions/gdal_wrap.cpp:3869: error: expected ‘,’ or ‘;’ before ‘{’ token
extensions/gdal_wrap.cpp: In function ‘void DisableAsyncReaderWrapper(void*)’:
extensions/gdal_wrap.cpp:3885: error: ‘struct GDALAsyncReaderWrapper’ has no member named ‘hAsyncReader’
extensions/gdal_wrap.cpp: At global scope:
extensions/gdal_wrap.cpp:3892: error: ‘GDALAsyncStatusType’ does not name a type
extensions/gdal_wrap.cpp: In function ‘void GDALAsyncReaderShadow_GetBuffer(GDALAsyncReaderShadow*, void**)’:
extensions/gdal_wrap.cpp:3905: error: ‘GDALAsyncReaderH’ was not declared in this scope
extensions/gdal_wrap.cpp:3905: error: expected ‘;’ before ‘hReader’
extensions/gdal_wrap.cpp:3906: error: ‘hReader’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘int GDALAsyncReaderShadow_LockBuffer(GDALAsyncReaderShadow*, double)’:
extensions/gdal_wrap.cpp:3915: error: ‘GDALAsyncReaderH’ was not declared in this scope
extensions/gdal_wrap.cpp:3915: error: expected ‘;’ before ‘hReader’
extensions/gdal_wrap.cpp:3916: error: ‘hReader’ was not declared in this scope
extensions/gdal_wrap.cpp:3920: error: ‘hReader’ was not declared in this scope
extensions/gdal_wrap.cpp:3920: error: ‘GDALARLockBuffer’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘void GDALAsyncReaderShadow_UnlockBuffer(GDALAsyncReaderShadow*)’:
extensions/gdal_wrap.cpp:3923: error: ‘GDALAsyncReaderH’ was not declared in this scope
extensions/gdal_wrap.cpp:3923: error: expected ‘;’ before ‘hReader’
extensions/gdal_wrap.cpp:3924: error: ‘hReader’ was not declared in this scope
extensions/gdal_wrap.cpp:3928: error: ‘hReader’ was not declared in this scope
extensions/gdal_wrap.cpp:3928: error: ‘GDALARUnlockBuffer’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘GDALAsyncReaderShadow* GDALDatasetShadow_BeginAsyncReader(GDALDatasetShadow*, int, int, int, int, int, char*, void*, int, int, GDALDataType, int, int*, int, int, int, char**)’:
extensions/gdal_wrap.cpp:4095: error: ‘GDALAsyncReaderH’ was not declared in this scope
extensions/gdal_wrap.cpp:4095: error: expected ‘;’ before ‘hAsyncReader’
extensions/gdal_wrap.cpp:4098: error: ‘hAsyncReader’ was not declared in this scope
extensions/gdal_wrap.cpp:4100: error: ‘GDALAsyncReader’ was not declared in this scope
extensions/gdal_wrap.cpp:4100: error: expected primary-expression before ‘)’ token
extensions/gdal_wrap.cpp:4100: error: expected ‘;’ before ‘CreateAsyncReaderWrapper’
extensions/gdal_wrap.cpp:4100: error: ‘CreateAsyncReaderWrapper’ cannot be used as a function
extensions/gdal_wrap.cpp: In function ‘void GDALDatasetShadow_EndAsyncReader(GDALDatasetShadow*, GDALAsyncReaderShadow*)’:
extensions/gdal_wrap.cpp:4114: error: ‘GDALAsyncReaderH’ was not declared in this scope
extensions/gdal_wrap.cpp:4114: error: expected ‘;’ before ‘hReader’
extensions/gdal_wrap.cpp:4115: error: ‘hReader’ was not declared in this scope
extensions/gdal_wrap.cpp:4119: error: ‘hReader’ was not declared in this scope
extensions/gdal_wrap.cpp:4119: error: ‘GDALEndAsyncReader’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘CPLErr GDALRasterBandShadow_SetUnitType(GDALRasterBandShadow*, const char*)’:
extensions/gdal_wrap.cpp:4305: error: ‘GDALSetRasterUnitType’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘int GDALTransformerInfoShadow_TransformGeolocations(GDALTransformerInfoShadow*, GDALRasterBandShadow*, GDALRasterBandShadow*, GDALRasterBandShadow*, int (*)(double, const char*, void*), void*, char**)’:
extensions/gdal_wrap.cpp:5010: error: ‘GDALTransformGeolocations’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘GIntBig wrapper_GDALGetCacheMax()’:
extensions/gdal_wrap.cpp:5015: error: ‘GDALGetCacheMax64’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘GIntBig wrapper_GDALGetCacheUsed()’:
extensions/gdal_wrap.cpp:5021: error: ‘GDALGetCacheUsed64’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘void wrapper_GDALSetCacheMax(GIntBig)’:
extensions/gdal_wrap.cpp:5027: error: ‘GDALSetCacheMax64’ was not declared in this scope
extensions/gdal_wrap.cpp:5027: error: return-statement with a value, in function returning 'void'
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_VSIFReadL(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:5220: error: ‘VSILFILE’ was not declared in this scope
extensions/gdal_wrap.cpp:5220: error: ‘arg4’ was not declared in this scope
extensions/gdal_wrap.cpp:5220: error: expected primary-expression before ‘)’ token
extensions/gdal_wrap.cpp:5220: error: expected ‘;’ before numeric constant
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_GOA2GetAuthorizationURL(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:5553: error: ‘GOA2GetAuthorizationURL’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_GOA2GetRefreshToken(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:5606: error: ‘GOA2GetRefreshToken’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_GOA2GetAccessToken(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:5661: error: ‘GOA2GetAccessToken’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_ReadDirRecursive(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:6081: error: ‘VSIReadDirRecursive’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_VSIFOpenL(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:6935: error: ‘VSILFILE’ was not declared in this scope
extensions/gdal_wrap.cpp:6935: error: ‘result’ was not declared in this scope
extensions/gdal_wrap.cpp:6956: error: expected primary-expression before ‘)’ token
extensions/gdal_wrap.cpp:6956: error: expected ‘;’ before ‘VSIFOpenL’
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_VSIFCloseL(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:6983: error: ‘VSILFILE’ was not declared in this scope
extensions/gdal_wrap.cpp:6983: error: ‘arg1’ was not declared in this scope
extensions/gdal_wrap.cpp:6983: error: expected primary-expression before ‘)’ token
extensions/gdal_wrap.cpp:6983: error: expected ‘;’ before numeric constant
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_VSIFSeekL(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:7013: error: ‘VSILFILE’ was not declared in this scope
extensions/gdal_wrap.cpp:7013: error: ‘arg1’ was not declared in this scope
extensions/gdal_wrap.cpp:7013: error: expected primary-expression before ‘)’ token
extensions/gdal_wrap.cpp:7013: error: expected ‘;’ before numeric constant
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_VSIFTellL(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:7063: error: ‘VSILFILE’ was not declared in this scope
extensions/gdal_wrap.cpp:7063: error: ‘arg1’ was not declared in this scope
extensions/gdal_wrap.cpp:7063: error: expected primary-expression before ‘)’ token
extensions/gdal_wrap.cpp:7063: error: expected ‘;’ before numeric constant
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_VSIFTruncateL(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:7102: error: ‘VSILFILE’ was not declared in this scope
extensions/gdal_wrap.cpp:7102: error: ‘arg1’ was not declared in this scope
extensions/gdal_wrap.cpp:7102: error: expected primary-expression before ‘)’ token
extensions/gdal_wrap.cpp:7102: error: expected ‘;’ before numeric constant
extensions/gdal_wrap.cpp:7126: error: ‘VSIFTruncateL’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_VSIFWriteL(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:7147: error: ‘VSILFILE’ was not declared in this scope
extensions/gdal_wrap.cpp:7147: error: ‘arg5’ was not declared in this scope
extensions/gdal_wrap.cpp:7147: error: expected primary-expression before ‘)’ token
extensions/gdal_wrap.cpp:7147: error: expected ‘;’ before numeric constant
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_MajorObject_SetMetadata__SWIG_0(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:7501: warning: deprecated conversion from string constant to ‘char*’
extensions/gdal_wrap.cpp: In function ‘PyObject* _wrap_AsyncReader_GetNextUpdatedRegion(PyObject*, PyObject*)’:
extensions/gdal_wrap.cpp:10998: error: ‘GDALAsyncStatusType’ was not declared in this scope
extensions/gdal_wrap.cpp:10998: error: expected ‘;’ before ‘result’
extensions/gdal_wrap.cpp:11019: error: ‘result’ was not declared in this scope
extensions/gdal_wrap.cpp:11019: error: expected ‘;’ before ‘GDALAsyncReaderShadow_GetNextUpdatedRegion’
extensions/gdal_wrap.cpp:11027: error: ‘result’ was not declared in this scope
error: command 'gcc' failed with exit status 1

I've also tried editing setup.cfg to put

gdal_config =/usr/bin/gdal-config
near the top, and then running
pip install –no-download GDAL

but that failed as well, with the original header file failures:

Installing collected packages: GDAL Running setup.py install for
GDAL
building 'osgeo._gdal' extension
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector –paeneric -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protectotune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/include/pyhonSprawl/lib/python2.7/site-packages/numpy/core/include
-I/usr/include -c extensions/gdal_wrap.cpp -o build/temp.li.o
extensions/gdal_wrap.cpp:2853:22: error: cpl_port.h: No such file or directory
extensions/gdal_wrap.cpp:2854:24: error: cpl_string.h: No such file or directory
extensions/gdal_wrap.cpp:2855:27: error: cpl_multiproc.h: No such file or directory
extensions/gdal_wrap.cpp:2856:22: error: cpl_http.h: No such file or directory
extensions/gdal_wrap.cpp:2858:18: error: gdal.h: No such file or directory
extensions/gdal_wrap.cpp:2859:23: error: gdal_priv.h: No such file or directory
extensions/gdal_wrap.cpp:2860:22: error: gdal_alg.h: No such file or directory
extensions/gdal_wrap.cpp:2861:24: error: gdalwarper.h: No such file or directory
extensions/gdal_wrap.cpp:4625:22: error: gdalgrid.h: No such file or directory
extensions/gdal_wrap.cpp:2878: error: ‘CPLErrorHandler’ does not name a type
extensions/gdal_wrap.cpp:2881: error: expected initializer before ‘PythonBindingErrorHandler’
extensions/gdal_wrap.cpp:2753: warning: ‘swig_module’ defined but not used
error: command 'gcc' failed with exit status 1
Complete output from command /home/projects/sprawl/pythonSprawl/bin/python2.7 -c "import
setuptools;file='/hild/GDAL/setup.py';exec(compile(open(file).read().replace('\r\n',
'\n'), file, 'exec'))" install –record /t.txt
–single-version-externally-managed –install-headers /home/projects/sprawl/pythonSprawl/include/site/python2.7 running
install

running build

running build_py

running build_ext

building 'osgeo._gdal' extension

gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector –param=ic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector –=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/include/pythonprawl/lib/python2.7/site-packages/numpy/core/include
-I/usr/include -c extensions/gdal_wrap.cpp -o build/temp.linux- extensions/gdal_wrap.cpp:2853:22: error: cpl_port.h: No such file or
directory

extensions/gdal_wrap.cpp:2854:24: error: cpl_string.h: No such file or
directory

extensions/gdal_wrap.cpp:2855:27: error: cpl_multiproc.h: No such file
or directory

extensions/gdal_wrap.cpp:2856:22: error: cpl_http.h: No such file or
directory

extensions/gdal_wrap.cpp:2858:18: error: gdal.h: No such file or
directory

extensions/gdal_wrap.cpp:2859:23: error: gdal_priv.h: No such file or
directory

extensions/gdal_wrap.cpp:2860:22: error: gdal_alg.h: No such file or
directory

extensions/gdal_wrap.cpp:2861:24: error: gdalwarper.h: No such file or
directory

extensions/gdal_wrap.cpp:4625:22: error: gdalgrid.h: No such file or
directory

extensions/gdal_wrap.cpp:2878: error: ‘CPLErrorHandler’ does not name
a type

extensions/gdal_wrap.cpp:2881: error: expected initializer before
‘PythonBindingErrorHandler’

extensions/gdal_wrap.cpp:2753: warning: ‘swig_module’ defined but not
used

error: command 'gcc' failed with exit status 1

—————————————- Command /home/projects/sprawl/pythonSprawl/bin/python2.7 -c "import
setuptools;file='/home/projects/sprawl/pythoompile(open(file).read().replace('\r\n',
'\n'), file, 'exec'))" install –record
/tmp/pip-MFmW2k-record/insternally-managed –install-headers
/home/projects/sprawl/pythonSprawl/include/site/python2.7 failed with
error code 1awl/build/GDAL

Edit: btw, I think I got my sysadmin to instal gdal-devel successfully:

yum info gdal-devel
Loaded plugins: priorities, refresh-packagekit, rhnplugin, security
This system is receiving updates from RHN Classic or RHN Satellite.
1803 packages excluded due to repository priority protections
Installed Packages
Name        : gdal-devel
Arch        : x86_64
Version     : 1.7.2
Release     : 2.puias6
Size        : 16 M
Repo        : installed
From repo   : PUIAS_6_computational
Summary     : Development Libraries for the GDAL file format library
URL         : http://www.gdal.org/
License     : MIT
Description : The GDAL library provides support to handle multiple GIS file formats.

Can anyone give me advice on how to install the python GDAL package(s) into my virtualenv on RHEL?

Edits: explicit outcomes of more things I've tried:

python setup.py build_ext --gdal-config=/usr/bin/gdal-config --include-dirs=/usr/include/gdal --library-dirs=/usr/lib64

building 'osgeo._gdal' extension
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=gener
ic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune
=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/gdal -I/usr/include/python2.7 -I/home/projects/sprawl/pythonSprawl/lib/python2.7/site-package
s/numpy/core/include -I/usr/include -c extensions/gdal_wrap.cpp -o build/temp.linux-x86_64-2.7/extensions/gdal_wrap.o
extensions/gdal_wrap.cpp:3000: error: ‘VSILFILE’ has not been declared
extensions/gdal_wrap.cpp: In function ‘int wrapper_VSIFReadL(void**, int, int, int*)’:
extensions/gdal_wrap.cpp:3035: error: cannot convert ‘int*’ to ‘FILE*’ for argument ‘4’ to ‘size_t VSIFReadL(void*, size_t, size_t, FILE*)’
extensions/gdal_wrap.cpp: In function ‘int PyProgressProxy(double, const char*, void*)’:
extensions/gdal_wrap.cpp:3237: warning: the address of ‘_Py_NoneStruct’ will never be NULL
extensions/gdal_wrap.cpp: In function ‘void PyCPLErrorHandler(CPLErr, int, const char*)’:
extensions/gdal_wrap.cpp:3373: error: ‘CPLGetErrorHandlerUserData’ was not declared in this scope

etc etc (many errors)

Best Answer

My guess is that the gdal-dev version you have is outdated and that the version of python binding you got in pip is not compatible any more. I accidentally uninstalled python-gdal from RHEL, and unsuccessfully tried to install gdal with pip today. The first error I got was

extensions/gdal_wrap.cpp:3373: error: ‘CPLGetErrorHandlerUserData’ was not declared in this scope
extensions/gdal_wrap.cpp: In function ‘CPLErr PushErrorHandler(void (*)(CPLErr, int, const char*), void*)’:

which is the same you got near the beginning. I downloaded gdal v1.10.1 source today, and I grepped CPLGetErrorHanlerUserData, and found it in include/cpl_error.h, and not in the file in version 1.8.1 I had. After I compiled gdal 1.10.1, installed it and rerun pip install gdal. It gave a bit of warning but installed successfully.