I had been putting off sorting this problem out for a long time but finally have a proper solution. The situation is this:
- I need a 64bit version of Python 32 and GDAL for a package other than
QGIS.
- I need a 32bit version of GDAL to work with Python 2.7 for QGIS.
- QGIS consistently loads the wrong DLL and complains that it is not a 32bit Windows application.
The reason and solution are simple. I have environment variables which point to the 64 bit GDAL installation - which was installed before QGIS and I need to leave alone so my other applications will work. OSGeo appends its path information to the system path but because it is put at the end of the path, my pre-existing installs for 64-bit Python 3.2 are always picked up. So, I added the following code at the start of the GdalTools.py file (I used the one in C:\Users\MyAccount.qgis\python\plugins\GdalTools):
# hack the path to force QGIS to look in OSGeo4w directory first
import sys
sys.path.reverse()
Now QGIS starts with zero errors and I can use the GDAL tools, without switching to another machine.
EDIT (Windows 64 bit version July 2014):
I STILL have to do this hack because QGIS still uses Python27 and I need Python32 for other packages I use. I still get the same error message and forcing the paths in config section for QGIS still doesn't solve entirely the problem. However, this hack continues to neatly cure the issue (I am now editing the version of gdaltools.py in C:\OSGeo4W64\apps\qgis\python\plugins\GdalTools
on my system.
I also create ArcPy geoprocessing tools using PyDev for Eclipse. Generally, I run my scripts as in-process geoprocessing tools in a custom toolbox (.tbx) from within ArcMap so that the import arcpy
happens instantly. If I run a script from outside of ArcMap, the import arcpy
step usually takes about 5 seconds on a Sandy Bridge Core i5.
I think your trouble may lie in the use of 64-bit Python and the ArcGIS 64-bit background geoprocessing package. I don't know why this would take so much longer to import compared to 32-bit ArcPy. If you want confirmation, you could iteratively add/remove Python libraries from the Python Interpreters menu you have and see if a specific folder is causing the excessive import times.
Personally, I haven't found a compelling need to use 64-bit geoprocessing. Even in some of the massive Spatial Analyst workflows I've had to implement, geoprocessing usually crashes before I hit memory limits. I prefer to divide my inputs before processing and merge them together at the end. This also ensures better compatibility with 3rd part modules, most of which are already compiled for 32-bit Python.
I also think there's a bunch of tweaks you can make to your code to make it faster, besides the import arcpy
issue and 64- vs. 32-bit Python. Creating a new searchCursor
object for each iteration of your fc_nodes
features is really slow. This means you are creating 6290 * 6290 = 39,564,100
search cursor objects. Creating each cursor object has processing overhead, so it's not surprising that this is taking many minutes to run. You want to avoid this code structure with nested cursors.
It appears that you are trying to find the counts of features in fc_links
with the value of the TONODENO
field equal to each possible value of the NO
field in fc_nodes
. Instead of using these nested cursors, it would be much easier to use the arcpy.Statistics_analysis()
tool, with the TONODENO
as the case field:
fc_links = "Rede_Cbr_MM_2008_WithoutMetro"
fieldname = "TONODENO"
statisticsTable = "OutputStatisticsTable"
arcpy.Statistics_analysis(fc_links, statisticsTable, [[fieldname,"COUNT"]], fieldname)
This is similar to do a Group By query in SQL, or a Totals query in Excel. In the output table, the FREQUENCY
column gives the number of features in fc_links
that have that value for the TONODENO
field. You can join this back table to the fc_nodes
table, or store it in a dictionary or something like that.
Best Answer
As commented by @AlexTereshenkov installing a 64bit version of Python 2.7.3 with ArcGIS Desktop 9.3 is ill advised.
If you uninstall ArcGIS Desktop and Python, and then let ArcGIS Desktop install Python for you I suspect you will be up and running within an hour.
As you can see from the System Requirements for ArcGIS Desktop 9.3/9.3.1 the Python version needed appears to be 2.5.1: