[GIS] SP1 for ArcGIS 10.1 has extension for 64-bit Python for background geoprocessing — implications for other Python use

arcgis-10.1arcpybackground-geoprocessing

If the 64-bit Background Geoprocessing for ArcGIS Desktop 10.1sp1 is installed how does that affect Python configuration for general use?

Esri has released SP1 for the ArcGIS 10.1 product, and for the Desktop component has made available a 64-bit Python 2.7.2 instance that when installed replaces Esri 32-bit Python 2.7 calls with 64-bit calls for background geoprocessing (processing sent out side of an active ArcGIS Desktop GUI session).

Installation is optional–i.e. without it ArcGIS Desktop continues with only 32-bit user memory space available. With it, a \bin64 entry is made to the Esri program folder, and registry keys are established for 64 bit processing at "HKEY_LOCAL_MACHINE\SOFTWARE\ESRI\ArcGIS 10.1 SP1 for Desktop Background Geoprocessing (64-bit)" and background geoprocessing is then done in 64-bit memory space.

Both Esri's 32-bit and now its 64-bit Python are full 2.7.2 installations.

During background geoprocessing Esri's registry, path and configuration files take care of calling the 64-bit Python executables. While use of 32-bit Python is retained in the ArcGIS Desktop session where needed.

So, what is left to each user is decide whether to integrate the installed 64-bit Python 2.7.2 into their computing needs outside of ArcGIS. And how best to do that when both 32 and 64 bit versions of Python are resident on the system?

Asked another way–Esri takes care of integration of 64-bit Python within its ArcGIS Desktop, so what ways will work to configure it for the rest of the use cases beyond ArcGIS?

Best Answer

So, what is left to each user is decide whether to integrate the installed 64-bit Python 2.7.2 into their computing needs outside of ArcGIS. And how best to do that when both 32 and 64 bit versions of Python are resident on the system?

As I understood you are asking the way we can distinguish different versions of python and use it according to our needs.

The easiest way is to exactly specify the interpreter you are going to use to run your script. More details can be found in this SO q/a.

Related Solutions

[GIS] Importing ArcPy taking many seconds and code is really slow for GeoProcessing Tools 64-bit (ArcGIS 10.1 SP1)

I also create ArcPy geoprocessing tools using PyDev for Eclipse. Generally, I run my scripts as in-process geoprocessing tools in a custom toolbox (.tbx) from within ArcMap so that the import arcpy happens instantly. If I run a script from outside of ArcMap, the import arcpy step usually takes about 5 seconds on a Sandy Bridge Core i5.

I think your trouble may lie in the use of 64-bit Python and the ArcGIS 64-bit background geoprocessing package. I don't know why this would take so much longer to import compared to 32-bit ArcPy. If you want confirmation, you could iteratively add/remove Python libraries from the Python Interpreters menu you have and see if a specific folder is causing the excessive import times.

Personally, I haven't found a compelling need to use 64-bit geoprocessing. Even in some of the massive Spatial Analyst workflows I've had to implement, geoprocessing usually crashes before I hit memory limits. I prefer to divide my inputs before processing and merge them together at the end. This also ensures better compatibility with 3rd part modules, most of which are already compiled for 32-bit Python.

I also think there's a bunch of tweaks you can make to your code to make it faster, besides the import arcpy issue and 64- vs. 32-bit Python. Creating a new searchCursor object for each iteration of your fc_nodes features is really slow. This means you are creating 6290 * 6290 = 39,564,100 search cursor objects. Creating each cursor object has processing overhead, so it's not surprising that this is taking many minutes to run. You want to avoid this code structure with nested cursors.

It appears that you are trying to find the counts of features in fc_links with the value of the TONODENO field equal to each possible value of the NO field in fc_nodes. Instead of using these nested cursors, it would be much easier to use the arcpy.Statistics_analysis() tool, with the TONODENO as the case field:

fc_links = "Rede_Cbr_MM_2008_WithoutMetro"
fieldname = "TONODENO"
statisticsTable = "OutputStatisticsTable"
arcpy.Statistics_analysis(fc_links, statisticsTable, [[fieldname,"COUNT"]], fieldname)

This is similar to do a Group By query in SQL, or a Totals query in Excel. In the output table, the FREQUENCY column gives the number of features in fc_links that have that value for the TONODENO field. You can join this back table to the fc_nodes table, or store it in a dictionary or something like that.

ArcGIS 10.2 – Is 64-bit Background Geoprocessing Native?

There is an option in the ArcGIS 10.2 Desktop Install for 64bit background Geo-Processing.

enter image description here

Best Answer

Related Solutions

[GIS] Importing ArcPy taking many seconds and code is really slow for GeoProcessing Tools 64-bit (ArcGIS 10.1 SP1)

ArcGIS 10.2 – Is 64-bit Background Geoprocessing Native?

Related Question