Main Question: Is there a geodatabase type that seamlessly works with both ESRI and open source products?
Details: My company currently passes around geodata in various vector and raster file formats and isn't particularly organized. Most processing is done using open source tools, but some customers demand that we host our data in ESRI Map service format. We have an ArcGIS server instance already that hosts these services, but we still do our processing upstream using open source tools before dumping the results in an ESRI arcsde geodatabase. From what I can tell, ogr/gdal have somewhat limited abilities to read/write from an arcsde geodatabase, but not enough that I could reasonably store all of our data in an arcsde geodatabase and expect our processing tools to be able to use the arcsde geodatabase as the primary data store for the company. Am I wrong? If I switched to an open source geodatabase such as postgis, our processing tools would work fine but I don't think that I could use a postgis database as the data store for hosting ESRI services. Am I wrong? Is there a geodatabase type out there that can service both open source and ESRI seamlessly? If not is it common practice to maintain parallel geodatabases, one for open source and one for ESRI?
Additional wrinkle:
The answer I got was to use PostgreSQL/PostGIS . The only potential wrinkle with that is that I'd like to host this database on Amazon Web Services (AWS) and if possible I'd like to use Amazon Relational Database Service (RDS) to minimize our database management overhead (backups, load balancing, etc). However, RDS only currently supports MySQL, Oracle and SQL Server as database engines. I'm thinking that if RDS starts supporting PostgreSQL that this would be the ideal scenario, but just to double check, PostgreSQL is my only option for ArcGIS + gdal/ogr-based open source interoperability correct? If so I'll likely create a PostgreSQL database on AWS, I'll just have to spend more resources managing it.
Best Answer
You can use a native PostGIS database as an SDE data store.
http://resources.arcgis.com/en/help/main/10.1/index.html#//002p0000006v000000
That link describes the basic setup to register a native PostGIS table with SDE. The drawback is that ESRI only supports a narrow range of Postgres and PostGIS versions, here's the list of what they support:
http://resources.arcgis.com/en/help/system-requirements/10.1/index.html#//015100000075000000