FDO RFC 42 - Direct ArcSDE FDO Provider
This page contains a request for comments document (RFC) for the FDO Open Source project. More FDO RFCs can be found on the RFCs page.
Status
| RFC Template Version | (1.0) |
| Submission Date | 17 October 2009 |
| Last Modified | Crispin Hoult 17 October 2009 |
| Author | Crispin Hoult, Alan Douglas, Haris Kurtagic |
| RFC Status | Adopted |
| Implementation Status | Internal Prototype / Proof of Concept / Build Candidate |
| Proposed Milestone | 3.5.0 |
| Assigned PSC guide(s) | Greg Boone |
| Voting History | |
| +1 | Jackie Ng, Haris Kurtagic, Jason Birch, Traian Stanev, Frank Warmerdam |
| +0 | Greg Boone, Orest Halustchak |
| -0 | |
| -1 |
Overview
The purpose of the RFC is to provide a direct provider for SDE datasources in Oracle without the requirement to utilise the ESRI ArcSDE SDK or ArcSDE server service.
Motivation
ArcSDE is the most common "enterprise" geospatial database engine encountered by 1Spatial in solution deployment. Of these datastores the majority are on an Oracle platform with data in the ESRI SDE binary format. An overview of the format can be found in "Resources"details below. It is recognised that there is an existing OSGeo provider for ArcSDE data (ArcGIS servers 9.1, 9.2 and 9.3 at FDO 3.4). The existing provider supports multiple connection methodologies (client and direct-connect), data types (SDE and SDO on Oracle) and server technologies (Oracle, SQLServer). However the platform technology (SDK/API) this is based on causes issues for the ongoing support and development outlined below.
The provision and adoption of a new provider is no small task, and the creation of one that parallels existing functionality requires some explanation. The drivers behind this RFC are as follows:
- The existing OSGeo ArcSDE provider builds on the ArcSDE SDK - this is a commercial offering from ESRI and it's cost and install-base limit community involvement in the open source support and maintenance of the provider
- The use of the ArcSDE SDK requires a more complex implementation/deployment of the FDO provider where the relevant supporting ESRI client DLLs need to be obtained and copied to the FDO folder - this adds version support issues and client licensing questions into implementation
- The use of the ArcSDE SDK is built against specific versions of the ArcSDE server and SDK - there is a reliance on ESRI to support new server versions (e.g. ArcGIS 9.3) in the SDK and on the current owner (OSGeo) to source and rebuild appropriately and timely
- The use of the ArcSDE SDK utilises calls directly to the ArcSDE server - where a partner has a restricted ArcSDE license is ("Application Specific License") there are licensing issues relating to the current provider and use of the ArcSDE server
- The use of the ArcSDE SDK is limited to 32-bit builds of the provider - there is a wider requirement to have a 64-bit provider for ArcSDE data and the current approach does not support this
With consideration of the current providers limitations above the motivation for a non SDK-based provider are proven and this RFC details the proposed solution below.
Proposed Solution
The initial set of functionality for this provider will include
- Support for simple geometries
- Limited capabilities support for spatial and attribute filters (no expression evaluation)
- Support for select/sqlcommand only (no insert/update/delete)
- Limited platform support to Oracle database
Specific capabilities are built on the KingOra platform and are therefore identical. As capabilities are defined at the provider level and not based on the connection parameters some consideration will have to be given to the "SDE Schema" connection being more constrained than a pure KingOra SDO connection
Connection
- ThreadCapability: PerConnectionThreaded
- SpatialContextExtent: Static
- SupportsLocking: : false
- SupportsTimeout: : false
- SupportsTransactions: : false
- SupportsLongTransactions: : false
- !SupportsSQL: : true
- SupportsConfiguration: : true
Schema
- Class
- FeatureClass
- Class
- Data
- Boolean
- Byte
- DateTime
- Decimal
- Double
- Int16
- Int32
- Int64
- Single
- String
- SupportsInheritance: false
- SupportsMultipleSchemas: true
- SupportsObjectProperties: false
- SupportsAssociationProperties: false
- SupportsSchemaOverrides: true
- SupportsNetworkModel: false
- SupportsAutoIdGeneration: false
- SupportsDataStoreScopeUniqueIdGeneration: false
- SupportedAutoGeneratedTypes: false
- SupportsSchemaModification: false
Command
- SupportedCommands
- Select
- SQLCommand
- DescribeSchema
- GetSpatialContexts
- CreateSpatialContext
- SupportsParameters: true
- SupportsTimeout: false
- SupportsSelectExpressions: true
- SupportsSelectFunctions: true
- SupportsSelectDistinct: true
- SupportsSelectOrdering: true
- SupportsSelectGrouping: true
Filter
- Condition
- Comparison
- Like
- In
- Null
- Spatial
- Spatial
- Intersects
- EnvelopeIntersects
- SupportsGeodesicDistance: false
- SupportsNonLiteralGeometricOperations: false
Expression
- Function
- SpatialExtents
Raster
- SupportsRaster: false
- SupportsStitching: false
- SupportsSubsampling: false
Topology
- SupportsTopology: false
- SupportsTopologicalHierarchy: false
- BreaksCurveCrossingsAutomatically: false
- ActivatesTopologyByArea: false
- ConstrainsFeatureMovements: false
Geometry
- Types
- Point
- LineString
- Polygon
- MultiPolygon
- Components
- LinearRing
- Dimensionality: 2
Limitations
The main limitations of this provider at present are the following. It is envisioned that community support and development can extend the provider into these areas:.
- No support for complex geometries or curves
- Only SDE datastores on Oracle will be accessed
- The provider will only be tested on SDE 9.1 and 9.2 schemas with geometry in "normal" precision (64-bit "high" precision not supported - yet)
- There will be no expression support
- There will be no insert/update/delete capability (read-only provider)
- The spatial expressions will be restricted
- Indexing/performance will be based on simple bounding-box filters
- Spatial queries will be based on simple bounding-box filter
Implications
This provider does not require any API changes to the FDO core or other providers. It will need to be added to the standard build and test procedure.
Test Plan
The Direct SDE Provider for ArcSDE will include unit tests. Unit tests will be run against a sample SDE schema dumped, Oracle exported and imported. The provider will be built as a trunk (3.5) and 3.4 compatible provider and tested in AutoCAD Map 2010 and MapGuide OpenSource 2.1 Where reources are available a FDO 3.3 build will also be provided.
References
The following list of resources will be used to support creation of the provider.
- http://edndoc.esri.com/arcsde/9.1/capi_concepts/arcsde_compressed_binary.htm
- http://edndoc.esri.com/arcsde/9.2/concepts/geometry/geometrystorage/binary.htm
- http://edndoc.esri.com/arcsde/9.0/general_topics/binary_geometry_format.htm
- http://www.codeplex.com/SharpMap/Thread/View.aspx?ThreadId=9346
- http://trac.osgeo.org/gdal/browser/trunk/gdal/ogr/ogrsf_frmts/pgeo/ogrpgeolayer.cpp (may be relationship between Personal Geodatabase geometry and SDO
Funding/Resources
The resources for this provider are being contributed by 1Spatial with foundation (KingOra) and expertise by SL-King.
