FDO RFC 34 - FDO Reader Access By Index

This page contains an change request (RFC) for the FDO Open Source project. More FDO RFCs can be found on the RFCs page.

Status

RFC Template Version	(1.0)
Submission Date	April 9, 2009
Last Modified	Greg Boone Timestamp
Author	Greg Boone
RFC Status	Ready For Review
Implementation Status	Pending
Proposed Milestone	3.5.0.0
Assigned PSC guide(s)	Greg Boone
Voting History	(vote date)
+1
+0
-0
-1

Motivation

To provide faster access to data returned from feature, data and SQL readers though the FDO API.
To provide enhanced compatibility with ADO.Net Reader/Record Interfaces.

Overview

Currently, data returned in Feature readers, data or SQL readers can only be accessed using input function parameters that reflect the name of the property or column being read. In the following example, the feature reader returned from the select command has to be accessed using the GetDouble method, with an input parameter that is the feature property name. In such an implementation, there is an associated cost to determining which property is being requested. Such a cost is typically encountered in performing a lookup based on the parameter name, which inevitably results in some form of a string comparison.

FdoPtr<FdoISelect> spSelectCmd = static_cast<FdoISelect>(mConnection->CreateCommand(FdoCommandType_Select));
spSelectCmd->SetFeatureClassName (L"Foo");

FdoPtr<FdoIdentifierCollection> spIds = spSelectCmd->GetPropertyNames ();
FdoPtr<FdoIdentifier> spId = FdoComputedIdentifier::Create(L"AVG_ID", FdoPtr<FdoExpression>(FdoExpression::Parse(L"Avg(ID)")));
spIds->Add(spId);

FdoPtr<FdoIFeatureReader> spReader = spSelectCmd->Execute ();
while (spReader->ReadNext())
{
    double avg = spReader->GetDouble(L"AVG_ID");
}

The goal of this RFC is to add overloaded methods to the various FDO reader interfaces that will accept an integer argument representing the indexed location of the property in the reader's in-memory representation of the feature or table. Readers will return the total count of properties being returned and be able to indicate which property or column is indexed a specified location. It is expected that this will speed up processing and provide a better end-user experience. Users will be able to access their data using an index into the feature reader collection as follows:

FdoPtr<FdoISelect> spSelectCmd = static_cast<FdoISelect>(mConnection->CreateCommand(FdoCommandType_Select));
spSelectCmd->SetFeatureClassName (L"Foo");

FdoPtr<FdoIdentifierCollection> spIds = spSelectCmd->GetPropertyNames ();
FdoPtr<FdoIdentifier> spId = FdoComputedIdentifier::Create(L"AVG_ID", FdoPtr<FdoExpression>(FdoExpression::Parse(L"Avg(ID)")));
spIds->Add(spId);

FdoPtr<FdoIFeatureReader> spReader = spSelectCmd->Execute ();
while (spReader->ReadNext())
{
    double avg = spReader->GetDouble(0); // 0 is the index of the AVG_ID property
}

Requirements

The following FDO reader interfaces will be updated to allow for indexed access. Providers will be required to implement these methods.

Interface IReader

/// \brief
/// The FdoIReader interface provides a forward-only, read-only iterator
/// for reading feature data.  FdoIReader is the base interface for 
/// feature and data readers that are returned from the Select, 
/// and SelectAggregates commands. FdoIReader provides the funtion
/// definitions for the GetXxxx properties and the definition of IsNull() 
/// and ReadNext(). Because the initial position of the reader is prior 
/// to the first item, you must call ReadNext to begin accessing any data.
class FdoIReader: public FdoIDisposable
{
public:
...
...
...
    /// \brief
    /// Returns true if the value of the property at the specified 
    /// index is null.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns true if the value is null.
    /// 
    FDO_API virtual FdoBoolean IsNull(FdoInt32 index) = 0;

    /// \brief
    /// Gets the Boolean value of the property specified at the index position. 
    /// No conversion is performed, thus the property must be FdoDataType_Boolean 
    /// or an exception is thrown.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the Boolean value.
    /// 
    FDO_API virtual FdoBoolean GetBoolean(FdoInt32 index) = 0;

    /// \brief
    /// Gets the Byte value of the property specified at the index position. 
    /// No conversion is performed, thus the property must be FdoDataType_Byte 
    /// or an exception is thrown.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the byte value.
    /// 
    FDO_API virtual FdoByte GetByte(FdoInt32 index) = 0;

    /// \brief
    /// Gets the date and time value of the of the property specified at 
    /// the index position. No conversion is performed, thus the property 
    /// must be FdoDataType_DateTime or an exception is thrown.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the date and time value.
    /// 
    FDO_API virtual FdoDateTime GetDateTime(FdoInt32 index);

    /// \brief
    /// Gets the double-precision floating point value of the property specified at 
    /// the index position. No conversion is performed, thus the property must be 
    /// FdoDataType_Double or an exception is thrown.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the double floating point value
    /// 
    FDO_API virtual FdoDouble GetDouble(FdoInt32 index) = 0;

    /// \brief
    /// Gets the 16-bit integer value of the property specified at 
    /// the index position. No conversion is performed, thus the 
    /// property must be FdoDataType_Int16 or an exception is thrown.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the FdoInt16 value.
    /// 
    FDO_API virtual FdoInt16 GetInt16(FdoInt32 index) = 0;

    /// \brief
    /// Gets the 32-bit integer value of the property specified at 
    /// the index position. No conversion is performed, thus the 
    /// property must be FdoDataType_Int32 or an exception is thrown.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the FdoInt32 value
    /// 
    FDO_API virtual FdoInt32 GetInt32(FdoInt32 index) = 0;

    /// \brief
    /// Gets the 64-bit integer value of the property specified at 
    /// the index position. No conversion is performed, thus the 
    /// property must be FdoDataType_Int64 or an exception is thrown.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the FdoInt64 value.
    /// 
    FDO_API virtual FdoInt64 GetInt64(FdoInt32 index) = 0;

    /// \brief
    /// Gets the Single floating point value of the property specified at 
    /// the index position. No conversion is performed, thus the property 
    /// must be FdoDataType_Single or an exception is thrown.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the single value
    /// 
    FDO_API virtual FdoFloat GetSingle(FdoInt32 index) = 0;

    /// \brief
    /// Gets the string value of the property specified at the index
    /// position. No conversion is performed, thus the property must
    /// be FdoDataType_String or an exception is thrown.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the string value
    /// 
    FDO_API virtual FdoString* GetString(FdoInt32 index) = 0;

    /// \brief
    /// Gets a LOBValue reference to the property specified at the index
    /// position. The LOB is fully read in and data available.
    /// Because no conversion is performed, the property must be 
    /// FdoDataType_BLOB or FdoDataType_CLOB etc. (a LOB type)
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the reference to LOBValue
    /// 
    FDO_API virtual FdoLOBValue* GetLOB(FdoInt32 index) = 0;

    /// \brief
    /// Gets a reference to the specified LOB property, specified at the index
    /// position. The reference is returned as an FdoBLOBStreamReader or an 
    /// FdoCLOBStreamReader, to allow reading in blocks of data. Because 
    /// no conversion is performed, the property must be FdoDataType_BLOB 
    /// or FdoDataType_CLOB etc. (a LOB type) Cast the FdoIStreamReader 
    /// to the appropiate LOB Stream Reader.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns a reference to a LOB stream reader
    /// 
    FDO_API virtual FdoIStreamReader* GetLOBStreamReader(FdoInt32 index) = 0;

    /// \brief
    /// Gets the geometry value of the property, at the specified index, as  
    /// a byte array in FGF format. Because no conversion is performed, the 
    /// property must be of Geometric type; otherwise, an exception is thrown.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the byte array in FGF format.
    /// 
    FDO_API virtual FdoByteArray* GetGeometry(FdoInt32 index) = 0;

    /// \brief
    /// Gets the raster object of the property at the specified index.
    /// Because no conversion is performed, the property must be
    /// of Raster type; otherwise, an exception is thrown.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the raster object.
    /// 
    FDO_API virtual FdoIRaster* GetRaster(FdoInt32 index) = 0;
};

Interface IFeatureReader

/// \brief
/// The FdoIFeatureReader interface provides a forward-only, read-only iterator
/// for reading feature data.  A reference to an FdoIFeatureReader is returned
/// from the Select command. Because the initial position of the FdoIFeatureReader 
/// is prior to the first item, you must call ReadNext to begin accessing any data.
class FdoIFeatureReader: public FdoIReader
{
public:
...
...
...
    /// \brief
    /// Gets the geometry value of the property, at the specified index, 
    /// as a byte array in FGF format. Because no conversion is performed, 
    /// the property must be of Geometric type; otherwise, an exception is thrown. 
    /// This method is a language-specific performance optimization that returns a
    /// pointer to the array data, rather than to an object that encapsulates
    /// the array.  The array's memory area is only guaranteed to be valid
    /// until a call to ReadNext() or Close(), or the disposal of this reader
    /// object.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// \param count 
    /// Output the number of bytes in the array.
    /// 
    /// \return
    /// Returns a pointer to the byte array in FGF format.
    /// 
    FDO_API virtual const FdoByte * GetGeometry(FdoInt32 index, FdoInt32* count) = 0;

    /// \brief
    /// Gets a reference to an FdoIFeatureReader to read the data contained in
    /// the object or object collection property defined at the specified index 
    /// position. If the property is not an object property, an exception is thrown.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the nested feature reader
    /// 
    FDO_API virtual FdoIFeatureReader* GetFeatureObject(FdoInt32 index) = 0;
};

Interface IDataReader

/// \brief
/// The FdoIDataReader interface provides a forward-only, read-only
/// iterator for reading relational table data. A reference to an
/// FdoIDataReader is returned from the SQLCommands ExecuteReader method.
/// The initial position of the FdoIDataReader interface is prior to the first item.
/// Thus, you must call ReadNext to begin accessing any data.
class FdoIDataReader: public FdoIReader
{
public:
...
...
...
    /// \brief
    /// Gets the data type of the property at the specified index position.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the type of the property.
    /// 
    FDO_API virtual FdoDataType GetDataType(FdoInt32 index) = 0;

    /// \brief
    /// Gets the FDO property type of the property at the given index. This is used
    /// to indicate if a given property is a geometric property or a data property. 
    /// If the property is a FdoPropertyType_DataProperty, then GetDataType 
    /// can be used to to find the data type of the property.
    /// 
    /// \param index 
    /// Input the index of the property.
    /// 
    /// \return
    /// Returns the FDO property type.
    /// 
    FDO_API virtual FdoPropertyType GetPropertyType(FdoInt32 index) = 0;
};

Interface ISQLDataReader

/// \brief
/// The FdoISQLDataReader interface provides a forward-only, read-only
/// iterator for reading relational table data. A reference to an
/// FdoISQLDataReader is returned from the SQLCommands ExecuteReader method.
/// The initial position of the FdoISQLDataReader interface is prior to the first item.
/// Thus, you must call ReadNext to begin accessing any data.
class FdoISQLDataReader: public FdoIDisposable
{
public:
...
...
...
    /// \brief
    /// Returns true if the value of the specified column is null.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns true if the value is null.
    /// 
    FDO_API virtual FdoBoolean IsNull(FdoInt32 index) = 0;

    /// \brief
    /// Gets the data type of the column at the specified index.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the type of the column.
    /// 
    FDO_API virtual FdoDataType GetColumnType(FdoInt32 index) = 0;

    /// \brief
    /// Gets the FDO property type of the column at the specified index. This is used
    /// to indicate if a given column is a geometric property or a data property. If the column is
    /// a FdoPropertyType_DataProperty, then GetColumnType can be used to find the data type of the column.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the FDO property type of the column.
    /// 
    FDO_API virtual FdoPropertyType GetPropertyType(FdoInt32 index) = 0;

    /// \brief
    /// Gets the Boolean value of the specified column. No conversion is
    /// performed, thus the column must be FdoDataType_Boolean or an
    /// exception is thrown.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the Boolean value
    /// 
    FDO_API virtual FdoBoolean GetBoolean(FdoInt32 index) = 0;

    /// \brief
    /// Gets the byte value of the specified column. No conversion is
    /// performed, thus the column must be FdoDataType_Byte or an
    /// exception is thrown.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the byte value.
    /// 
    FDO_API virtual FdoByte GetByte(FdoInt32 index) = 0;

    /// \brief
    /// Gets the date time value of the specified column. No conversion
    /// is performed, thus the column must be FdoDataType_DateTime or
    /// an exception is thrown.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the date and time value.
    /// 
    FDO_API virtual FdoDateTime GetDateTime(FdoInt32 index) = 0;

    /// \brief
    /// Gets the double-precision floating point value of the specified column.
    /// No conversion is performed, thus the column must be of type
    /// Double or an exception is thrown.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the double value.
    /// 
    FDO_API virtual FdoDouble GetDouble(FdoInt32 index) = 0;

    /// \brief
    /// Gets the signed 16-bit integer value of the specified column. No conversion is
    /// performed, thus the column must be FdoDataType_Int16 or an
    /// exception is thrown.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the FdoInt16 value.
    /// 
    FDO_API virtual FdoInt16 GetInt16(FdoInt32 index) = 0;

    /// \brief
    /// Gets the signed 32-bit integer value of the specified column. No conversion is
    /// performed, thus the column must be FdoDataType_Int32 or an
    /// exception is thrown.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the FdoInt32 value.
    /// 
    FDO_API virtual FdoInt32 GetInt32(FdoInt32 index) = 0;

    /// \brief
    /// Gets the signed 64-bit integer value of the specified column. No conversion
    /// is performed, thus the column must be FdoDataType_Int64 or an
    /// exception is thrown.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the FdoInt64 value.
    /// 
    FDO_API virtual FdoInt64 GetInt64(FdoInt32 index) = 0;

    /// \brief
    /// Gets the single-precision floating point value of the specified column.
    /// No conversion is performed, thus the column must be FdoDataType_Single
    /// or an exception is thrown.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the single value
    /// 
    FDO_API virtual FdoFloat GetSingle(FdoInt32 index) = 0;

    /// \brief
    /// Gets the string value of the specified column. No conversion is
    /// performed, thus the column must be FdoDataType_String or an
    /// exception is thrown.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the string value.
    /// 
    FDO_API virtual FdoString* GetString(FdoInt32 index) = 0;

    /// \brief
    /// Gets a LOBValue reference. The LOB is fully read in and data available.
    /// Because no conversion is performed, the property must be FdoDataType_BLOB or
    /// FdoDataType_CLOB etc. (a LOB type)
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the reference to LOBValue
    /// 
    FDO_API virtual FdoLOBValue* GetLOB(FdoInt32 index) = 0;

    /// \brief
    /// Gets a reference of the specified LOB property as a FdoBLOBStreamReader or
    /// FdoCLOBStreamReader etc. to allow reading in blocks of data. Because 
    /// no conversion is performed, the property must be FdoDataType_BLOB 
    /// or FdoDataType_CLOB etc. (a LOB type) Cast the FdoIStreamReader 
    /// to the appropiate LOB Stream Reader.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns a reference to a LOB stream reader
    /// 
    FDO_API virtual FdoIStreamReader* GetLOBStreamReader(FdoInt32 index) = 0;

    /// \brief
    /// Gets the geometry value of the specified column as a byte array
    /// in FGF format. No conversion is performed, thus the column
    /// must be of Geometric type or an exception is thrown.
    /// 
    /// \param index 
    /// Input the position of the column.   
    /// 
    /// \return
    /// Returns the FGF byte array value.
    /// 
    FDO_API virtual FdoByteArray* GetGeometry(FdoInt32 index) = 0;
};

Default implementation

A high level class diagram of existing reader related interfaces in FDO is as illustrated below. Both FdoISqlReader and FdoIDataReader have the function to get the column name or property name according to the index. With this capability, it should be pretty easy to provide default implementation for all access by index functions. Take GetInt32() as an example:

FdoInt32 GetInt32(FdoInt32 index)
{
    FdoStringP propName = GetPropertyName(index);
    return GetInt32(propName);
}

As for FdoIFeatureReader, there is no such function of FdoString* GetPropertyName(FdoInt32 index). The default implementation will use GetClassDefinition() to enumerate the property names. The index order is the same as the collection of properties returned by the class definition.

Managed FDO API

The FDO Managed Interfaces will be updated in a similar manner to reflect the proposed changes.

Provider Implementation

Providers will be required to implement the above functionality. Instead of each provider to implement all of newly added pure virtual functions, three utility abstract classes are added to provide the default implementation as mentioned above under Fdo/Utilities/Common: FdoDefaultFeatureReader, FdoDefaultDataReader, FdoDefaultSqlReader. Then the provider specific implementation will simply derive from the new classes. That involves changing one line in header file for each provider (some providers may not link FdoCommon and need to add that dependancy). Take FdoDefaultDataReader as an example:

class FdoDefaultDataReader: public FdoIDataReader
{
public:
    virtual bool FdoIReader::GetBoolean(FdoString* propertyName) = 0;
    virtual FdoByte FdoIReader::GetByte(FdoString* propertyName) = 0;
    virtual FdoDateTime FdoIReader::GetDateTime(FdoString* propertyName) = 0;
    virtual double FdoIReader::GetDouble(FdoString* propertyName) = 0;
    virtual FdoInt16 FdoIReader::GetInt16(FdoString* propertyName) = 0;
    virtual FdoInt32 FdoIReader::GetInt32(FdoString* propertyName) = 0;
    virtual FdoInt64 FdoIReader::GetInt64(FdoString* propertyName) = 0;
    virtual float FdoIReader::GetSingle(FdoString* propertyName) = 0;
    virtual FdoString* FdoIReader::GetString(FdoString* propertyName) = 0;
    virtual FdoLOBValue* FdoIReader::GetLOB(FdoString* propertyName) = 0;
    virtual FdoIStreamReader* FdoIReader::GetLOBStreamReader(const wchar_t* propertyName ) = 0;
    virtual bool FdoIReader::IsNull(FdoString* propertyName) = 0;
    virtual FdoByteArray* FdoIReader::GetGeometry(FdoString* propertyName) = 0;
    virtual FdoIRaster* FdoIReader::GetRaster(FdoString* propertyName) = 0;
    virtual FdoDataType FdoIDataReader::GetDataType(FdoString* propertyName) = 0;
    virtual FdoPropertyType FdoIDataReader::GetPropertyType(FdoString* propertyName) = 0;

    virtual FdoBoolean IsNull(FdoInt32 index);
    virtual FdoBoolean GetBoolean(FdoInt32 index);
    virtual FdoByte GetByte(FdoInt32 index);
    virtual FdoDateTime GetDateTime(FdoInt32 index);
    virtual FdoDouble GetDouble(FdoInt32 index);
    virtual FdoInt16 GetInt16(FdoInt32 index);
    virtual FdoInt32 GetInt32(FdoInt32 index);
    virtual FdoInt64 GetInt64(FdoInt32 index);
    virtual FdoFloat GetSingle(FdoInt32 index);
    virtual FdoString* GetString(FdoInt32 index); 
    virtual FdoLOBValue* GetLOB(FdoInt32 index);
    virtual FdoIStreamReader* GetLOBStreamReader(FdoInt32 index);
    virtual FdoByteArray* GetGeometry(FdoInt32 index);
    virtual FdoIRaster* GetRaster(FdoInt32 index); 
    virtual FdoString* GetPropertyName(FdoInt32 index);
    virtual const FdoByte * GetGeometry(FdoInt32 index, FdoInt32* count);
    virtual FdoByteArray* GetGeometry(FdoInt32* index);
};

Note: it needs to expose access by property name functions as well, otherwise they will be hiden by access by index functions that can't be visible to derived classes. All pure virtual functions for access by index are overrided with the default implementation in a similar way. e.g.

FdoBoolean FdoDefaultDataReader::IsNull(FdoInt32 index)
{
    FdoStringP propName = GetPropertyName(index);
    Return IsNull(propName);
}

Changes to provider specific implementation:

Class XYZDataReader: public FdoDefaultDataReader
{
    // no other changes 
}

It is expected that resourcing will be supplied to modify all the existing FDO Open Source providers. Unless there are noticeable performance complaints, all of the open source providers will use the default implementation for this time.

Test Plan

Existing FDO Core and Provider level unit tests will be expanded to test the proposed enhancements defined above.

Funding/Resources

Autodesk to provide resources / funding