MapGuide RFC 86 - The Pinned FDO Connection

This page contains a change request (RFC) for the MapGuide Open Source project. More MapGuide RFCs can be found on the RFCs page.

Status

RFC Template Version	(1.0)
Submission Date	Sept. 24, 2009
Last Modified	(Leaf Li) Timestamp
Author	(Leaf Li)
RFC Status	Read for review
Implementation Status	(pending)
Proposed Milestone	(2.2)
Assigned PSC guide(s)	(Tom Fukushima)
Voting History	(vote date)
+1
+0
-0
-1
no vote

Overview

This proposal is to extend MapGuide Feature Service so that calls to Feature Service can use the same FDO connection to avoid some unexpected errors. The solution is to add two methods PinConnection(MgResourceIdentifier featureSourceId) and UnpinConnection(MgResourceIdentifier featureSourceId) so that users are able to pin/unpin an FDO connection to the current user session.

Motivation

Currently MapGuide Server maintains an FDO connection pool in order to improve performance. Once MapGuide Server receives a request to access data in a feature source, it will try to find an available FDO connection in FDO connection pool. It avoids exhausting FDO connections to the underlying RDBMS and improves performance of MapGuide Server when many users use MapGuide Server in the same time. However, it means that one user session may use the different FDO connection for the same feature source at the different time. Sometimes, it will result in some unexpected errors. Let’s take a look at one example.

    featureService.ExecuteSqlNonQuery(featureSourceId, "SAVEPOINT X", null, null);
    try
    {
        MgSqlDataReader reader1 = featureService.ExecuteSqlNonQuery(featureSourceId, "SELECT FID FROM DUMMY", null, null);
        featureService.ExecuteSqlNonQuery(featureSourceId, "SAVEPOINT Y", null, null);
        try
        {
            featureService.ExecuteSqlNonQuery(featureSourceId, "INSERT INTO TEST2 (FID) VALUES (1)", null, null);
        }
        catch (System.Exception)
        {
            featureService.ExecuteSqlNonQuery(featureSourceId, "ROLLBACK TO Y", null, null);
        }

        while (reader1.ReadNext())
        {
            MgParameterCollection parameters = new MgParameterCollection();
            MgInt32Property inFid = new MgInt32Property("myInput", reader1.GetInt32("FID"));
            MgParameter inParameter = new MgParameter(inFid, MgParameterDirection.Input);
            parameters.Add(inParameter);
            featureService.ExecuteSqlNonQuery(featureSourceId, "INSERT INTO TEST(FID) VALUES(:myInput)", parameters, null);
        }
        reader1.Close();
        reader1.Dispose();
    }
    catch (System.Exception)
    {
        featureService.ExecuteSqlNonQuery(featureSourceId, "ROLLBACK TO X", null, null);
    }
    featureService.ExecuteSqlNonQuery(featureSourceId, "COMMIT", null, null);

In the example above, it is possible that FDO connections used by executing SQL statements "SAVEPOINT X", "SAVEPOINT Y", "ROLLBACK TO X", "ROLLBACK TO Y" and "COMMIT" are completely different FDO connections. If so, it will result in an error when executing SQL statement "ROLLBACK TO X", "ROLLBACK TO Y" or "COMMIT". If it isn't a nested transaction, it can be resolved by passing an MgTransaction instance when executing SQL statements because MapGuide Server will guarantee that operations in the same transaction use the same FDO connection. However, it still has the following two issues.

• There are only three methods supporting transaction, MgFeatureService::UpdateFeatures, MgFeatureService::ExecuteSqlQuery and MgFeatureService::ExecuteSqlNoQuery.
• MgFeatureService::BeginTransaction will do something on the DRBMS serverwhen opening a transaction. This has consequences such as updates are only visible in this transaction, and data gets locked.

Another issue is that it may result in a whole bunch of connections are open when using MgFeatureReader, MgDataReader or MgSqlDataReader. Each reader will use an FDO connection exclusively. So in the example below, it will have all of a sudden a whole bunch of admin connections open when you open a connection with admin rights.

    MgSqlDataReader reader1 = featureService.ExecuteSqlQuery(featureSourceId, commandText, mgParameters, null);
    while (reader1.ReadNext())
    {
        MgSqlDataReader reader2 = featureService.ExecuteSqlQuery(featureSourceId, commandText, mgParameters, null);
        while (reader2.ReadNext())
        {
            ......
        }
        reader2.Close();
        reader2.Dispose();
    }
    reader1.Close();
    reader1.Dispose();

So we need a mechanism to pin an FDO connection to one user session. Then MapGuide always uses this pinned FDO connection when executing any FDO commands for the same feature source in this session. Moreover, we need provide a mechanism to unpin the FDO connection which is pinned to one user session so that FDO connection is available for other user sessions after the user doesn’t need to pin the FDO connection to his session.

Proposed Solution

In this RFC, we extend Feature Service to add the ability to use the same FDO connection for calls to Feature Service. The following two methods will be added to class MgFeatureService to allow users to pin/unpin an FDO connection to the current session. Method PinConnection(...) will select an FDO connection from FDO connection pool for the specified feature source if there is one FDO connection available. Otherwise, create a new FDO connection and add it to FDO connection pool. Then this FDO connection is pinned to current session. It means this FDO connection becomes an exclusive connection and can’t be used by other sessions. Since then, any call to Feature Service for the feature source and the session, which needs an FDO connection, will always use the pinned FDO connection till users call method UnpinConnection(...) to unpin this FDO connection or current session or the pinned connection is time-out. By default, one session will be time-out after 20 minutes. Users can modify this setting through property [SiteServiceProperties]->SessionTimeOut in MapGuide Server configuration file "serverconfig.ini".

    /// \brief
    /// Pin the FDO connection to current session for a feature source. 
    /// 
    /// \param featureSourceId (MgResourceIdentifier)
    /// A resource identifier referring to a feature source.
    ///
    /// \return
    /// Returns nothing.
    ///    
    void PinConnection(MgResourceIdentifier featureSourceId) = 0;

    /// \brief
    /// Unpin the FDO connection pinned to current session for the specified feature
    /// source. 
    /// 
    /// \param featureSourceId (MgResourceIdentifier)
    /// A resource identifier referring to a feature source.
    ///
    /// \return
    /// Returns nothing.
    ///
    void UnpinConnection(MgResourceIdentifier featureSourceId) = 0;

Some of methods in MgFeatureService don’t depend on an existing feature source and an FDO connection. So this change will not affect them. For example, MgFeatureService::GetFeatureProviders, MgFeatureService::XmlToSchema, and so on.

To avoid exhausting the avaiable FDO connections through pinned connections, properties 'PinnedDataConnectionSize' and 'PinnedDataConnectionSizeCustom' will be added into serverconfig.ini to set a limitation of pinned connections per FDO providers. Moreover, users can set time duration in seconds for when an idle pinned FDO connection is dropped.

    [FeatureServiceProperties]
    # *****************************************************************************
    # F E A T U R E  S E R V I C E
    #
    # Property Name                      Description
    # -----------------------------------------------------------------------------
    # PinnedDataConnectionSize           Default # of pinned FDO connections per provider. 
    #                                       Value < 0  The default number of pinned connection is 
    #                                                  same as value of DataConnectionPoolSize.
    #                                       Value = 0  Can not use pinned FDO connection.
    #                                       Value > 0  Its value has to be less than or equal to
    #                                                  the value of DataConnectionPoolSize.
    # PinnedDataConnectionSizeCustom     Custom # of pinned FDO connections for specified provider
    #                                       0 <= Length <= 1024
    #                                       Example: OSGeo.SDF:5,OSGeo.SHP:5
    # PinnedDataConnectionTimeout        Time duration in seconds for when an idle pinned FDO 
    #                                    connection is dropped.
    #                                       0 < Value <= 1800 
    # PinnedDataConnectionTimerInterval  Time interval in seconds for when the server 
    #                                    checks for idle pinned FDO connection. 
    #                                       0 < Value <= 1800 
    # *****************************************************************************
    PinnedDataConnectionSize           = 20
    PinnedDataConnectionSizeCustom     = 
    PinnedDataConnectionTimeout        = 360
    PinnedDataConnectionTimerInterval  = 60

Implications

This is new API only. There will be no effect on existing applications. However, API documentation needs to be updated.

Test Plan

Add unit tests for those new methods into the existing Feature Service unit tests.

Funding/Resources

Supplied by Autodesk.