| 35 | | Using the metric of the number of conversions per second from UTM27-13 to CO83-C in a pure measurement environment (i.e. no coordinate retrieval or delivery code), the underlying CS-MAP library is capable of producing approximately 1 million conversions per second on an average desktop machine. Changes in the MapGuide API, therefore, cannot get us beyond this limit. Thus, in this RFC, we will write of performance in terms of the percentage of this theoretical maximum which the API can/will deliver. The current implementation of the API delivers performance of approximately 80% of this maximum. Research and test implementations indicate that it is not unreasonable to expect an improvement to 91% of the theoretical maximum when using the most efficient of the Transform function overloads. |
| | 35 | Using the metric of the number of conversions per second from !UTM27-13 to !CO83-C in a pure measurement environment (i.e. no coordinate retrieval or delivery code), the underlying !CS-MAP library is capable of producing approximately 1 million conversions per second on an average desktop machine. Changes in the !MapGuide !API, therefore, cannot get us beyond this limit. Thus, in this !RFC, we will write of performance in terms of the percentage of this theoretical maximum which the !API can/will deliver. The current implementation of the !API delivers performance of approximately 80% of this maximum. Research and test implementations indicate that it is not unreasonable to expect an improvement to 91% of the theoretical maximum when using the most efficient of the Transform function overloads. |
| 43 | | Please note that the referenced CS-MAP RFC includes two phases. The first phase includes only the identification of those transformations which are known to be reentrant and those which are not. The first phase is a relatively simple task which is easily completed with available resources. The second phase includes some, perhaps non-trivial, efforts to make reentrant most, if not all, CS_MAP conversions and transformations which are known to be non-reentrant. Thus, as this work progresses, further performance enhancements will inure to applications without and code changes required. |
| 44 | | |
| 45 | | Further, it is proposed that the MgCoordinateSystemTransform object provide this information to the consuming application, and thus applications will be able to determine if multi-threading a large conversion will provide any benefits. |
| 46 | | The performance enhancement of removing the Critical Section will be enjoyed automatically by all applications using the API without code change. However, the following API will be added to the MgCoordinateSystemTransform object to enable applications to further improve the performance of large coordinate conversions: |
| 47 | | |
| 48 | | {{{ bool MgCoordinateSystemTransform::IsReentrant (); }}} |
| 49 | | |
| 50 | | The function would return true only in the case where all conversions and transformations referenced by the transformation object have been classified as reentrant. |
| | 43 | Please note that the referenced !CS-MAP !RFC includes two phases. The first phase includes only the identification of those transformations which are known to be reentrant and those which are not. The first phase is a relatively simple task which is easily completed with available resources. The second phase includes some, perhaps non-trivial, efforts to make threadsafe most, if not all, !CS_MAP conversions and transformations which are known to be non-threadsafe. Thus, as this work progresses, further performance enhancements will inure to applications without and code changes required. |
| 60 | | The default behavior of the API is to throw an exception whenever such a warning is received from the CS-MAP library. This default behavior can be, and often is, modified at run-time using the IgnoreDatumShiftWarning and IgnoreOutsideDomainWarning members of the MgCoordinateSystemTransform interface. Thus, it is recommended that applications using the API disable the exception throwing behavior of the API. It is further proposed that the MgCoordinateSystemTransform object be enhanced to provide a status accumulation feature. By status accumulation, we refer to the concept of: a)counting all source projective CRS warnings issued, b)counting all datum shift warnings issued, and c)counting all target projective CRS warnings. |
| | 53 | The default behavior of the !API is to throw an exception whenever such a warning is returned by the !CS-MAP library. This default behavior can be, and often is, modified at run-time using the !IgnoreDatumShiftWarning and !IgnoreOutsideDomainWarning members of the !MgCoordinateSystemTransform interface. Thus, it is recommended that applications using the API disable the exception throwing behavior of the API. It is further proposed that the !MgCoordinateSystemTransform object be enhanced to provide a status accumulation feature. By status accumulation, we refer to the concept of: a)counting all source projective CRS warnings issued, b)counting all datum shift warnings issued, and c)counting all target projective CRS warnings. |
| 64 | | For example, a conversion where the target CRS warning count exceeds, say, 20% of the total number of points suggests that the target CRS chosen by the user is incorrect. On the other hand, warning counts which are less than, say, 20% of the total point count suggest a perfectly normal conversion. |
| | 57 | For example, a conversion where the target !CRS warning count exceeds, say, 20% of the total number of points suggests that the target !CRS chosen by the user is inappropriate for the data set being converted. On the other hand, warning counts which are less than, say, 20% of the total point count suggest a normal conversion. |
| 86 | | A batch coordinate conversion capability currently exists in the |
| | 79 | A batch coordinate conversion capability currently exists in the !MgCoordinateSystemTransform object. The performance of this capability is expected to increase due to the refactoring of the Transform code proposed immediately above. However, this function requires that, for example, 3D coordinates are provided in three distinct arrays; specifically the easting/X/Longitude coordinates in one single dimensional array of doubles, the northing/Y/Latitude coordinates in a separate single dimension array of doubles, and a third separate and distinct array of double for the elevation/Z/height coordinate. There are few, if any, applications which maintain or utilize coordinate data in this form. |
| 99 | | === Single Thread Operation === |
| 100 | | |
| 101 | | It is expected that the above changes will improve the performance of the coordinate conversion API without changing its behavior in a multi-threaded environment. It is considered likely that further performance enhancements can be achieved if the MgCoordinateSystemTransform object can assume that it is operating in a single threaded environment. Thus, the following new member of the MgCoordinateSystemTransform interface is also proposed: |
| 102 | | |
| 103 | | {{{ |
| 104 | | bool MgCoordinateSystemTransform::AssumeSingleThread (bool true); |
| 105 | | }}} |
| 106 | | |
| 107 | | This function would inform the MgCoordinateSystemTransform object that the current instance may assume it is operating in a single thread environment, and thus deliver any extra performance it can under that assumption. The member will return the previous state of the AssumeSingleThread flag. |
| 108 | | |
| 109 | | Construction of a new MgCoordinateSystemTransform object will have the AssumeSingleThread flag set to false. Use of the SetSourceAndTarget member function, will cause the AssumeSingle Thread flag to be reset to false. |
| 110 | | |
| 115 | | It would be nice to assume that all current CS-MAP coordinate conversion algorithms can be made reentrant without a serious affect on resources and/or performance, and that all future additions to the CS-MAP library will be implemented in a reentrant manner. However, the ability to have non-reentrant conversion/transformation methods in the CS-MAP library is reserved. Thus, we retain the Critical Section to keep multiple threads from using a non-reentrant conversion or transformation at the same time. Given the implementation of CS-MAP RFC !#5, however, we will only need to actually use it when truly necessary. |
| | 96 | It would be nice to assume that all current CS-MAP coordinate conversion algorithms can be made threadsafe without a serious affect on resources and/or performance, and that all future additions to the !CS-MAP library will be implemented in a threadsafe manner. However, the ability to have a non-threadsafe conversion/transformation method in the !CS-MAP library is reserved. Thus, we retain the Critical Section to keep multiple threads from using a non-threadsafe conversion or transformation at the same time. Given the implementation of !CS-MAP !RFC !#5, however, we will only need to actually use it when truly necessary. |
| 121 | | That is, the status count feature cannot be implemented in an a "separate instance per thread" manner and passed to a reentrant MgCoordinateSystemTransform object. Thus, the data elements in which the status accumulation occurs must be included in the Transform object itself. This leads to the fact that using the same Transform object for the conversion for two distinct datasets (as would be possible if total reentrancy was achievable) will produce the correct numerical results, but all status warnings encountered in the two different datasets would be accumulated in the same data accumulation variables and thus conversion of a dataset which converted without warning be considered a failure due to the failure of the second dataset. |
| 122 | | |
| 123 | | Conversion of a very large dataset, a point cloud for example, can be achieved in a multi-threaded environment using the same MgCoordinateSystemTransform object as the resulting status accumulation will accurately reflect that status of the entire conversion effort. As this is possible and desirable, we propose this as the optimum balance of performance versus functionality. |
| | 102 | Thus, the choice has been made to require that one distinct and separate !MgCoordinateSystemTransform object be created for each thread that needs to use same. |
| 128 | | 1. In the existing code, the behavior of the API with regard to the status of returned results in the event of an exception being thrown is inconsistent. In the proposed code, conversion results will always be provided, even in the event of an exception being thrown. Thus, the proposed behavior will provide consistent return results and also contribute to higher performance levels. That is, even in the event of an exception, all coordinates requested to be converted will have been converted. |
| 129 | | 2. The four status values returned in the m_nTransformStatus member of the MgCoordinateSystemTransform object will be adjusted to form a severity level sequence which rates a geodetic datum “outside range” as more severe than a projected “outside range”. The names used will not change, only the numeric values assigned to them; so this should not require any coding changes. |
| 130 | | 3. The overloads of the MgCoordinateSystemTransform::Transform which deal with arrays will now always complete the conversion of the entire array before throwing any exception with regard to non-normal status encountered in the conversion. Also, these overloads will be modified so that the value of the m_nTransformStatus member, upon return, will always reflect the worst status encountered (per the severity level described in 2 above) in the transformation of the array (as opposed to the status of the last conversion performed as is currently done). |
| 131 | | 4. All overloads of the |
| | 107 | 1. In the existing code, the behavior of the !API with regard to the status of results in the event of an exception being thrown is inconsistent. In the proposed code, conversion results will always be provided, even in the event of an exception being thrown. Thus, the proposed behavior will provide consistent return results and also contribute to higher performance levels. That is, even in the event of an exception, all coordinates requested to be converted will have been converted. |
| | 108 | 2. The four status values returned in the !m_nTransformStatus member of the !MgCoordinateSystemTransform object will be adjusted to form a severity level sequence which rates a geodetic datum “outside range” as more severe than a projected “outside range”. The names used will not change, only the numeric values assigned to them; so this should not require any coding changes. |
| | 109 | 3. The overloads of the !MgCoordinateSystemTransform::Transform which deal with arrays will now always complete the conversion of the entire array before throwing any exception with regard to warning status values encountered in the conversion. Also, these overloads will be modified so that the value of the !m_nTransformStatus member, upon return, will always reflect the worst status encountered (per the severity level described in 2 above) in the transformation of the array (as opposed to the status of the last conversion performed as is currently done). |
| | 110 | 4. All overloads of the !TransformM variety will now always calculate and return the ‘m’ value. Currently, when an exception is thrown, the XYZ coordinate values would be converted, but the ‘m’ value would not always be. |
| 135 | | In the case of projective conversions, the "rational result" is based on the nature of the projection. For many of the projections supported, the "rational result" is simply what the projection mathematics produce, even though the coordinate is known to be outside the region for which the projection's parameters suggest is the useful range of the conversion. In other cases, the projection will have singularity points, such as either pole in the case of the traditional Mercator. In such cases the "rational result" typically includes one or more coordinates with an unmistakably large number which suggests infinity, but will not cause a floating point exception if the value is used for any normal calculation. |
| | 114 | In the case of projective conversions, the "rational result" is based on the nature of the projection. For many of the projections supported, the "rational result" is simply what the projection mathematics produce, even though the coordinate is known to be outside the region for which the projection's parameters suggest is the useful range of the conversion. In other cases, the projection will have singularity points, such as either pole in the case of the traditional !Mercator. In such cases the "rational result" typically includes one or more ordinates with an unmistakably large number which suggests infinity, but will '''not''' cause a floating point exception if the value is used for any normal calculation. |
| 140 | | * Construct for each unique conversion pair in the test data table an MgCoordinateSystemTransform object, copying the pointer as necessary to provide a pointer for each occurrence in the table. |
| 141 | | * A function capable of performing all conversions in the table, in sequence, using the |
| 142 | | * The host test application will create threads causing each individual thread to execute the test conversion function using the exact same test data and, specifically, a pointer to the same Transformation objects. |
| | 119 | * Construct for each unique conversion pair in the test data table a !MgCoordinateSystemTransform object, copying the pointer as necessary to provide a pointer for each occurrence in the table. |
| | 120 | * A function capable of performing all conversions in the table, in sequence, using the !MgCoordinateSystemTransform pointer in each test case, shall be written. |
| | 121 | * The host test application will create threads causing each individual thread to execute the test conversion function using the exact same test data and, specifically, a pointer to the same Transform objects. |
