Opened 7 years ago
#929 new defect
ISelect.SetFilter corrupts decimal values passed to data source
| Reported by: | beginbenoit | Owned by: | gregboone |
|---|---|---|---|
| Priority: | major | Milestone: | 4.1.0 |
| Component: | FDO API | Version: | 4.0.0 |
| Severity: | 2 | Keywords: | |
| Cc: | External ID: |
Description
When setting the filter on an ISelect command, it appears as though certain decimal values get corrupted. An example of this is the value 65.54 stored in a NUMBER(6,2) field in Oracle or in a DECIMAL(6,2) field in SQL Server. In both of these cases, the actual filter sent by FDO will be 65.54000000000001.
The query obviously returns no results when this occurs. It doesn't seem to be provider specific or platform specific, as the issue was duplicated on Autodesk.Oracle and OSGeo.SQLServerSpatial on both AIMS 2017, Map 3D 2017, FDO Toolbox and MGOS 3.0 and 3.1.
Running the query with no filter will also tend to yield the proper value when going through the C# APIs, but if you hit up Mapguide via the mapagent.fcgi to run the query the invalid floating point value will be in the result XML.
I know floating point memory structures are often not overly reliable, but in this case a user can get confused why he can get to the data he's looking for. I'm not sure if something can be done to improve the precision of the floating points used to make sure we atleast get 4-5 digits of precision.
