MapGuide RFC 12 - Unmanaged Data API Changes

This page contains an change request (RFC) for the MapGuide Open Source project. More MapGuide RFCs can be found on the RFCs page.

Status

RFC Template Version	1.0
Submission Date	15 January 2007
Last Modified	jasonbirch Timestamp
Author	Tony Fang
RFC Status	adopted
Implementation Status	completed
Proposed Milestone	1.2
Assigned PSC guide(s)	Tom Fukushima
Voting History	Jan 24, 2007
+1	Tom, Bruce, Jason, Paul, Haris, Andy, Bob
+0
-0
-1

Overview

Changes to the Unmanaged Data API specified in ​http://wiki.osgeo.org/index.php/MapGuide_RFC_2_-_Unmanaged_Data_APIs.

Also changes to the Server Admin PHP pages allowing editing of the serverconfig.ini are specified in more detail.

Motivation

These changes will:

• allow scaling to sites containing many unmanaged data resource files.
• disambiguate the response.
• treat the alias component as a folder grammatically.
• simplify the API implementation.
• eliminate the need for the redundant EnumerateUnmanagedDataMappings API method.
• give the client better control of desired query results.

Proposed Solution

These are the proposed changes to the Unmanaged Data APIs.

• Remove EnumerateUnmanagedDataMappings method.
• Modify EnumerateUnmanagedData API method as specified in the Technical section.
  • modify arguments to path, recursive, select, and filter
  • path="", recursive=false, select="FOLDERS" yields the same result as EnumerateUnmanagedDataMappings
  • All file types are returnable with a client-customizable filter
  • Return an UnmanagedDataList (see UnmanagedDataList-1.0.0.xsd below) instead of a StringCollection.
• Windows hidden folders and files are returned (already works this way after RFC2 implementation).
• All other aspects of this API method remain the same as before.
  • Unicode mappings and filenames are supported.
  • Square brackets are not allowed for mapping names.
  • Spaces are allowed in mapping names.
  • Mappings can be physical drives or network paths.

These additions to the Server Admin PHP pages will allow editing of the UnmanagedDataMappings section in the serverconfig.ini.

• Below "Configure WFS", add an entry called "Configure Unmanaged Data".
• "Configure Unmanaged Data" will display a table with 3 columns:
  • radio button denoting current selection
  • Mapping Name
  • Location
• The table will also have 3 buttons:
  • "Add Mapping" will display a page which allows you to add a new mapping.
  • "Edit Mapping" will display a page which allows you to edit the currently selected mapping location.
  • "Delete Mapping" will delete the currently selected mapping.
• This is similar to the "Manage Users" table, except with only 3 columns.
• The server must be re-started before any Unmanaged Data serverconfig.ini changes will take effect.

Technical

virtual MgByteReader * MgResourceService::EnumerateUnmanagedData(CREFSTRING path, bool recursive, CREFSTRING select, CREFSTRING filter);

Enumerates the available unmanaged data from the server.

Parameters:

path Specifies an unmanaged data mapping with optional subfolder path. It must begin with a legitimate mapping. e.g. "[alias1]" or "[alias1]subfolder1/subfolder2". It cannot begin with a drive or folder path. Default value is "", which corresponds to the "Unmanaged Data Root". At the top level of the "Unmanaged Data Root", you can only see the unmanaged data mappings. No data files can be seen in the top level of the "Unmanaged Data Root".

recursive If it is true, it will recurse into all subfolders looking for files and/or subfolders. If it is false, it will only seek files and/or subfolders at the current level. Default value is false.

select Specifies the type of data to return. The choices are "FILES", "FOLDERS", or "BOTH". Default value is "FOLDERS".

filter Specifies an optional file extension filter on files returned. Use a ";" to delimit multiple file extension filters. e.g. "shp", "shp;shx;prj", or "jpg;jpeg;jpe;bmp;png". The file filter is case-insensitive. If select="FOLDERS", then filter is ignored. Default value is "", which returns all files.

Returns:

Returns an MgByteReader object containing the list of all unmanaged data files and/or folders. The list is in XML format using the UnmanagedDataList schema.

The new schema will be named UnmanagedDataList-1.0.0.xsd. This is closely analogous to ResourceList-1.0.0.xsd which specifies the response of EnumerateResources.

  <xs:element name="UnmanagedDataList">
    <xs:annotation>
      <xs:documentation>Unmanaged data list</xs:documentation>
    </xs:annotation>
    <xs:complexType>
      <xs:choice minOccurs="0" maxOccurs="unbounded">
        <xs:element name="UnmanagedDataFolder">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="UnmanagedDataId" type="xs:string"/>
              <xs:element name="CreatedDate" type="xs:dateTime"/> (MAYBE)
              <xs:element name="ModifiedDate" type="xs:dateTime"/> (MAYBE)
              <xs:element name="NumberOfFolders" type="xs:integer"/>
              <xs:element name="NumberOfFiles" type="xs:integer"/>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element name="UnmanagedDataFile">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="UnmanagedDataId" type="xs:string"/>
              <xs:element name="CreatedDate" type="xs:dateTime"/> (MAYBE)
              <xs:element name="ModifiedDate" type="xs:dateTime"/> (MAYBE)
              <xs:element name="Size" type="xs:integer"/> (MAYBE)
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:choice>
    </xs:complexType>
  </xs:element>

API Examples

Windows Server

serverconfig.ini contains:

[UnmanagedDataMappings]
alias1 = c:\mydata\sdf
alias2 = \\other_machine\public_data

The server machine contains the following files:

• c:\mydata\sdf\sewers.sdf
• c:\mydata\sdf\buildings.sdf
• c:\mydata\sdf\canada\bc.sdf
• c:\mydata\sdf\canada\ab.sdf
• c:\mydata\sdf\canada\sk.sdf
• 
  other_machine\public_data\readme.txt
• 
  other_machine\public_data\raster\world.bmp
• 
  other_machine\public_data\raster\africa.tiff
• 
  other_machine\public_data\raster\usa\usa.jpeg
• 
  other_machine\public_data\shp\squares.shp
• 
  other_machine\public_data\shp\circles.shp

EnumerateUnmanagedData API Examples on Windows server

path	recursive	select	filter	FOLDERS RESULT	FILES RESULT	Notes
""	false	"FOLDERS"	""	[alias1] [alias2]	Equivalent to removed EnumerateUnmanagedDataMappings. This is the "Unmanaged Data Root."
""	true	"FOLDERS"	""	[alias1] [alias1]canada/ [alias2] [alias2]raster/ [alias2]raster/usa/	Note the forward slash.
""	false	"FILES"	""	No files returned.
""	true	"FILES"	""	[alias1]sewers.sdf [alias1]buildings.sdf [alias1]canada/bc.sdf [alias1]canada/ab.sdf [alias1]canada/sk.sdf [alias2]readme.txt [alias2]raster/world.bmp [alias2]raster/africa.tiff [alias2]raster/usa/usa.jpeg [alias2]shp/squares.shp [alias2]shp/circles.shp		All files returned.
""	true	"BOTH"	""	[alias1] [alias1]canada/ [alias2] [alias2]raster/ [alias2]raster/usa/	[alias1]sewers.sdf [alias1]buildings.sdf [alias1]canada/bc.sdf [alias1]canada/ab.sdf [alias1]canada/sk.sdf [alias2]readme.txt [alias2]raster/world.bmp [alias2]raster/africa.tiff [alias2]raster/usa/usa.jpeg [alias2]shp/squares.shp [alias2]shp/circles.shp	All folders and files returned.
"[alias1]"	false	"FILES"	""	[alias1]sewers.sdf [alias1]buildings.sdf
"[alias2]raster/usa"	true or false	"FILES"	""	[alias2]raster/usa/usa.jpeg		The result is the same whether recursive is true or false.
"[alias2]" or ""	true	"FILES"	"bmp;tiff"	[alias2]raster/world.bmp [alias2]raster/africa.tiff		The result is the same whether path is "[alias2]" or "".

Linux Server

serverconfig.ini contains:

[UnmanagedDataMappings]
alias1 = /user/Thelonious/sdf
alias2 = /mnt/other_machine/public_data

The server machine contains the following files:

• /user/Thelonious/sdf/sewers.sdf
• /user/Thelonious/sdf/Sewers.SDF
• /user/Thelonious/sdf/readme.txt
• /user/Thelonious/sdf/sewers.sdf.bak
• /mnt/other_machine/public_data/readme.txt
• /mnt/other_machine/public_data/raster/world.bmp
• /mnt/other_machine/public_data/raster/africa.tiff
• /mnt/other_machine/public_data/raster/usa/usa.jpeg
• /mnt/other_machine/public_data/shp/squares.shp
• /mnt/other_machine/public_data/shp/circles.shp

EnumerateUnmanagedData API Examples on Linux server

path	recursive	select	filter	FOLDERS RESULT	FILES RESULT	Notes
""	false	"FOLDERS"	""	[alias1] [alias2]		Equivalent to removed EnumerateUnmanagedDataMappings. This is the "Unmanaged Data Root."
""	true	"FOLDERS"	""	[alias1] [alias2] [alias2]raster/ [alias2]raster/usa/		Note the forward slash.
""	false	"FILES"	""			No files returned.
""	true	"FILES"	""		[alias1]sewers.sdf [alias1]Sewers.SDF [alias1]readme.txt [alias1]sewers.sdf.bak [alias2]readme.txt [alias2]raster/world.bmp [alias2]raster/africa.tiff [alias2]raster/usa/usa.jpeg [alias2]shp/squares.shp [alias2]shp/circles.shp	All files returned. Notice sewers.sdf and Sewers.SDF are both returned.
""	true	"BOTH"	""	[alias1] [alias2] [alias2]raster/ [alias2]raster/usa/	[alias1]sewers.sdf [alias1]Sewers.SDF [alias1]readme.txt [alias1]sewers.sdf.bak [alias2]readme.txt [alias2]raster/world.bmp [alias2]raster/africa.tiff [alias2]raster/usa/usa.jpeg [alias2]shp/squares.shp [alias2]shp/circles.shp	All folders and files returned. Notice sewers.sdf and Sewers.SDF are both returned.
"[alias1]"	false	"FILES"	"sdf"		[alias1]sewers.sdf [alias1]Sewers.SDF	Note the mixed case of the sdf extension. Notice sewers.sdf and Sewers.SDF are both returned.
"[alias2]raster/usa"	true or false	"FILES"	""		[alias2]raster/usa/usa.jpeg	The result is the same whether recursive is true or false.
"[alias2]" or ""	true	"FILES"	"bmp;tiff"		[alias2]raster/world.bmp [alias2]raster/africa.tiff	The result is the same whether path is "[alias2]" or "".

Implications

API documentation will need to be updated.

Test Plan

• Update webtier test pages to use the updated API methods.
• Remove test for eliminated EnumerateUnmanagedDataMappings.

Funding/Resources

Autodesk to supply the resources to make the required changes.

Addendum

For a feature source, an alias must be in the form: %MG_DATA_PATH_ALIAS[alias]%

This follows the existing conventions in the server for replacing text tags (e.g. %MG_DATA_FILE_PATH%). The server performance will be much better compared to using only [alias] as it will reduce the amount of potential false-positive hits while performing the alias re-mapping.

Example:

<?xml version="1.0" encoding="utf-8"?>
<FeatureSource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xsi:noNamespaceSchemaLocation="FeatureSource-1.0.0.xsd">
  <Provider>OSGeo.SDF</Provider>
  <Parameter>
    <Name>File</Name>
    <Value>%MG_DATA_PATH_ALIAS[alias1]%LIPOFGUTTER.sdf</Value>
  </Parameter>
  <Parameter>
    <Name>ReadOnly</Name>
    <Value>TRUE</Value>
  </Parameter>
</FeatureSource>

For the EnumerateUnamangedData API method, the "select" parameter should be renamed to "type". This follows the convention used by the other API methods.

virtual !MgByteReader  * MgResourceService::!EnumerateUnmanagedData(CREFSTRING ''path'', bool ''recursive'', CREFSTRING ''type'', CREFSTRING ''filter''); 

Here is the http signature for the EnumerateUnmanagedData API:

OPERATION=ENUMERATEUNMANAGEDDATA
VERSION=1.0.0
PATH=
RECURSIVE=0
TYPE=Folders
FILTER=