FDO TextReader Enhancements

Overview

The FDO API provides a number of classes to support streamed I/O to files and in-memory buffers. Clients can add derived classes to support additional devices. A number of Reader and Writer classes are provided to read and write formatted data from and to these devices. One of these classes is the FdoIoTextReader, which reads data from a stream in text format.

The FdoIoTextReader class is incomplete. Streaming support was added to FDO to satisfy the requirement to read and write XML documents. Although XML documents tend to be in text format, FdoIoTextReader was not actually needed to read XML; the FdoXmlReader uses Xerces instead. Therefore, FdoIoTextReader has constructor and destructor but no actual text reading functions.

The FdoIoTextReader should be completed for the following reasons:

• It rounds out the API. The converse class (FdoIoTextWriter) is fully implement so finishing FdoIoTextReader would make the API more consistent
  
• Developers using FDO 3.2 could have used FdoIoTextReader to convert stream contents to a string. Instead, they had to write workaround code since this class was not complete
  
• The API changes can be made in a way that will provide performance benefits for writing XML documents. This will be seen later in this document when the API changes are discussed
  

API Changes

FdoIoTextReader

The text reader can be enhanced by adding functions to read formatted (delimited) text.

class FdoIoTextReader : public FdoIDisposable
{
public:

	FDO_API_COMMON FdoBoolean ReadLine(FdoStringP& outString);

	FDO_API_COMMON FdoBoolean Read(FdoStringP& outString); 

	FDO_API_COMMON FdoBoolean Read( 
		FdoStringP& outString,
		FdoString* leftDelimiters
	);

	FDO_API_COMMON FdoBoolean Read( 
		FdoStringP& outString,
		FdoString* leftDelimiters, 
		FdoString* rightDelimiters 
	);

	FDO_API_COMMON FdoBoolean Read( 
		FdoStringP& outString,
		FdoString* leftDelimiters, 
		FdoString* rightDelimiters, 
		FdoString* skipChars
	);

	FDO_API_COMMON FdoBoolean Read( 
		FdoStringP& outString,
		FdoString* leftDelimiters, 
		FdoString* rightDelimiters, 
		FdoString* skipChars,
		FdoString* separators
	);
};

Parameters:

outString: the string that was read

leftDelimiters: list of left delimiter characters; defaults to empty string (L"").

rightDelimiters: list of right delimiter characters; defaults to empty string (L""). Position is important: 1st left delimiter corresponds to 1st right delimiter and so on. If a right delimiter is not explicitly specified for a particular left delimiter then the right delimiter is the same as the left.

skipChars: list of characters that can be skipped while looking for the left delimiter; defaults to blank character plus end-of line (L" \n").

separators: list of separator characters. These are similar to skip characters except that a separator can only occur once between each string that is read; defaults to empty string (L"").

Description:

ReadLine() Reads the current line (from the current position to the next end-of-line character or end-of-stream).

Read() provides the ability to read delimited strings.

Reading is done according to the following steps:

• starting at the current position, any characters in the skipChars set are skipped.
  
• if the next character is a left delimiter, outString becomes all characters between but not including the left and corresponding right delimiter
  
• otherwise. outString becomes all characters from the current character, up to but not including the next skipChar or separator (or end-of-stream, whichever comes first).
  
• Any subsequent skipChars or separators are skipped. However, Read() stops if a second separator character is encountered. In this case, the 2nd separator is not skipped but the current position is set to it.
  

Returns:

ReadLine() returns true if a line was read and false if at the end of the underlying stream.

Read() returns true if a string was read and false otherwise. false is returned if the end of the stream is reached before a string can be extracted.

It is possible for both of the above to return true with outString being an empty string. For ReadLine(), this happens when the character at the current position is the end-of-line character. For Read() this can happen when there are no characters between the left and right delimeters.

Exceptions:

Read() throws an FdoException if a left delimiter is encountered but end-of-stream is reached before the right delimiter is found.

Example 1:

if the data at and after the input stream’s current position looks like this:

" {'T – true', 'F – false'} {'red' 'green', 'blue'}"

then the following table shows the return value and string that is read. Current position on return from Read() is also shown by the remainder which shows the part of the stream at and after the new current position:

call	returns	outString	remainder
Read(outString, "", "", " {'" )	true	"T"	" – true’, ‘F – false’} {‘red’ ‘green’, ‘blue’}"
Read(outString)	true	"{'T"	" – true’, ‘F – false’} {‘red’ ‘green’, ‘blue’}"
Read(outString, "'", "", "{ " )	true	"T – true"	", ‘F – false’} {‘red’ ‘green’, ‘blue’}"
Read(outString, "'", "", "{" )	false(*)	n/a	" {'T – true', 'F – false'} {'red' 'green', 'blue'}"
Read(outString,"", "", "" )	true	" {'T – true', 'F – false'} {'red' 'green', 'blue'}"	"" (next call to Read would return false)
Read(outString,"'{","'}"," "}	true	"'T – true', 'F – false'" ()	" {'red' 'green', 'blue'}"

(*) - " " is not a skip character so left delimiter is not reached
() – the 2nd ("{"} leftDelimiter is encountered before "'", so read is done until the 2nd right delimiter ("}") is reached

Example 2:

If a stream contains comma separated values where:

• values are separated by commas and any number of spaces
  
• values with embedded separators are enclosed in either single or double quotes
  
• other values are optionally delimited
  

e.g:

"abc def", , 123.45, 'ghi', jkl, "", 9876

then each value can be read by a call to:

Read(outString, "'\"", "", " ",",")

multiple calls to the above Read() on the above stream will read the following for each call:

"abc def"
""
"123.45"
"ghi"
"jkl"
""
"9876"

Example 3:

ReadLine is actually just a convenience function. It is equivalent to:

Read(outString, "", "", "", "\n" )