[mapguide-internals] Generating Xml for "Select Features"

[Kenneth, GEOGRAF A/S]

Ok, I agree that raising the limit would be a bad idea.
The "write to file" solution is better, but still requires the server to 
buffer the entire resultset.
This will both require the entire resultset to be in memory at some 
time, and also delay the initial partial response.

I guess a single row should fit in 16Mb, so perhaps it would be enough 
to override the ProxyFeatureReader::ToXml(),
so it can also take a stream. Would it break the API to have an 
overloaded version, but not expose it?

I would use the stream method within the HttpHandler only, leaving it up 
to someone else to bring it into the other languages.
For the regular ToXml(), a file backed serialization would probably 
improve performance.
Can you give me a pointer to some code that uses temporary files, so 
that I may implement it in a standardized fashion?

Regards, Kenneth, GEOGRAF A/S



Trevor Wekel skrev:
> Hi Kenneth,
>
> ToXml() is marked as EXTERNAL_API in MgReader so changing the signature
> could break some code.  Also, all PUBLISHED_API and EXTERNAL_API within
> MapGuide have to function correctly for all 3 web languages - PHP, .Net,
> and Java.  Implementing API support for native streaming in all three
> languages would be a non-trivial effort.
>
> To get around the 16MB limit, write the string to a file and use new
> MgByteSource("filename", true).  I would only do this is the string is
> too large.
>
> Increasing the 16MB limit on MgByte is probably not a good idea,
> especially for this API.  Selecting an entire feature source (like the
> US roads network) into an in-memory XML blob would be bad.
>
> Thanks,
> Trevor
>
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of
> Kenneth, GEOGRAF A/S
> Sent: Tuesday, June 26, 2007 5:26 AM
> To: MapGuide Internals Mail List
> Subject: [mapguide-internals] Generating Xml for "Select Features"
>
> When using the "Select Features" from the Http API, the result comes 
> back in Xml.
> The call chain is:
>
> ProxyFeatureReader::ToXml()
> MgByteSource::MgByteSource(char* data, int len)
> ByteSourceMemoryImpl::ByteSourceMemoryImpl(char* data, int len)
> MgByte::MgByte(char* data, int len, AllocatorType allocator)
>
> When the size of the Xml to return grows, MgByte throws an exception in 
> line 34:
>     if (len > MgByte::MaxSize && allocator != MgByte::None)
>     {
>         throw new MgArgumentOutOfRangeException(L"MgByte.MgByte", 
> __LINE__, __WFILE__, NULL, L"", NULL);
>     }
>
> This happens because the ByteSourceMemoryImpl does not pass an 
> AllocatorType argument, and thus defaults to MgByte::None.
> A simple solution would be to fix this error in ByteSourceMemoryImpl, 
> but I don't have enough overview to see if this has other implications.
>
> I will happily submit a patch for ByteSourceMemoryImpl, but is that an 
> acceptable solution?
>
> I would prefer if the code was changed, so that 
> ProxyFeatureReader::ToXml() took a stream as input, and wrote to that 
> stream.
> That would also improve response/perfomance time, as the current 
> implementation buffers the entire result, and also uses the std::string 
> + operator, which has a high overhead for large strings.
>
> Is the second solution something that could be accepted as a patch? I 
> so, I will happily submit a patch for it.
>
>