Re: http_url_post_raw encoding Issue

[Scott Klement <sk@xxxxxxxxxxxxxxxx>]

Hi Raj,

There are two levels of encoding needed here:

1) You need to make your XML document into a valid (well-formed) XML 
document by encoding characters that are special to XML.

2) In some circumstances (and I'm guessing yours is one) you have to 
encode your data so that it's valid as an HTTP URL.  (Called "URL Encoding.)

You are doing the 2nd level, but are missing the first.

Let's see how that affects you.  You start out with this:

    <DESC>GENERAL & ACCESSORIES</DESC>

This is _not_ a valid XML document.  Then you URL encode it, and you end 
up with this:

instring=%3cDESC%3eGENERAL%20%26%20ACCESSORIES%3c/DESC%3e

That's perfectly valid as HTML form data, and can be legitimately put 
into a URL.  HTTPAPI's encoder has done it's job!  Hurray!  So it's sent 
over the network to the computer on the opposite end.

That computer receives the URL encoded string, and decodes it (since XML 
parsers have no concept of URL encoding.) and the result is this once again:

    <DESC>GENERAL & ACCESSORIES</DESC>

Now the XML parser tries to parse it... and fails because it's not a 
well-formed (valid) XML document.  You can't have a lone & in the middle 
of chardata in XML.

That's what's happening... and HTTPAPI is doing exactly what it should, 
it's encoding your data so it's valid for HTTP.  HTTPAPI doesn't even 
know that your data is XML!  And even if it somehow was smart enough to 
figure that out for you, how would it know whether & was intended to be 
the literal & character, or whether it's intended to be an entity?  You 
seem to think it can read your mind, but it cannot.

For the process to work, you need to start with a _valid_ XML document. 
  In other words, you need to start with this:

    <DESC>GENERAL &amp; ACCESSORIES</DESC>

After URL encoding, it will look like this:

    instring=%3cDESC%3eGENERAL%20%26amp;%20ACCESSORIES%3c/DESC%3e

Now it goes over HTTP protocol successfully... when it's received, the 
receiving HTTP server decodes the URL encoding back to this:

    <DESC>GENERAL &amp; ACCESSORIES</DESC>

And the XML parser will now parse it successfully.


On 10/8/2010 1:50 PM, Natarajan Palani wrote:
>
>     Hi Scott,
>
>     I am using http_url_post_raw. It works like a charm.
>
>     The input to the server is in XML. During our testing we came across a
>     scenario where the XML data had and "&" char for an element
>     (<DESC>GENERAL&  ACCESSORIES</DESC>).
>
>     And we got the error response from the target Server as
>
>     <ERROR DESCRIPTION>Unable to parse input XML entity reference names
>     can  not start with character&apos;&apos; (position: START_TAG seen
>     ...&lt;VALUE&gt;GENERAL&amp; ... @1:2497)</ERROR DESCRIPTION>
>
>     I thought by using WEBFORM_setPtr&  WEBFORM_postData the data gets
>     converted to URL encoded data. But it did not convert my "&" char to
>     &amp;. Is there any reason behind this or am I doing something wrong
>     here. Below is my code for Ur reference:
>
>                 form = WEBFORM_open();
>                 WEBFORM_setPtr(form: 'instring': %addr(@XML_Input) :
>                                %len(%trimR(@XML_Input)) );
>                 WEBFORM_postData(form: postData: postDataSize );
>                 // Post data and get document
>                 rc = http_url_post_raw(%trim(@ServerName)
>                           : postData
>                           : postDataSize
>                           : 1
>                           : %paddr('RECEIVEDATA')
>                           : @HTTPPostTimeout
>                           : *OMIT
>                           : %trim(@URLEncodApp) );
>                 WEBFORM_close(form);
>                 if rc<>  1;
>                   DS_HTTP_StatusData.HTTPErrDesc = 'HTTP Post Failed. ' +
>                                                     http_error();
>                   DS_HTTP_StatusData.ReturnCd    = HTTPPostFailed;
>                 Else;
>                   callp Translate(retlen: retdata: 'QTCPEBC');
>                   DS_HTTP_StatusData.XML_Response = %trim(retdata);
>                 endif;
>
>     Please let me know.
>
>     Thanks for your help.
>
>     Raj
>
>
>
>
> -----------------------------------------------------------------------
> This is the FTPAPI mailing list.  To unsubscribe, please go to:
> http://www.scottklement.com/mailman/listinfo/ftpapi
> -----------------------------------------------------------------------

-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------