Decoding Base64 data does not return the original string value

Discussions relating to writing software in ILE RPG (RPG IV). This includes both fixed and free format RPG.
Post Reply
Roldao
Posts: 2
Joined: Mon May 16, 2022 4:59 pm

Decoding Base64 data does not return the original string value

Post by Roldao »

What I'm supposed to do is read a base64 encoded value

eyJzb3VyY2UiOiJDQ0RWIiwidHJhY2VpZCI6IjNiM2MxZDZjZTU2Y2QxZTBkNThlNDA4NDY2YWVkZWE3In0=

from a field in a database file and decode it to obtain the original JSON statement

Code: Select all

{"source":"CCDV","traceid":"3b3c1d6ce56cd1e0d58e408466aedea7"}
I tested it in an online converter https://codebeautify.org/base64-to-json-converter and it worked perfectly.

The result is quite different when I try to decode it in my RPG ILE program. I thought: if I encode the original JSON statement and then decode the result i should get the JSON statement again. Well, it doesn't! Why?

I tried treating the data as binary. The base64 encoding is very different from the previous one but when I decode it I get the original JSON statement.
What am I doing wrong?
Here follows the program code. It was compiled for Version 7.3. Thank you for your attention.

Code: Select all

**FREE

DCL-S sCode char(512) inz;
DCL-S uCode uns(10) inz;
DCL-S sDcode char(255) inz;
DCL-S uDcode uns(10) inz;
DCL-S uLen uns(10) inz;

DCL-S sVar char(512) inz;

DCL-S sJSON char(255) inz('{"source":"CCDV","traceid":"3b3c1d6ce56cd1e0d 58e408466aedea7"}');

DCL-PR API_APR_Code64 INT(10) EXTPROC('apr_base64_encode');
xsCode64 char(65535) options(*VARSIZE);
xpOriginal pointer value options(*STRING);
xnCode64 int(10) value;
END-PR;

DCL-PR API_APR_Decode64 INT(10) EXTPROC('apr_base64_decode');
xsPlainText char(65535) options(*VARSIZE);
xpCode64 pointer value options(*STRING);
END-PR;

DCL-PR API_APR_Code64Binary INT(10) EXTPROC('apr_base64_encode_binary');
xsCode64 char(65535) options(*VARSIZE);
xpOriginal pointer value options(*STRING);
xnCode64 int(10) value;
END-PR;

DCL-PR API_APR_Decode64Binary INT(10) EXTPROC('apr_base64_decode_binary');
xsPlainText char(65535) options(*VARSIZE);
xpCode64 pointer value options(*STRING);
END-PR;

sVar = sJSON;
uLen = %LEN(%TRIMR(sJSON));

// sCode = eyJzb3VyY2UiOiJDQ0RWIiwidHJhY2VpZCI6IjNiM2MxZDZjZT U2Y2QxZTBkNThlNDA4NDY2YWVkZWE3In0=
uCode = API_APR_Code64(sCode : %subst(sVar : 1 : uLen) : uLen) - 1;

// sDcode = #?Ë?ÍÊÄÁ???ääàî???ÈÊ/ÄÁÑÀ????Â?Ä?À?ÄÁ??ÄÀ?Á?À??Á??????/ÁÀÁ/??'
uDcode = API_APR_Decode64(sDcode : %subst(sCode : 1 : uCode));

sVar = sJSON;
uLen = %LEN(%TRIMR(sJSON));

// sCode = wH+ilqSZg4V/en/Dw8Tlf2t/o5mBg4WJhH96f/OC84PxhPaDhfX2g4TxhfCE9fiF9PD49Pb2gYWEhYH3f9A
uCode = API_APR_Code64Binary(sCode : %subst(sVar : 1 : uLen) : uLen) - 1;

// sDcode = {"source":"CCDV","traceid":"3b3c1d6ce56cd1e0d58e40 8466aedea7"}
uDcode = API_APR_Decode64Binary(sDcode : %subst(sCode : 1 : uCode));


*INLR = *ON;
jonboy49
Posts: 200
Joined: Wed Jul 28, 2021 8:18 pm

Re: Decoding Base64 data does not return the original string value

Post by jonboy49 »

Difficult to know what you did wrong because you didn't include the code for your custom encode/decode routines!

This works though using Scott's original code from his site.

Code: Select all

**FREE
Ctl-Opt DftActGrp(*No)  BndDir('BASE64V11/BASE64');

/copy base64v11/qrpglesrc,base64_h

DCL-S Encoded     char(510) inz;
DCL-S EncodedLen  uns(10) inz;

DCL-S Decoded     char(255) inz;
DCL-S Decodedlen  uns(10) inz;


DCL-S sJSON varchar(255) inz('{"source":"CCDV","traceid":"3b3c1d6ce56cd1e0d 58e408466aedea7"}');

Dsply (' JSON string is ' + %Char(%Len(sJSON)) + ' long');

EncodedLen = base64_encode( %Addr( sJSON: *Data) : %Len(sJSON)
                          : %Addr( Encoded ) : %Len(Encoded) );

DecodedLen = base64_decode( %Addr( Encoded ) : EncodedLen
                           : %Addr( Decoded ) : %Len(Decoded) );

Dsply ('EncodedLen = ' + %char(EncodedLen) + ' - DecodedLen = ' + %char(DecodedLen));

If sJSON = Decoded;
   Dsply 'That worked out fine';
Else;
   Dsply 'Hmmm -problem - mismatch';
EndIf;
Hopefully you'll be able to see what you did wrong.

*INLR = *ON;
Scott Klement
Site Admin
Posts: 635
Joined: Sun Jul 04, 2021 5:12 am

Re: Decoding Base64 data does not return the original string value

Post by Scott Klement »

Jon,

He's using the Apache Portable Runtime (APR) routines that are provided with the operating system. Here is the documentation from Apache:
https://apr.apache.org/docs/apr-util/0. ... ase64.html

IBM provides these routines in the service program QSYSDIR/QAXIS10HT which is installed with 5770-SS1 option 3 (Extended Base Directory Support).
Scott Klement
Site Admin
Posts: 635
Joined: Sun Jul 04, 2021 5:12 am

Re: Decoding Base64 data does not return the original string value

Post by Scott Klement »

Roldao,

I'm certainly no expert on the APR routines. However, I know that when you call apr_base64_encode, it does not treat your data as binary. It converts it from EBCDIC to what the APR considers "plain text". I'm not 100% sure what that is... but it's either some flavor of ASCII or UTF-8. (I hope it is UTF-8!) Once it has converted it, only then does it base64 encode it. So what you will have is not encoded EBCDIC, it's encoded UTF-8.

When you decode it, it doesn't know that it was originally EBCDIC. All it knows is what it currently is -- so it decodes it to UTF-8 rather than EBCDIC.

Since RPG supports UTF-8 natively, I would recommend that you use that to convert the data back to EBCDIC. For example:

Code: Select all

**FREE
DCL-S sCode char(512) inz;
DCL-S uCode uns(10) inz;
DCL-S sDcode_utf8 char(255) inz ccsid(*utf8);
DCL-S sDcode char(255) inz;
DCL-S uDcode uns(10) inz;
DCL-S uLen uns(10) inz;

DCL-S sVar char(512) inz;

DCL-S sJSON char(255) inz('{"source":"CCDV","traceid":"3b3c1d6ce56cd1e0d 58e408466aedea7"}');

DCL-PR API_APR_Code64 INT(10) EXTPROC('apr_base64_encode');
xsCode64 char(65535) options(*VARSIZE);
xpOriginal pointer value options(*STRING);
xnCode64 int(10) value;
END-PR;

DCL-PR API_APR_Decode64Binary INT(10) EXTPROC('apr_base64_decode_binary');
xsPlainText char(65535) options(*VARSIZE) ccsid(*utf8);
xpCode64 pointer value options(*STRING);
END-PR;

sVar = sJSON;
uLen = %LEN(%TRIMR(sJSON));

// sCode = eyJzb3VyY2UiOiJDQ0RWIiwidHJhY2VpZCI6IjNiM2MxZDZjZT U2Y2QxZTBkNThlNDA4NDY2YWVkZWE3In0=
uCode = API_APR_Code64(sCode : %subst(sVar : 1 : uLen) : uLen) - 1;

// sDcode = {"source":"CCDV","traceid":"3b3c1d6ce56cd1e0d58e40 8466aedea7"}
uDcode = API_APR_Decode64Binary(sDcode_utf8 : %subst(sCode : 1 : uCode));
sdCode = %subst(sDcode_utf8:1:uDcode);

*INLR = *ON;
 
jonboy49
Posts: 200
Joined: Wed Jul 28, 2021 8:18 pm

Re: Decoding Base64 data does not return the original string value

Post by jonboy49 »

Scott Klement wrote: Tue May 17, 2022 7:18 am Jon,

He's using the Apache Portable Runtime …
Thanks Scott - that’ll teach me assume that on your forums people only ask about your routines!

I’d not heard of the APR ones.
Roldao
Posts: 2
Joined: Mon May 16, 2022 4:59 pm

Re: Decoding Base64 data does not return the original string value

Post by Roldao »

First of all, thank you very much for your replies.
Your code is a solution to my problem, Scott. I noticed you decoded the base64 string as binary data. I tried it invoking the non-binary procedure API_APR_Decode64 and it worked as well. Is there any advantage in decoding it as binary over this last alternative?

One more question if you please: how would I code this for version 7.1 which doesn't allow the keyword CCSID in the definition of character variables?
Scott Klement
Site Admin
Posts: 635
Joined: Sun Jul 04, 2021 5:12 am

Re: Decoding Base64 data does not return the original string value

Post by Scott Klement »

Roldao wrote: Wed May 18, 2022 10:52 am Your code is a solution to my problem, Scott. I noticed you decoded the base64 string as binary data. I tried it invoking the non-binary procedure API_APR_Decode64 and it worked as well. Is there any advantage in decoding it as binary over this last alternative?
Personally, I find it less confusing to use binary mode because I can see clearly both of the steps (1) decoding and (2) converting from UTF-8 to EBCDIC.
Roldao wrote: Wed May 18, 2022 10:52 am One more question if you please: how would I code this for version 7.1 which doesn't allow the keyword CCSID in the definition of character variables?
You could use an IBM API such as iconv() to translate to/from UTF-8.

But... keep in mind that 7.1 is no longer supported, so anyone still using that is putting their business at risk. (Also, 7.2 is also no longer supported -- so anyone still running 7.1 is VERY much beyond support.)
Post Reply